Developer Creating hooks and scripts with mod_lua

This document expands on the mod_lua documentation and explores additional ways of using mod_lua for writing hooks and scripts.

 mod_lua Developing modules for Apache 2.4 Request Processing in Apache 2.4 Apache 2.x Hook Functions
Introduction
What is mod_lua

Stuff about what mod_lua is goes here.

What we will be discussing in this document

This document will discuss several cases where mod_lua can be used to either ease up a phase of the request processing or create more transparency in the logic behind a decision made in a phase.

Prerequisites

First and foremost, you are expected to have a basic knowledge of how the Lua programming language works. In most cases, we will try to be as pedagogical as possible and link to documents describing the functions used in the examples, but there are also many cases where it is necessary to either just assume that "it works" or do some digging yourself into what the hows and whys of various function calls.

Optimizing mod_lua for production servers
Setting a scope for Lua states

Setting the right LuaScope setting for your Lua scripts can be essential to your server's performance. By default, the scope is set to once, which means that every call to a Lua script will spawn a new Lua state that handles that script and is destroyed immediately after. This option keeps the memory footprint of mod_lua low, but also affects the processing speed of a request. If you have the memory to spare, you can set the scope to thread, which will make mod_lua spawn a Lua state that lasts the entirety of a thread's lifetime, speeding up request processing by 2-3 times. Since mod_lua will create a state for each script, this may be an expensive move, memory-wise, so to compromise between speed and memory usage, you can choose the server option to create a pool of Lua states to be used. Each request for a Lua script or a hook function will then acquire a state from the pool and release it back when it's done using it, allowing you to still gain a significant performance increase, while keeping your memory footprint low. Some examples of possible settings are:

LuaScope once LuaScope thread LuaScope server 5 40

As a general rule of thumb: If your server has none to low usage, use once or request, if your server has low to medium usage, use the server pool, and if it has high usage, use the thread setting. As your server's load increases, so will the number of states being actively used, and having your scope set to once/request/conn will stop being beneficial to your memory footprint.

Note: The min and max settings for the server scope denotes the minimum and maximum states to keep in a pool per server process, so keep this below your ThreadsPerChild limit.

Using code caching

By default, mod_lua stats each Lua script to determine whether a reload (and thus, a re-interpretation and re-compilation) of a script is required. This is managed through the LuaCodeCache directive. If you are running your scripts on a production server, and you do not need to update them regularly, it may be advantageous to set this directive to the forever value, which will cause mod_lua to skip the stat process and always reuse the compiled byte-code from the first access to the script, thus speeding up the processing. For Lua hooks, this can prove to increase peformance, while for scripts handled by the lua-script handler, the increase in performance may be negligible, as files httpd will stat the files regardless.

Keeping the scope clean

For maximum performance, it is generally recommended that any initialization of libraries, constants and master tables be kept outside the handle's scope:

--[[ This is good practice ]]-- require "string" require "someLibrary" local masterTable = {} local constant = "Foo bar baz" function handle(r) do_stuff() end --[[ This is bad practice ]]-- require "string" function handle(r) require "someLibrary" local masterTable = {} local constant = "Foo bar baz" do_stuff() end
Example 1: A basic remapping module

These first examples show how mod_lua can be used to rewrite URIs in the same way that one could do using Alias or RewriteRule, but with more clarity on how the decision-making takes place, as well as allowing for more complex decisions than would otherwise be allowed with said directives.

LuaHookTranslateName /path/too/foo.lua remap --[[ Simple remap example. This example will rewrite /foo/test.bar to the physical file /internal/test, somewhat like how mod_alias works. ]]-- function remap(r) -- Test if the URI matches our criteria local barFile = r.uri:match("/foo/([a-zA-Z0-9]+)%.bar") if barFile then r.filename = "/internal/" .. barFile end return apache2.OK end --[[ Advanced remap example. This example will evaluate some conditions, and based on that, remap a file to one of two destinations, using a rewrite map. This is similar to mixing AliasMatch and ProxyPass, but without them clashing in any way. Assuming we are on example.com, then: http://example.com/photos/test.png will be rewritten as /uploads/www/test.png http://example.com/ext/foo.html will be proxied to http://www.external.com/foo.html URIs that do not match, will be served by their respective default handlers ]]-- local map = { photos = { source = [[^/photos/(.+)\.png$]], destination = [[/uploads/www/$1.png]], proxy = false }, externals = { source = [[^/ext/(.*)$]], destination = [[http://www.external.com/$1]], proxy = true } } function interpolateString(s,v) return s:gsub("%$(%d+)", function(a) return v[tonumber(a)] end) end function remap(r) -- browse through the rewrite map for key, entry in pairs(map) do -- Match source regex against URI local match = r:regex(entry.source, r.uri) then if match and match[0] then r.filename = interpolateString(entry.destination, match) -- Is this a proxied remap? if entry.proxy then r.handler = "proxy-server" -- tell mod_proxy to handle this r.proxyreq = apache2.PROXYREQ_REVERSE -- We'll want to do a reverse proxy r.filename = "proxy:" .. r.filename -- Add the proxy scheme to the destination end return apache2.OK end end return apache2.DECLINED end

bla bla

Example 2: Mass virtual hosting

As with simple and advanced rewriting, you can use mod_lua for dynamically assigning a hostname to a specific document root, much like mod_vhost_alias does, but with more control over what goes where. This could be as simple as a table holding the information about which host goes into which folder, or more advanced, using a database holding the document roots of each hostname.

LuaHookTranslateName /path/too/foo.lua mass_vhost --[[ Simple mass vhost script This example will check a map for a virtual host and rewrite filename and document root accordingly. ]]-- local vhosts = { { domain = "example.com", home = "/www/example.com" }, { domain = "example.org", home = "/nfs/ext1/example.org" } } function mass_vhost(r) -- Match against our hostname for key, entry in pairs(vhosts) do -- match against either host or *.host: if apache2.strcmp_match(r.hostname, entry.domain) or apache2.strcmp_match(r.hostname, "*." .. entry.domain) then -- If it matches, rewrite filename and set document root local filename = r.filename:sub(r.document_root:len()+1) r.filename = entry.home .. filename apahce2.set_document_root(entry.home) return apache2.OK end end return apache2.DECLINED end --[[ Advanced mass virtual hosting This example will query a database for vhost entries and save them for 60 seconds before checking for updates. For best performance, such scripts should generally be run with LuaScope set to 'thread' or 'server' ]]-- local cached_vhosts = {} local timeout = 60 -- Function for querying the database for saved vhost entries function query_vhosts(r) local host = r.hostname if not cached_vhosts[host] or (cached_vhosts[host] and cached_vhosts[host].updated < os.time() - timeout) then local db,err = ap.dbopen(r,"mod_dbd") local _host = db:escape(r,host) local res, err = db:query(r, ("SELECT `destination` FROM `vhosts` WHERE `hostname` = '%s' LIMIT 1"):format(_host) ) if res and #res == 1 then cached_vhosts[host] = { updated = os.time(), destination = res[1][1] } else cached_vhosts[host] = { updated = os.time(), destination = nil } -- don't re-query whenever there's no result, wait a while. end db:close() end if cached_vhosts[host] then return cached_vhosts[host].destination else return nil end end function mass_vhost(r) -- Check whether the hostname is in our database local destination = query_vhosts(r) if destination then -- If found, rewrite and change document root local filename = r.filename:sub(r.document_root:len()+1) r.filename = destination .. filename ap.set_document_root(r,destination) return apache2.OK end return apache2.DECLINED end

Example 3: A basic authorization hook

With the authorization hooks, you can add custom auth phases to your request processing, allowing you to either add new requirements that were not previously supported by httpd, or tweaking existing ones to accommodate your needs.

LuaHookAuthChecker /path/too/foo.lua check_auth --[[ A simple authentication hook that checks a table containing usernames and passwords of two accounts. ]]-- local accounts = { bob = 'somePassword', jane = 'Iloveponies' } -- Function for parsing the Authorization header into a username and a password function parse_auth(str) local user,pass = nil, nil if str and str:len() > 0 then str = apache2.base64_decode(auth):sub(7)); user, pass = auth:match("([^:]+)%:([^:]+)") end return user, pass end -- The authentication hook function check_auth(r) local user, pass = parse_auth(r.headers_in['Authorization']) local authenticated = false if user and pass then if accounts[user] and accounts[user] == pass then authenticated = true r.user = user end end r.headers_out["WWW-Authenticate"] = 'Basic realm="Super secret zone"' if not authenticated then return 401 else return apache2.OK end end --[[ An advanced authentication checker with a database backend, caching account entries for 1 minute ]]-- local timeout = 60 -- Set account info to be refreshed every minute local accounts = {} -- Function for parsing the Authorization header into a username and a password function parse_auth(str) local user,pass = nil, nil if str and str:len() > 0 then str = apache2.base64_decode(auth):sub(7)); user, pass = auth:match("([^:]+)%:([^:]+)") end return user, pass end -- Function for querying the database for the account's password (stored as a salted SHA-1 hash) function fetch_password(user) if not accounts[user] or (accounts[user] and accounts[user].updated < os.time() - timeout) then local db = apache2.dbopen(r, "mod_dbd") local usr = db:escape(user) local res, err = db:query( ("SELECT `password` FROM `accounts` WHERE `user` = '%s' LIMIT 1"):format(usr) ) if res and #res == 1 then accounts[user] = { updated = os.time(), password = res[1][1] } else accounts[user] = nil end db:close() end if accounts[user] then return accounts[user].password else return nil end end -- The authentication hook function check_auth(r) local user, pass = parse_auth(r.headers_in['Authorization']) local authenticated = false if user and pass then pass = apache2.sha1("addSomeSalt" .. pass) local stored_pass = fetch_password(user) if stored_pass and pass == stored_pass then authenticated = true r.user = user end end r.headers_out["WWW-Authenticate"] = 'Basic realm="Super secret zone"' if not authenticated then return 401 else return apache2.OK end end
Example 4: Authorization using LuaAuthzProvider

If you require even more advanced control over your authorization phases, you can add custom authz providers to help you manage your server. The example below shows you how you can split a single htpasswd file into groups with different permissions:

LuaAuthzProvider rights /path/to/lua/script.lua rights_handler <Directory "/www/private"> Require rights member </Directory> <Directory "/www/admin"> Require rights admin </Directory> --[[ This script has two user groups; members and admins, and whichever is referred to by the "Require rights" directive is checked to see if the authenticated user belongs to this group. ]]-- local members = { "rbowen", "humbedooh", "igalic", "covener" } local admins = { "humbedooh" } function rights_handler(r, what) if r.user == nil then return apache2.AUTHZ_AUTHZ_DENIED_NO_USER end if what == "member" then for k, v in pairs(members) do if r.user == v then return apache2.AUTHZ_GRANTED end end elseif what == "admin" then for k, v in pairs(admins) do if r.user == v then return apache2.AUTHZ_GRANTED end end end return apache2.AUTHZ_DENIED end
Example 5: A rudimentary load balancer

This is an example of how you can create a load balancing mechanism. In this example, we will be setting/getting the number of requests served by each backend using IVM variables, and preferring the backend with least requests served in total:

LuaHookTranslateName /path/to/script.lua proxy_handler --[[ This script uses a basic IVM table to determine where to send the request. ]]-- local backends = { "http://backend1.foo.com/", "http://backend2.foo.com/", "http://backend3.foo.com/" } function pick_backend(r) local chosen_backend = 1 -- default to backend1 local lowest_count = nil for i = 1, #backends, 1 do -- Loop through all backends local count = r:ivm_get("proxy_request_count_" .. i) if not count then -- If this backend hasn't been used at all, prefer it chosen_backend = i lowest_count = 0 break end if not lowest_count or lowest_count > count then -- If this backend has had less requests, pick it for now chosen_backend = i lowest_count = count end end lowest_count = lowest_count + 1 r:ivm_set("proxy_request_count_" .. chosen_backend, lowest_count) return chosen_backend end function proxy_handler(r) local backend = pick_backend(r) -- Pick a backend based on no. of requests served r.handler = "proxy-server" r.proxyreq = apache2.PROXYREQ_REVERSE r.filename = "proxy:" .. backends[backend] .. r.uri return apache2.DECLINED -- let the proxy handler do this instead end
Example 6: Overlays using LuaMapHandler

Coming soon!

LuaMapHandler ^/portal/([a-z]+)/ /path/to/lua/script.lua handle_$1
Example 6: Basic Lua scripts

Also coming soon