Creating hooks and scripts with mod_lua

Introduction ¶

What is mod_lua

Stuff about what mod_lua is goes here.

What we will be discussing in this document

This document will discuss several cases where mod_lua can be used to either ease up a phase of the request processing or create more transparency in the logic behind a decision made in a phase.

Prerequisites

First and foremost, you are expected to have a basic knowledge of how the Lua programming language works. In most cases, we will try to be as pedagogical as possible and link to documents describing the functions used in the examples, but there are also many cases where it is necessary to either just assume that "it works" or do some digging yourself into what the hows and whys of various function calls.

Optimizing mod_lua for production servers ¶

Setting a scope for Lua states

Setting the right LuaScope setting for your Lua scripts can be essential to your server's performance. By default, the scope is set to once, which means that every call to a Lua script will spawn a new Lua state that handles that script and is destroyed immediately after. This option keeps the memory footprint of mod_lua low, but also affects the processing speed of a request. If you have the memory to spare, you can set the scope to thread, which will make mod_lua spawn a Lua state that lasts the entirety of a thread's lifetime, speeding up request processing by 2-3 times. Since mod_lua will create a state for each script, this may be an expensive move, memory-wise, so to compromise between speed and memory usage, you can choose the server option to create a pool of Lua states to be used. Each request for a Lua script or a hook function will then acquire a state from the pool and release it back when it's done using it, allowing you to still gain a significant performance increase, while keeping your memory footprint low. Some examples of possible settings are:

LuaScope once
LuaScope thread
LuaScope server 5 40

As a general rule of thumb: If your server has none to low usage, use once or request, if your server has low to medium usage, use the server pool, and if it has high usage, use the thread setting. As your server's load increases, so will the number of states being actively used, and having your scope set to once/request/conn will stop being beneficial to your memory footprint.

Note: The min and max settings for the server scope denotes the minimum and maximum states to keep in a pool per server process, so keep this below your ThreadsPerChild limit.

Using code caching

By default, mod_lua stats each Lua script to determine whether a reload (and thus, a re-interpretation and re-compilation) of a script is required. This is managed through the LuaCodeCache directive. If you are running your scripts on a production server, and you do not need to update them regularly, it may be advantageous to set this directive to the forever value, which will cause mod_lua to skip the stat process and always reuse the compiled byte-code from the first access to the script, thus speeding up the processing. For Lua hooks, this can prove to increase performance, while for scripts handled by the lua-script handler, the increase in performance may be negligible, as files httpd will stat the files regardless.

Keeping the scope clean

For maximum performance, it is generally recommended that any initialization of libraries, constants and master tables be kept outside the handle's scope:

--[[ This is good practice ]]--
require "string"
require "someLibrary"
local masterTable = {}
local constant = "Foo bar baz"

function handle(r)
    do_stuff()
end

--[[ This is bad practice ]]--
require "string"

function handle(r)
    require "someLibrary"
    local masterTable = {}
    local constant = "Foo bar baz"
    do_stuff()
end

Example 1: A basic remapping module ¶

These first examples show how mod_lua can be used to rewrite URIs in the same way that one could do using Alias or RewriteRule, but with more clarity on how the decision-making takes place, as well as allowing for more complex decisions than would otherwise be allowed with said directives.

LuaHookTranslateName /path/too/foo.lua remap

--[[
    Simple remap example.
    This example will rewrite /foo/test.bar to the physical file
    /internal/test, somewhat like how mod_alias works.
]]--

function remap(r)
    -- Test if the URI matches our criteria
    local barFile =  r.uri:match("/foo/([a-zA-Z0-9]+)%.bar")
    if barFile then
        r.filename = "/internal/" .. barFile
    end
    return apache2.OK
end

--[[
    Advanced remap example.
    This example will evaluate some conditions, and based on that,
    remap a file to one of two destinations, using a rewrite map.
    This is similar to mixing AliasMatch and ProxyPass, but
    without them clashing in any way. Assuming we are on example.com, then:

    http://example.com/photos/test.png will be rewritten as /uploads/www/test.png
    http://example.com/ext/foo.html will be proxied to http://www.external.com/foo.html
    URIs that do not match, will be served by their respective default handlers
]]--

local map = {
      photos = {
                   source = [[^/photos/(.+)\.png$]],
                   destination = [[/uploads/www/$1.png]],
                   proxy = false
                },
      externals = {
                   source = [[^/ext/(.*)$]],
                   destination = [[http://www.external.com/$1]],
                   proxy = true
                }
}

function interpolateString(s,v)
    return s:gsub("%$(%d+)", function(a) return v[tonumber(a)] end)
end

function remap(r)
    -- browse through the rewrite map
    for key, entry in pairs(map) do
        -- Match source regex against URI
        local match = r:regex(entry.source, r.uri) then
        if match and match[0] then
            r.filename = interpolateString(entry.destination, match)
            -- Is this a proxied remap?
            if entry.proxy then
                r.handler = "proxy-server" -- tell mod_proxy to handle this
                r.proxyreq = apache2.PROXYREQ_REVERSE -- We'll want to do a reverse proxy
                r.filename = "proxy:" .. r.filename -- Add the proxy scheme to the destination
            end
            return apache2.OK
        end
    end
    return apache2.DECLINED
end

bla bla

Example 2: Mass virtual hosting ¶

As with simple and advanced rewriting, you can use mod_lua for dynamically assigning a hostname to a specific document root, much like mod_vhost_alias does, but with more control over what goes where. This could be as simple as a table holding the information about which host goes into which folder, or more advanced, using a database holding the document roots of each hostname.

LuaHookTranslateName /path/too/foo.lua mass_vhost

--[[
    Simple mass vhost script
    This example will check a map for a virtual host and rewrite filename and
    document root accordingly.
]]--

local vhosts = {
    { domain = "example.com", home = "/www/example.com" },
    { domain = "example.org", home = "/nfs/ext1/example.org" }
}

function mass_vhost(r)
    -- Match against our hostname
    for key, entry in pairs(vhosts) do
        -- match against either host or *.host:
        if apache2.strcmp_match(r.hostname, entry.domain) or
           apache2.strcmp_match(r.hostname, "*." .. entry.domain) then
            -- If it matches, rewrite filename and set document root
            local filename = r.filename:sub(r.document_root:len()+1)
            r.filename = entry.home .. filename
            apahce2.set_document_root(entry.home)
            return apache2.OK
        end
    end
    return apache2.DECLINED
end

--[[
    Advanced mass virtual hosting
    This example will query a database for vhost entries and save them for
    60 seconds before checking for updates. For best performance, such scripts
    should generally be run with LuaScope set to 'thread' or 'server'
]]--

local cached_vhosts = {}
local timeout = 60

-- Function for querying the database for saved vhost entries
function query_vhosts(r)
    local host = r.hostname
    if not cached_vhosts[host] or (cached_vhosts[host] and cached_vhosts[host].updated < os.time() - timeout) then
        local db,err = ap.dbopen(r,"mod_dbd")
        local _host = db:escape(r,host)
        local res, err = db:query(r, ("SELECT `destination` FROM `vhosts` WHERE `hostname` = '%s' LIMIT 1"):format(_host) )
        if res and #res == 1 then
            cached_vhosts[host] = { updated = os.time(), destination = res[1][1] }
        else
            cached_vhosts[host] = { updated = os.time(), destination = nil } -- don't re-query whenever there's no result, wait a while.
        end
        db:close()
    end
    if cached_vhosts[host] then
        return cached_vhosts[host].destination
    else
        return nil
    end
end

function mass_vhost(r)
    -- Check whether the hostname is in our database
    local destination = query_vhosts(r)
    if destination then
        -- If found, rewrite and change document root
        local filename = r.filename:sub(r.document_root:len()+1)
        r.filename = destination .. filename
        ap.set_document_root(r,destination)
        return apache2.OK
    end
    return apache2.DECLINED
end

Example 3: A basic authorization hook ¶

With the authorization hooks, you can add custom auth phases to your request processing, allowing you to either add new requirements that were not previously supported by httpd, or tweaking existing ones to accommodate your needs.

LuaHookAuthChecker /path/too/foo.lua check_auth

--[[
     A simple authentication hook that checks a table containing usernames and
     passwords of two accounts.
]]--
local accounts = {
    bob  = 'somePassword',
    jane = 'Iloveponies'
}

-- Function for parsing the Authorization header into a username and a password
function parse_auth(str)
    local user,pass = nil, nil
    if str and str:len() > 0 then
        str = apache2.base64_decode(auth):sub(7));
        user, pass = auth:match("([^:]+)%:([^:]+)")
    end
    return user, pass
end

-- The authentication hook
function check_auth(r)
    local user, pass = parse_auth(r.headers_in['Authorization'])
    local authenticated = false
    if user and pass then
        if accounts[user] and accounts[user] == pass then
            authenticated = true
            r.user = user
        end
    end
    r.headers_out["WWW-Authenticate"] = 'Basic realm="Super secret zone"'
    if not authenticated then
        return 401
    else
        return apache2.OK
    end
end

--[[
     An advanced authentication checker with a database backend,
     caching account entries for 1 minute
]]--

local timeout = 60 -- Set account info to be refreshed every minute
local accounts = {}

-- Function for parsing the Authorization header into a username and a password
function parse_auth(str)
    local user,pass = nil, nil
    if str and str:len() > 0 then
        str = apache2.base64_decode(auth):sub(7));
        user, pass = auth:match("([^:]+)%:([^:]+)")
    end
    return user, pass
end

-- Function for querying the database for the account's password (stored as a salted SHA-1 hash)
function fetch_password(user)
    if not accounts[user] or (accounts[user] and accounts[user].updated < os.time() - timeout) then
        local db = apache2.dbopen(r, "mod_dbd")
        local usr = db:escape(user)
        local res, err = db:query( ("SELECT `password` FROM `accounts` WHERE `user` = '%s' LIMIT 1"):format(usr) )
        if res and #res == 1 then
            accounts[user] = { updated = os.time(), password = res[1][1] }
        else
            accounts[user] = nil
        end
        db:close()
    end
    if accounts[user] then
        return accounts[user].password
    else
        return nil
    end
end

-- The authentication hook
function check_auth(r)
    local user, pass = parse_auth(r.headers_in['Authorization'])
    local authenticated = false
    if user and pass then
        pass = apache2.sha1("addSomeSalt" .. pass)
        local stored_pass = fetch_password(user)
        if stored_pass and pass == stored_pass then
            authenticated = true
            r.user = user
        end
    end
    r.headers_out["WWW-Authenticate"] = 'Basic realm="Super secret zone"'
    if not authenticated then
        return 401
    else
        return apache2.OK
    end
end

Example 4: Authorization using LuaAuthzProvider ¶

If you require even more advanced control over your authorization phases, you can add custom authz providers to help you manage your server. The example below shows you how you can split a single htpasswd file into groups with different permissions:

LuaAuthzProvider rights /path/to/lua/script.lua rights_handler
<Directory "/www/private">
    Require rights member
</Directory>
<Directory "/www/admin">
    Require rights admin
</Directory>

--[[
     This script has two user groups; members and admins, and whichever
     is referred to by the "Require rights" directive is checked to see
     if the authenticated user belongs to this group.
]]--

local members = { "rbowen", "humbedooh", "igalic", "covener" }
local admins = { "humbedooh" }

function rights_handler(r, what)
    if r.user == nil then
        return apache2.AUTHZ_AUTHZ_DENIED_NO_USER
    end
    if what == "member" then
        for k, v in pairs(members) do
            if r.user == v then
                return apache2.AUTHZ_GRANTED
            end
        end
    elseif what == "admin" then
        for k, v in pairs(admins) do
            if r.user == v then
                return apache2.AUTHZ_GRANTED
            end
        end
    end
    return apache2.AUTHZ_DENIED
end

Example 5: A rudimentary load balancer ¶

This is an example of how you can create a load balancing mechanism. In this example, we will be setting/getting the number of requests served by each backend using IVM variables, and preferring the backend with least requests served in total:

LuaHookTranslateName /path/to/script.lua proxy_handler

--[[
     This script uses a basic IVM table to determine where to
     send the request.
]]--

local backends = {
  "http://backend1.foo.com/",
  "http://backend2.foo.com/",
  "http://backend3.foo.com/"
}

function pick_backend(r)
    local chosen_backend = 1 -- default to backend1
    local lowest_count = nil
    for i = 1, #backends, 1 do -- Loop through all backends
        local count = r:ivm_get("proxy_request_count_" .. i)
        if not count then -- If this backend hasn't been used at all, prefer it
            chosen_backend = i
            lowest_count = 0
            break
        end
        if not lowest_count or lowest_count > count then -- If this backend has had less requests, pick it for now
            chosen_backend = i
            lowest_count = count
        end
    end
    lowest_count = lowest_count + 1
    r:ivm_set("proxy_request_count_" .. chosen_backend, lowest_count)
    return chosen_backend
end

function proxy_handler(r)
    local backend = pick_backend(r) -- Pick a backend based on no. of requests served
    r.handler  = "proxy-server"
    r.proxyreq = apache2.PROXYREQ_REVERSE
    r.filename = "proxy:" .. backends[backend] .. r.uri
    return apache2.DECLINED -- let the proxy handler do this instead
end

Example 6: Overlays using LuaMapHandler ¶

Coming soon!

LuaMapHandler ^/portal/([a-z]+)/   /path/to/lua/script.lua handle_$1

Example 6: Basic Lua scripts ¶

Also coming soon