mod_policy HTTP protocol compliance enforcement. Extension mod_policy.c policy_module

The HTTP protocol recommends that clients should be "liberal in what they accept", and servers "strict with what they send". In some cases it can be difficult to detect when a server or an application has been misconfigured, is serving uncacheable content or is behaving suboptimally, as an HTTP client might be compensating for the server. These problems can potentially lead to excessive bandwidth consumption, or a server outage under load.

The mod_policy module consists of a set of filters that test servers for HTTP protocol compliance. These tests allow the server administrator to log violations of, or outright reject responses where certain defined conditions exist.

This could be used as a way to set minimum HTTP protocol compliance criteria for a restful application. Alternatively, a reverse proxy or cache could be configured to protect itself from misconfigured origin servers or unexpectedly uncacheable content, or as a mechanism to detect configuration mistakes within the server itself.

 Filters HTTP Protocol Compliance
Actions

If a policy is violated, one of the following actions can be taken:

ignore
The policy check will be ignored for the given URL space, even if the filter is present.
log
The policy check will be executed, and if a violation is detected a warning will be logged to the server error_log, and a Warning header added to the response for the benefit of the client.
enforce
The policy check will be executed, and if a violation is detected an error will be logged to the server error_log, a Warning header added to the response, and a 502 Bad Gateway will be returned to the client. Optional links to explanatory documentation can be added to each error message, detailing the origin of each policy.

It is also possible to selectively disable all policies for a given URL space, should the need arise, using the PolicyFilter directive.

Alternatively, the PolicyEnvironment directive can be used to specify an environment variable, which if present, will cause the policies to be selectively downgraded or bypassed.

Policy Tests

The following policy filters are available:

POLICY_TYPE : Enforce valid content types
Content types that are syntactically invalid or blank can be detected and the request rejected. Types can be restricted to a specific list containing optional wildcards ? and *.
POLICY_LENGTH : Enforce the presence of a Content-Length
The length of responses can be specified in one of three ways, by specifying an explicit length in advance, using chunked encoding to set the length, or by setting no length at all and terminating the request when complete. The absence of a specific content length can affect the cacheability of the response, and prevents the use of keepalive during HTTP/1.0 requests. This policy enforces the presence of an explicit content length on the response.
POLICY_KEEPALIVE : Enforce the option to keepalive
Less restrictive than the POLICY_LENGTH test, this policy enforces the possibility that the response can be kept alive. If the response doesn't have a protocol defined zero length, and the response isn't already an error, and the response has neither a Content-Length or is declared HTTP/1.1 and lacks Content-Encoding: chunked, then this response will be rejected.
POLICY_VARY : Enforce the absence of certain headers within Vary headers
If the Vary header contains any of the headers specified, this policy will reject the request. The typical case is the presence of the User-Agent within Vary, which is likely to cause a denial of service condition to a cache.
POLICY_VALIDATION: Enforce the presence of Etag and/or Last-Modified
The ability for a cache to determine whether a cached entity can be refreshed is dependent on whether a valid Etag and/or Last-Modified header is present to revalidate against. The absence of both headers, or the invalid syntax of a header will cause this policy to be rejected.
POLICY_CONDITIONAL: Enforce correct operation of conditional requests
When conditional headers are present in the request, a server should respond with a 304 Not Modified or 412 Precondition Failed response where appropriate. A server may ignore conditional headers, and this affects the efficiency of the HTTP caching mechanism. This policy rejects requests where a conditional header is present, and a 304 or 412 response code was expected, but a 2xx response was seen instead.
POLICY_NOCACHE : Enforce cacheable responses
When a response is encountered that declares itself explicitly uncacheable, the request is rejected. A response is considered uncacheable if it specifies any of the following:
  • Cache-Control: no-cache
  • Pragma: no-cache
  • Cache-Control: no-store
  • Cache-Control: private
POLICY_MAXAGE : Enforce a minimum maxage
When a response is encountered where the freshness lifetime is less than the given value, or the freshness lifetime is heuristic, the request is rejected. A response is checked in the following order:
  • If s-maxage is present but too small; or
  • If max-age is present but too small; or
  • If Expires is present and invalid; or
  • Date is present and invalid; or
  • Expires minus Date is too small; or
  • No s-maxage, maxage, or Expires/Date declared at all
POLICY_VERSION : Enforce a minimum HTTP version within a request
When a request is encountered with an HTTP version number less than the required minimum version, the request is rejected. The following version numbers are recognised:
  • HTTP/1.1
  • HTTP/1.0
  • HTTP/0.9
Example Configuration

A typical configuration protecting a server serving static content might be as follows:

<Location /> SetOutputFilter POLICY_TYPE;POLICY_LENGTH;POLICY_KEEPALIVE;POLICY_VARY;POLICY_VALIDATION; \ POLICY_CONDITIONAL;POLICY_NOCACHE;POLICY_MAXAGE;POLICY_VERSION # content type must be present and valid, but can be anything PolicyType enforce */* # reject if no explicitly declared content length PolicyLength enforce # covered by the policy length filter PolicyKeepalive ignore # reject if User-Agent appears within Vary headers PolicyVary enforce User-Agent # we want to enforce validation PolicyValidation enforce # non-functional conditional responses should be rejected PolicyConditional enforce # no-cache responses should be rejected PolicyNocache enforce # maxage must be at least a day PolicyMaxage enforce 86400 # request version can be anything PolicyVersion ignore HTTP/1.1 </Location> # suppress policy protection for server-status <Location /server-status> PolicyFilter off </Location>
PolicyFilter Enable or disable policies for the given URL space. PolicyFilter on|off on server configvirtual host directory PolicyFilter is only available in Apache 2.5.0 and later.

Master switch to enable or disable policies for a given URL space.

Example # enabled by default <Location /> PolicyFilter on </Location> # suppress policy protection for server-status <Location /server-status> PolicyFilter off </Location> PolicyEnvironment Override policies based on an environment variable. PolicyEnvironment variable log-value ignore-value none server configvirtual host directory PolicyEnvironment is only available in Apache 2.5.0 and later.

Downgrade policies to logging only or ignored based on the presence of an environment variable. If the given variable is present and equal to the log-value, enforced policies will be logged instead. If the given variable is present and equal to the ignore-value, all policies will be ignored.

Example # downgrade if POLICY_CONTROL was present PolicyEnvironment POLICY_CONTROL log ignore PolicyConditional Enable the conditional request policy. PolicyConditional ignore|log|enforce ignore server configvirtual host directory PolicyConditional is only available in Apache 2.5.0 and later.

When logged or enforced, a response that should have been conditional but wasn't will be rejected.

Example # non-functional conditional responses should be rejected PolicyConditional enforce PolicyConditionalURL URL describing the conditional request policy. PolicyConditionalURL url none server configvirtual host directory PolicyConditionalURL is only available in Apache 2.5.0 and later.

Specify the URL of the documentation describing the conditional request policy, to appear within error messages.

 PolicyLength Enable the content length policy. PolicyLength ignore|log|enforce ignore server configvirtual host directory PolicyLength is only available in Apache 2.5.0 and later.

When logged or enforced, a response that lacks an explicit Content-Length header will be rejected.

Example # missing Content-Length header should be rejected PolicyLength enforce PolicyLengthURL URL describing the content length policy. PolicyLengthURL url none server configvirtual host directory PolicyLengthURL is only available in Apache 2.5.0 and later.

Specify the URL of the documentation describing the content length policy, to appear within error messages.

 PolicyKeepalive Enable the keepalive policy. PolicyKeepalive ignore|log|enforce ignore server configvirtual host directory PolicyKeepalive is only available in Apache 2.5.0 and later.

When logged or enforced, a response that lacks both an explicit Content-Length header and a Transfer-Encoding of chunked will be rejected.

Example # missing Content-Length or Transfer-Encoding should be rejected PolicyKeepalive enforce PolicyKeepaliveURL URL describing the keepalive policy. PolicyKeepaliveURL url none server configvirtual host directory PolicyKeepaliveURL is only available in Apache 2.5.0 and later.

Specify the URL of the documentation describing the keepalive policy, to appear within error messages.

 PolicyType Enable the content type policy. PolicyType ignore|log|enforce type [ type [ ... ]] ignore server configvirtual host directory PolicyType is only available in Apache 2.5.0 and later.

When logged or enforced, a response that lacks a Content-Type header, where the Content-Type header is malformed, or where the header does not match the given pattern or patterns will be rejected.

Example # enforce json or XML PolicyType enforce application/json text/xml Example # malformed content type should be rejected PolicyType enforce */* PolicyTypeURL URL describing the content type policy. PolicyTypeURL url none server configvirtual host directory PolicyTypeURL is only available in Apache 2.5.0 and later.

Specify the URL of the documentation describing the content type policy, to appear within error messages.

 PolicyVary Enable the Vary policy. PolicyVary ignore|log|enforce header [ header [ ... ]] ignore server configvirtual host directory PolicyVary is only available in Apache 2.5.0 and later.

When logged or enforced, a response that contains a Vary header which in turn contains one of the headers listed, will be rejected.

Example # reject reponses with "User-Agent" listed in the Vary header PolicyVary enforce User-Agent PolicyVaryURL URL describing the content type policy. PolicyVaryURL url none server configvirtual host directory PolicyVaryURL is only available in Apache 2.5.0 and later.

Specify the URL of the documentation describing the vary policy, to appear within error messages.

 PolicyValidation Enable the validation policy. PolicyValidation ignore|log|enforce ignore server configvirtual host directory PolicyValidation is only available in Apache 2.5.0 and later.

When logged or enforced, a response that lacks either a valid ETag header or a Last-Modified header, or where either header is syntactically incorrect, will be rejected.

Example # no ETag or Last-Modified will be rejected PolicyValidation enforce PolicyValidationURL URL describing the content type policy. PolicyValidationURL url none server configvirtual host directory PolicyValidationURL is only available in Apache 2.5.0 and later.

Specify the URL of the documentation describing the validation policy, to appear within error messages.

 PolicyNocache Enable the caching no-cache policy. PolicyNocache ignore|log|enforce ignore server configvirtual host directory PolicyNocache is only available in Apache 2.5.0 and later.

When logged or enforced, a response that defines itself uncacheable using the Cache-Control or Pragma headers will be rejected.

Example # Cache-Control: no-cache will be rejected PolicyNocache enforce PolicyNocacheURL URL describing the caching no-cache policy. PolicyNocacheURL url none server configvirtual host directory PolicyNocacheURL is only available in Apache 2.5.0 and later.

Specify the URL of the documentation describing the caching no-cache policy, to appear within error messages.

 PolicyMaxage Enable the caching minimum max-age policy. PolicyMaxage ignore|log|enforce age ignore server configvirtual host directory PolicyMaxage is only available in Apache 2.5.0 and later.

When logged or enforced, a response that lacks an explicit freshness lifetime defined with max-age, s-maxage or an Expires header, or where the explicit freshness lifetime is smaller than the given value, will be rejected.

Example # reject responses with a freshness lifetime shorter than a day PolicyMaxage enforce 86400 PolicyMaxageURL URL describing the caching minimum freshness lifetime policy. PolicyMaxageURL url none server configvirtual host directory PolicyMaxageURL is only available in Apache 2.5.0 and later.

Specify the URL of the documentation describing the caching minimum freshness lifetime policy, to appear within error messages.

 PolicyVersion Enable the version policy. PolicyVersion ignore|log|enforce HTTP/0.9|HTTP/1.0|HTTP/1.1 ignore server configvirtual host directory PolicyVersion is only available in Apache 2.5.0 and later.

When logged or enforced, a request with a version lower than specified will be rejected.

Example # reject requests with an HTTP version older than HTTP/1.1 PolicyVersion enforce HTTP/1.1 PolicyVersionURL URL describing the minimum request HTTP version policy. PolicyVersionURL url none server configvirtual host directory PolicyVersionURL is only available in Apache 2.5.0 and later.

Specify the URL of the documentation describing the minimum request HTTP version policy, to appear within error messages.