Worker Conf File Reference

Which virtual hosts should be registered with the hub.

  :hosts ("example.com" "www.example.com" "www.example.com:8080")

• This parameter is required otherwise the hub will not route any connections to this handler.
• Make sure you specify all hosts this worker should handle, including sub-domains, ports, IPv4 and IPv6 addresses, etc.
• Two separate handlers cannot both handle the same host (only the first defined will get connections).

Handle hosts even if they aren't specified in :hosts.

  :handle-all-hosts

• A handler that uses this option should appear last in the worker conf because no handlers specified later in the file will have a chance to handle the request.
• This option is only useful when combined with the install-hub-rewrite-host xconf in hub.conf.

Absolute path to the HTML tree that should be served.

  :root "/absolute/path/to/root"

• One of (but not both) a :root or a :simple-vhost-root argument is required.
• If the worker process is chroot()ed, it must be the absolute path from the new root, not the original root.

Absolute path to a directory containing HTML trees named by the virtual hosts being served. The names of these child directories must be names of registered vhosts for this worker.

  :simple-vhost-root "/absolute/path/to/root"

• One of (but not both) a :simple-vhost-root or a :root argument is required.
• If the worker process is chroot()ed, it must be the absolute path from the new root, not the original root.
• The mere fact that a directory exists is not enough to register a virtual host. Make sure that all hosts are registered in the :hosts parameter.
• The directory names should be the lowercase versions of the virtual hosts.
• You can use symlinks so that multiple virtual hosts are mapped to the same directory.
• :simple-vhost-root is for Antiweb3 and lighttpd compatibility.

Antiweb Tip
Although :rewrite can do much more than just rewrite request paths, it is named in recognition of Apache's mod_rewrite.:rewrite lets you compile Common Lisp code directly into the worker's HTTP dispatch function. You can have as many :rewrite parameters as you want. They are evaluated in sequence or until the connection is redirected or until (done-rewrite) or (keepalive) is evaluated. The rewriting happens before any other request processing is done. Here is a simple example that will redirect domains that start with www to their no-www versions:

  :rewrite (if-match (#~m/^www[.](.*)/ http-host)
             (redir 301 "http://~a~a" $1 http-path))

When redirecting a connection to a different worker or a different server altogether, you might choose to redir-close which tears down HTTP 1.1 keepalive connections:

  :rewrite (if-match (#~m/^www[.](.*)/ http-host)
             (redir-close 301 "http://~a~a" $1 http-path))

Supported codes for :redir and :redir-close are:

• 301 - Moved Permanently
• 302 - Found
• 307 - Temporary Redirect

There are many variables that you can access and even modify before the request is handled. Here is a partial list:

• http-method - The requested HTTP method. Always the symbol GET or the symbol POST.
• http-ver - The HTTP protocol version in use. Either "0.9", "1.0", or "1.1".
• http-path-encoded - The path portion of the URL before URL decoding.
• http-path - The decoded path portion of the URL. This will always begin with a / character. This will include the path-info portion of the path.
• http-args - The URL encoded URL arguments (portion of URL after the ? character, not including the ?).
• http-host - The lowercase version of the virtual host requested by the client.
• $u-anything-you-like - These are sub-lexical variables. You can use them for anything, including sharing values between rewrite rules or other Antiweb modules. They are initially bound to nil.
• $h-any-http-header - These are also sub-lexical variables. You can use them to access any HTTP header. To be honest, they are regular expressions. $h-referr?er will contain the contents of a Referer or Referrer header line (whichever appears first).

The form add-header-to-response adds an HTTP header to any successful response:

  :rewrite
    (if-match (#~m|^/logo[.]png$| http-path)
      (add-header-to-response "Cache-Control: public, max-age=3600, must-revalidate"))

Here is how to use :rewrite to add a virtual URL /stats.html that will display stats and room info about a worker process:

  :rewrite
    (if-match (#~m|^/stats[.]html$| http-path)
      (let ((body (format nil #"<html><title>Antiweb Worker Stats</title>
                                <h1>Antiweb Worker Stats</h1>
                                <h2>-stats</h2><pre>~a</pre><br>
                                <h2>-room</h2><pre>~a</pre></html>"#
                              (stats) (aw-room))))
        (send-http-response-headers 200
          ("Content-Type" "text/html; charset=utf-8")
          ("Content-Length" "~a" (length body)))
        (send-raw body)
        (keepalive)))

The above is a low-level way of building an HTML response, sending it to the client, and keeping the connection alive (HTTP/1.1 clients only). Another way this could have been written is with the send-html-response-and-keepalive macro:

  :rewrite
    (if-match (#~m|^/stats[.]html$| http-path)
      (send-html-response-and-keepalive
        #"<html><title>Antiweb Worker Stats</title>
          <h1>Antiweb Worker Stats</h1>
          <h2>-stats</h2><pre>"#
        (stats)
        #"</pre><br><h2>-room</h2><pre>"#
        (aw-room)
        #"</pre></html>"#))

Soon there will be more :rewrite recipes available here, in the FAQ, and elsewhere.

• Never paste something you don't understand into a conf file.

Sets the default MIME type for files with unrecognised extensions (or no extensions at all).

  :default-mime-type "text/plain; charset=utf-8"

A two element list. The first element is a file extension and the second is the MIME type and optionally charset to map the extension to.

  :mime-type (htm "text/html; charset=iso-8859-1")

• You can over-ride a default Antiweb MIME type with this parameter.
• The file extensions are case-insensitive.
• There can be multiple :mime-type options in a handler. The first one that matches is used.

A list of index files to be searched for when a directory is requested.

  :index ("index.html" "index.pl")

• The first index file found is the one used (left-to-right).
• A 404 Not Found is served if no index is found (unless :dir-listings are enabled).

If no index file is found, this option causes an HTML listing of the files in a directory to be served instead of a 404.

  :dir-listings

• Directory listings always tear down persistent HTTP connections to prevent recursive crawlers from taking advantage of HTTP pipelining.

Enables the sending of 206 Partial Content if the client indicates it already has a prefix range of a static file.

  :download-resuming

• No other byte-ranges are supported.

Enables the sending of etag (entity tag) HTTP headers when serving static files. Clients that cache the files are sent 304 Not Modified responses if they re-request the file and it hasn't changed.

  :etags

• Antiweb uses the standard(ish) etag format "INODE-SIZE-MTIME" (in hex).
• Last-Modified is never sent by Antiweb.

Uses the SHA1 algorithm to disguise the Etag generated with :etags to avoid exposing your inode numbers. Requires a secret key value.

  :etags :etag-hash "key value goes here"

• You must also enable the :etag xconf for this to work.
• See this FAQ entry for more information.

A trusted IP address that is permitted to set a connection's IP address with the X-Real-IP header. This is so that the client's IP address will be properly logged when reverse proxying is used.

  :accept-x-real-ip-from "127.0.0.1"

• This FAQ entry describes an example nginx configuration that sends this header when reverse proxying SSL connections.

A list of file extensions for files that should be gzip compressed and stored in the worker's cache directory and then served as static files. Only clients that indicate compression support are sent the compressed versions.

  :gzip (html txt css js)

• The gziped copies are updated when the original file's modification time (mtime) changes.
• There is no point compressing formats like png and mp3 because they are already compressed.

A list of two elements. The first is the minimum file size (in bytes) to compress and store in the cache. The second is the maximum file size (also in bytes).

  :gzip-size-range (256 1000000)

• This parameter is optional. If :gzip is used without :gzip-size-range, sensible defaults are used.

Enables "javascript minification". Files that end in .js in your HTML root will be minified, gzip compressed, and stored in the cache. Minification removes comments, unnecessary whitespace, etc from your javascript files for more efficient use of bandwidth and faster loading pages.

  :jsmin

• When the modification time (mtime) of the original .js file changes, the file will be re-minified on the next request.
• jsmin.lisp was written by Ury Marshak for ht-ajax. It is based on an algorithm by Douglas Crockford.
• Invalid javascript files, such as ones that forget to close comments, are not minified. Instead, the original files are sent to clients.
• :js-main fields in AWP files have their contents minified when the AWP file is compiled.

Enables "CSS minification". Files that end in .css in your HTML root will be minified, gzip compressed, and stored in the cache. Minification removes comments, unnecessary whitespace, etc from your CSS files for more efficient use of bandwidth and faster loading pages.

  :cssmin

• When the modification time (mtime) of the original .css file changes, the file will be re-minified on the next request.
• cssmin is based on an algorithm by Cal Henderson.
• :css fields in AWP files have their contents minified when the AWP file is compiled.

Adds a URL that serves a 1 pixel by 1 pixel transparent gif.

  :fast-1x1gif "/1x1.gif"

• The gif and its pre-generated HTTP headers are served directly from memory. This eliminates a stat call, generating the headers, opening the file, reading from the file, and closing the file.
• The 43 byte gif file is from nginx.

A list of URLs that should be served directly from a memory cache:

  :fast-files ("/favicon.ico" "/robots.txt")

• After they are first read, these files and their pre-generated HTTP headers are served directly from a cache in memory. This eliminates a stat call, generating the HTTP headers, opening the file, reading from the file (or mmap()+munmap()ing it), and closing the file.
• When the worker conf file is -reloaded, the memory cache is flushed and the files are then re-loaded from the filesystem on-demand. Alternatively, to avoid the slight delay when you re-compile your HTTP dispatch function, you can evaluate (flush-fast-files) while attached to a worker.
• If Antiweb is unable to access one of the fast files, pre-generated 404s are subsequently sent without even consulting the filesystem (this is called negative caching). Unlike normal 404s, these will not tear down a persistent connection and will pipeline just fine.
• You can see how many fast files, fast 404s, and bytes are currently in a worker's fast-file memory cache with the -stats command. The byte count includes pre-generated HTTP headers.
• If simple-vhost-root is used, each vhost will have its own memory cache.
• Only small files should be cached in memory. Medium and large files are usually served optimally from the filesystem.

HTTP headers to add to :fast-files served files.

  :fast-files-header "Cache-Control: public, max-age=3600, must-revalidate"

A list of file extensions to treat as CGI scripts.

  :cgi (pl php)

• The extensions are case-insensitive.
• See Design of CGI Processes in Antiweb4 for details.

A list of file extensions to treat as naked CGI scripts.

  :naked-cgi (npl)

• The extensions are case-insensitive.
• Naked CGI scripts are identical to regular CGI scripts except Antiweb will not prefix the following:

  HTTP/1.1 200 OK\r\nConnection: close\r\n

  The script must send this prefix itself.
• Naked CGI scripts are similar to NPH (No Parsed Headers).

Limit the number of file descriptors a CGI or naked CGI process may have open at once:

  :cgi-maxfiles 100

• If not specified, a sensible default value is used.
• This value must be smaller than the worker's max-fds value.

Use the NPROC resource limit to ensure that CGI or naked CGI processes launched from this handler cannot fork().

  :cgi-no-forking

• This option negates certain common mistakes in CGI scripts. For example, a bug that allows an attacker to launch system() or shell "backquotes" will result in the fork() call failing.
• This a very restrictive option and will prevent many perfectly legitimate CGI scripts from working. If you don't understand exactly what this does, you probably don't need it.
• We may add additional CGI resource limits in the future. Let us know which you find most useful.

Enable Anti Webpages.

  :awp

• Worker must have a cache directory.
• Because all code required to compile and serve Anti Webpages is stored in the lisp image, you can chroot workers that use :awp.
• Anti Webpages are affected by the :etags and :gzip options just like regular static files.
• See Anti Webpages for more details.