0) Index
********
0........Index
1........About
2........License
3........Compiling The Source
3.1....Feature Configuration
3.2....Make Targets
4........HTTP Server
4.1....Command line
4.2....Socket configuration
4.3....Virtual Host Configuration
4.4....Request path resolution
4.5....HTTPS support
4.6....Privilege Dropping
5........Database Server
6........JSON Parser & Serializer
7........Session managament
1) About
********
This directory contains a REST framework/HTTP server written in C89, but
utilizing Linux specific system calls and GNU library extensions to the
utmost extent possible to keep the server small, simple and efficient.
I developed this framework for university courses and private projects that
required me to build a REST service or other web applications, after being
annoyed with Tomcat+Jax-RS or similarly heavy weight C++ frameworks.
Currently supported features:
- Supports multiple virtual host configurations
- Supports binding to different addresses/ports
- Supports IPv4, IPv6 and UNIX domain sockets
- Supports HTTP GET, HEAD, POST, PUT and DELETE methods
- Supports HTTP keep-alive
- Supports cookies
- Guesses content-type from static files using file extension
- Serve gzip compressed, static files if the client supports it
- Dynamically generated data can be compressed on the fly
- Use zero-copy splice syscall to serve static files
- Tell clients to do caching on static files
- Propperly handle If-Modified requests for static files
- Map a specific path to REST service callback
- Includes a simple JSON parser that generates C-structs from JSON
- Includes a simple JSON serializer that genereats JSON from C-structs
- Simple document/page templateing system
- Minimal SQLite based data base server
- Built-in session management
- Hard timeouts and limits on requests to guarntee that
request handling terminates
- Gracefully handles segfaults when serving requests. Log the request and
send an error page.
- Configurable featureset, so the server can be only used to
only server static files etc...
However, there are currently a few limitations:
- Only absolute paths supported
- No directory views (requests to directories are rejected)
2) License
**********
For proprietary licensing options, please contact the author.
The copy of the source code of the HTTP and Database servers that you
received (all source files included in this directory and sub directories)
are subject to the terms and conditions of the GNU AFFERO GENERAL PUBLIC
LICENSE version 3.
See LICENSE file for details.
3) Compiling The Source
***********************
The GNU build system, aka autotools are required for compiling the server
binaries (that plus make and a C compiler).
The data base server requires the SQLite library.
The HTTP server requires the zlib library for compression of dynamic pages.
The programs make heavy use of GNU library extensions and Linux specific
system calls, so they should be available in the systems C standard library.
3.1) Feature Configuration
The following command line switches can be supplied when running the
configure script to tweak the HTTP & data base server binaries:
--disable-json-parser
Compile the server without JSON-to-C-struct deserializer.
--disable-json-serializer
Compile the server without C-struct-to-JSON serializer.
--disable-session
Compile the database server without session managament.
--disable-rest
Compile the server without dynamic page / rest API backend.
--disable-static
Compile the server without static file backend.
--with-client-timeout=<number>
Specify the maximum number of milli seconds to wait for a client
to send a request. Default is 2000 (= 2 seconds).
--with-request-timeout=<number>
Specify the maximum number of seconds for handling a request.
A SIGALRM is used to interrupt request handling if it takes
longer than this and an error page is sent to the client.
Default is 5 seconds.
--with-download-timeout=<number>
Specify the maximum number of seconds for processing a request
for a static file. A SIGALRM is used to interrupt request handling
if it takes longer than this and an error page is sent to the client.
Default is 7200 seconds (= 2h, roughly enough to download 48 MiB on
a 56k connection).
--with-requests=<number>
Specify the maximum number of requests to handle per TCP connection
for HTTP pipelining/keep-alive. Default is 1000.
3.2) Make Targets
Apart from the default automake make targets, a custom 'strip' target
has been added for stripping the output binaries.
The 'cert' target can be used to generate a simple SSL certificate for
use with stunnel.
4) HTTP Server
**************
4.1) Command line
To run the HTTP server, start it with the following command line arguments:
-c, --cfg <configfile> Specify a configuration file with virtual host
configuration. This option is required.
-f, --log <file> Append logging output to a specific file
-l, --loglevel <num> Higher level means more detailed/verbose output.
Possible values are:
0 - only print critical errors
1 - log critical errros and warnings (DEFAULT)
2 - also log generic information
3 - debug output
-r, --chroot <path> Filesystem root directory for the server
Note: The chroot is switched into before reading the configuration file, but
the log file is created before switching into the chroot.
The path for the configuration file has to be relative to the chroot
directory (if specified). The path is relative to the actual root.
The config file for the HTTP server is an .ini file with multiple "host"
sections for each virtual hosts and multiple "ipv4", "ipv6" and "unix"
sections for sockets to create.
A sample configuration is included with the source code.
The HTTP server can be forced to reload its configuration by sending it a
SIGHUP signal.
4.2) Socket configuration
Each "ipv4" and "ipv6" section in the config file must have a "bind" and
"port" field specifying what address ("ANY" for all incoming addresses) and
port to bind to.
The "unix" section only requires a "bind" argument for the unix socket path
to bind to.
Note: IPv6 sockets only accept IPv6 traffic. If a dual stack setup is
required, both IPv4 and IPv6 bindings have to be specified explicitly.
4.3) Virtual Host Configuration
Each "host" section in the config file can have the folowing keys:
hostname The value of the HTTP "Host:" request field to map.
restdir An optional, virtual directory to map to the internal
callback system.
datadir An optional directory from which to server static files.
templatedir An optional directory from which to load document
template files.
rootfile If the document root ("/") is requested, rewrite the request
path to this path (may be underneath the restdir or an
oridnary file)
4.4) Request path resolution
When a client requests a file, the server first tries to resolve it via the
dynamic backend.
If that backend also returns a 404 response, the server tries to serve a
static file instead.
This means, that the dynamic backend can override static files, even on a
forwarded directory, if it wants to.
When resolving a static file and the client supports gzip compression, the
server tries to append ".gz" to the path (unless the path already ends
with *.gz). If that file exists, the server returns it _instead_ of the
actually requested file and tells the client that it is gzip compressed.
If that fails, the server tries to open the actual path and transfers
that file. This allows serving precompressed files on the server if the
client supports it, with a fallback to uncompressed files.
If the requested path referes to a directory (or _anything_ other than a
regular file), the server returns a "403 Forbidden" response.
The server only tries to serve the index file, if the root document is
requested (e.g. "GET /"). Serving the index file for arbitrary
sub-directories, or serving directory views is not supported.
4.5) HTTPS support
The HTTP server itself does not support TLS connections. To support HTTPS
connections, a TLS reverse proxy is required that redirects the unencrypted
traffic to the HTTP server internally.
A simple demo configuration is provided on how to use stunnel as an TLS
wrapper on top of the HTTP server.
Run "make cert" to generate private key, certificate and DH parameters for
the server. Then run "stunnel ./tls.conf" to start up an TLS server that
accepts connections on port 4040, servers the generated certificate and
redirects the raw data to a Unix socket that the HTTP server provides.
4.6) Privilege Dropping
In the server configuration file, an optional section "user" can be specified
with the numeric values "uid" and "gid" to specify values for the real,
effective and saved user/group IDs that the server should assume.
Here is an example of what that section might look like:
[user]
uid = 1000 # Set real/effective/saved user ID to 1000
gid = 100 # Set real/effective/saved group ID to 100
In addition, the command line argument "--chroot <path>" can be used to
specify a directory that the server should chroot into before changing its
user and group IDs.
No files are accessed before the chroot, so all file paths (config file,
unix socket paths, etc...) must be specified relative to this directory.
Only the logfile is created before entering the chroot and can thus be
specified outside the chroot directory.
5) Database Server
******************
The database server implements a simple broker model. It receives a very
simple protocol over a UNIX domain socket and retrieves data from an SQLite
database. Buffer bounds and sanity of transfered values must be thoroughly
checked when accessing the payload from either side!
For a _clean_ and _secure_ seperation, the HTTP server itself does not
transfer acutal SQL queries to the DB server; it sends application specific
request tokens that the DB server translates to SQL queries. Idealy, the
HTTP server and DB server are run as different users that cannot access each
others files, thus allowing the HTTP server to only access data that it is
supposed to.
To run the databse server, start it with the following command
line arguments:
-d, --db <dbfile> Specify the data base file to access
-s, --sock <unixsocket> Specify the UNIX socket to listen on
-f, --log <file> Append logging output to a specific file
-l, --loglevel <num> Higher level means more detailed/verbose output.
Possible values are:
0 - only print critical errors
1 - log critical errros and warnings (DEFAULT)
2 - also log generic information
3 - debug output
6) JSON Parser & Serializer
***************************
A parser for a limited subset of JSON is provided. It deserializes JSON
directly to C structures in memory using a describtion table that can be
generated semiautomatically via the preprocessor.
The parser currently has the following limitations:
- The parser only supports arrays of objects (not strings or integers)
- Booleans are internally treated as integers
- Floating point values are not supported
- A JSON document must contain either an array or an object at the root
- Objects and arrays must not be empty (i.e. contain at least one element)
- Unknown JSON fields are skipped. Skiped fields may contain JSON values
without any restrictions to the JSON grammar.
In addition to the parser, a serializer is provided that can convert C
structs directly to JSON strings.
7) Session managament
*********************
If configured to do so, the database server can manage sessions in an
in-memory list. The HTTP server has wrapper functions (see user.h) for
managing sessions that talk to the database server.
A session has a unique, non-zero, randomly generated 32 bit session ID that
is (preferably) stored in a cookie called "session" in base 32
representation.
For each session, the database server stores an asociated 32 bit user ID and
timestamp when the session has been last accessed. It can be easily modified
to store additional data.
If a session has not been accessed for a certain amount of time (default 10
minutes, see session.h), it is removed.
Since the sessions are not stored on a persistent medium, all sessions are
lost if the database server is stopped.
The same user can have multiple sessions at a time.
Idealy, one would store a persisten user list in the database and verify
login credentials before handing out sessions. For simplicity, the session
demo lets the user pick a user ID and creates a session.