Lwan Web Server
Lwan is a high-performance & scalable web server for glibc/Linux platforms.
In development for almost 4 years, Lwan was until now a personal research effort that focused mostly on building a solid infrastructure for a lightweight and speedy web server:
- Low memory footprint (~500KiB for 10k idle connections)
- Minimal memory allocations & copies
- Minimal system calls
- Hand-crafted HTTP request parser
- Files are served using the most efficient way according to their size
- No copies between kernel and userland for files larger than 16KiB
- Smaller files are sent using vectored I/O of memory-mapped buffers
- Header overhead is considered before compressing small files
- Mostly wait-free multi-threaded design
- Diminute code base with roughly 14000 lines of C code
It is now transitioning into a fully working, capable HTTP server. It is not, however, as feature-packed as other popular web servers. But it is free software, so scratching your own itches and making Lwan hum the way you want it to is possible.
Features include:
- Mustache templating engine
- Used for directory listing & error messages
- Available for user-built handlers
- Easy to use API to create web applications or extend the web server
- Supports rebimboca da parafuseta
- Test suite written in Python tests the server as a black box
- No-nonsense configuration file syntax
- Supports a subset of HTTP/1.0 and HTTP/1.1
- systemd socket activation
- IPv6 ready
The web site has more details, including a FAQ about the name of the project and security concerns.
Performance
It can achieve good performance, yielding about 320000 requests/second on a Core i7 laptop for requests without disk access, and without pipelining.
When disk I/O is required, for files up to 16KiB, it yields about 290000 requests/second; for larger files, this drops to 185000 requests/second, which isn't too shabby either.
These results, of course, with keep-alive connections, and with weighttp running on the same machine (and thus using resources that could be used for the webserver itself).
Without keep-alive, these numbers drop around 6-fold.
Portability
While Lwan was written originally for Linux, it has been ported to BSD systems as well. The build system will detect the supported features and build support library functions as appropriate.
For instance, epoll has been implemented on top of kqueue, and Linux-only syscalls and GNU extensions have been implemented for the supported systems.
Building
Before installing Lwan, ensure all dependencies are installed. All of them are common dependencies found in any GNU/Linux distribution; package names will be different, but it shouldn't be difficult to search using whatever package management tool that's used by your distribution.
Required dependencies
Optional dependencies
The build system will look for these libraries and enable/link if available.
- Lua 5.1 or LuaJIT 2.0
- TCMalloc
- jemalloc
- Valgrind
- To run test suite:
- To run benchmark:
- Special version of Weighttp
- Matplotlib
- To build TechEmpower benchmark suite:
Common operating system package names
Minimum to build
- ArchLinux:
pacman -S cmake zlib
- FreeBSD:
pkg install cmake pkgconf
- Ubuntu 14+:
apt-get update && apt-get install git cmake zlib1g-dev pkg-config
Build all examples
- ArchLinux:
pacman -S cmake zlib sqlite luajit libmariadbclient gperftools valgrind
- FreeBSD:
pkg install cmake pkgconf sqlite3 lua51
- Ubuntu 14+:
apt-get update && apt-get install git cmake zlib1g-dev pkg-config lua5.1-dev libsqlite3-dev libmysqlclient-dev
Build commands
Clone the repository
~$ git clone git://github.com/lpereira/lwan
~$ cd lwan
Create the build directory
~/lwan$ mkdir build
~/lwan$ cd build
Select build type
Selecting a release version (no debugging symbols, messages, enable some optimizations, etc):
~/lwan/build$ cmake .. -DCMAKE_BUILD_TYPE=Release
If you'd like to enable optimiations but still use a debugger, use this instead:
~/lwan/build$ cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo
To disable optimizations and build a more debugging-friendly version:
~/lwan/build$ cmake .. -DCMAKE_BUILD_TYPE=Debug
Build Lwan
~/lwan/build$ make
This will generate a few binaries:
src/bin/lwan/lwan
: The main Lwan executable. May be executed with--help
for guidance.src/bin/testrunner/testrunner
: Contains code to execute the test suite.src/samples/freegeoip/freegeoip
: FreeGeoIP sample implementation. Requires SQLite.src/samples/techempower/techempower
: Code for the Techempower Web Framework benchmark. Requires SQLite and MySQL libraries.src/bin/tools/mimegen
: Builds the extension-MIME type table. Used during build process.src/bin/tools/bin2hex
: Generates a C file from a binary file, suitable for use with #include.
Remarks
Passing -DCMAKE_BUILD_TYPE=Release
will enable some compiler
optimizations (such as LTO)
and tune the code for current architecture. Please use this version
when benchmarking, as the default is the Debug build, which not only
logs all requests to the standard output, but does so while holding a
mutex.
The default build (i.e. not passing -DCMAKE_BUILD_TYPE=Release
) will build
a version suitable for debugging purposes. This version can be used under
Valgrind (if its headers are present), is built with Undefined Behavior
Sanitizer, and includes debugging messages that are stripped in the release
version. Debugging messages are printed for each and every request.
Which sanitizer will be used in a debug build can be selected by passing the following arguments to the CMake invocation line:
-DSANITIZER=ubsan
selects the Undefined Behavior Sanitizer.-DSANITIZER=address
selects the Address Sanitizer.-DSANITIZER=thread
selects the Thread Sanitizer.
Tests
~/lwan/build$ make teststuite
This will compile the testrunner
program and execute regression test suite
in src/scripts/testsuite.py
.
Benchmark
~/lwan/build$ make benchmark
This will compile testrunner
and execute benchmark script
src/scripts/benchmark.py
.
Coverage
Lwan can also be built with the Coverage build type by specifying
-DCMAKE_BUILD_TYPE=Coverage
. This enables the generate-coverage
make
target, which will run testrunner
to prepare a test coverage report with
lcov.
Every commit in this repository triggers the generation of this report, and results are publicly available.
Running
Set up the server by editing the provided lwan.conf
; the format is
very simple and should be self-explanatory.
Configuration files are loaded from the current directory. If no changes
are made to this file, running Lwan will serve static files located in
the ./wwwroot
directory. Lwan will listen on port 8080 on all interfaces.
Lwan will detect the number of CPUs, will increase the maximum number of open file descriptors and generally try its best to autodetect reasonable settings for the environment it's running on.
Optionally, the lwan
binary can be used for one-shot static file serving
without any configuration file. Run it with --help
for help on that.
IRC Channel
There is an IRC channel (#lwan
) on Freenode. A
standard IRC client can be used. A web IRC gateway
is also available.
Lwan in the wild
Here's a non-definitive list of third-party stuff that uses Lwan and have been seen in the wild. Help build this list!
- An experimental version of Node.js using Lwan as its HTTP server is maintained by @raadad.
- The beginnings of a C++11 web framework based on Lwan written by @vileda.
- A more complete C++14 web framework by @matt-42 offers Lwan as one of its backends.
- A word ladder sample program by @sjnam. Demo.
- A Shodan search listing some brave souls that expose Lwan to the public internet.
Some other distribution channels were made available as well:
- A
Dockerfile
is maintained by @jaxgeller, and is available from the Docker registry. - A buildpack for Heroku is maintained by @bherrera, and is available from its repo.
- Lwan is also available as a package in Biicode.
- User packages for Arch Linux and Ubuntu.
Lwan has been also used as a benchmark:
- Raphael Javaux's master thesis cites Lwan in chapter 5 ("Performance Analysis").
- Lwan is used as a benchmark by the PyParallel author.
- Kong uses Lwan as the backend API in its benchmark.
- TechEmpower Framework benchmarks feature Lwan since round 10.
- KrakenD used Lwan for the REST API in all official benchmarks
Mentions in academic journals:
- A dynamic predictive race detector for C/C++ programs uses Lwan as a "real world example".
Some talks mentioning Lwan:
- Talk about Lwan at Polyconf16, given by @lpereira.
- This talk about Iron, a framework for Rust, mentions Lwan as an insane C thing.
- University seminar presentation about Lwan.
- This presentation about Sailor web framework mentions Lwan.
Not really third-party, but alas:
- The author's blog.
- The project's webpage.
Build status
OS | Arch | Release | Debug | Static Analysis | Tests |
---|---|---|---|---|---|
Linux | x86_64 | Report history | |||
Linux | armv7 | ||||
FreeBSD | x86_64 | ||||
macOS | x86_64 |