A benchmark suite which, transparently, stress-tests web servers and generates a report in markdown. It measures the requests per second, data transferred and time between requests and responses.
It's true, there already enough of benchmark suites to play around. However, most of them don't even contain real-life test applications to benchmark, therefore the results are not always accurate e.g. a route handler executes SQL queries or reads and sends JSON. This benchmark suite is a fresh start, it can contain any type of tests as the tests are running as self-executables and the measuring is done by a popular and trusted 3rd-party software which acts as a real HTTP Client (one more reason of transparency). Contributions and improvements are always welcomed here.
Measure the performance of application(s) between different versions or implementations (or web frameworks).
This suite can be further customized, through its tests.yml file, in order to test personal or internal web applications before their public releases.
- Install Go, Bombardier, Node.js and .NET Core
- Clone the repository
- Stress-tests are described inside tests.yml file, it can be customized to fit your needs
- Execute:
go build -o server-benchmarks - Run and wait for the executable server-benchmarks (or server-benchmarks.exe for windows) to finish
- Read the results from the generated README.md file.
The only requirement is Docker.
$ docker run -v ${PWD}:/data kataras/server-benchmarksThe following generated README contains benchmark results from builtin tests between popular HTTP/2 web frameworks as of 2020.
Note: it's possible that the contents of this file will be updated regularly to accept even more tests cases and frameworks.
| Processor | Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz |
| RAM | 15.85 GB |
| OS | Microsoft Windows 10 Pro for Workstations |
| Bombardier | v1.2.4 |
| Go | go1.14.2 |
| .Net Core | 3.1.102 |
| Node.js | v13.10.1 |
Last updated: Apr 12, 2020 at 2:41am (UTC)
Name is the name of the framework(or router) used under a particular test.
Reqs/sec is the avg number of total requests could be processed per second (the higher the better).
Latency is the amount of time it takes from when a request is made by the client to the time it takes for the response to get back to that client (the smaller the better).
Throughput is the rate of production or the rate at which data are transferred (the higher the better, it depends from response length (body + headers).
Time To Complete is the total time (in seconds) the test completed (the smaller the better).
📖 Fires 1000000 requests, receives a static message as response.
| Name | Language | Reqs/sec | Latency | Throughput | Time To Complete |
|---|---|---|---|---|---|
| Iris | Go | 205603 | 606.83us | 35.86MB | 4.87s |
| Kestrel | C# | 194821 | 639.91us | 33.06MB | 5.13s |
| Chi | Go | 192223 | 648.71us | 33.54MB | 5.20s |
| Gin | Go | 187246 | 666.13us | 32.67MB | 5.34s |
| Echo | Go | 185804 | 671.59us | 32.40MB | 5.39s |
| Martini | Go | 160296 | 777.36us | 27.98MB | 6.24s |
| Koa | Javascript | 107892 | 1.14ms | 21.53MB | 9.17s |
| Express | Javascript | 91181 | 1.44ms | 22.20MB | 11.60s |
| Buffalo | Go | 39010 | 3.20ms | 6.81MB | 25.63s |
📖 Fires 250000 requests with a dynamic parameter of string, receives a hello text based on the parameter as response.
| Name | Language | Reqs/sec | Latency | Throughput | Time To Complete |
|---|---|---|---|---|---|
| Iris | Go | 192088 | 649.16us | 36.78MB | 1.30s |
| Chi | Go | 187973 | 663.14us | 36.00MB | 1.33s |
| Echo | Go | 177471 | 703.68us | 33.96MB | 1.41s |
| Kestrel | C# | 172411 | 728.64us | 31.78MB | 1.46s |
| Gin | Go | 169312 | 740.39us | 32.27MB | 1.48s |
| Martini | Go | 145707 | 0.85ms | 27.94MB | 1.71s |
| Koa | Javascript | 74756 | 1.67ms | 15.99MB | 3.35s |
| Express | Javascript | 65553 | 2.20ms | 15.57MB | 4.41s |
| Buffalo | Go | 38577 | 3.24ms | 7.40MB | 6.48s |
📖 Fires 250000 requests, sets a session and displays its value.
| Name | Language | Reqs/sec | Latency | Throughput | Time To Complete |
|---|---|---|---|---|---|
| Iris | Go | 102580 | 1.22ms | 35.22MB | 2.44s |
| Kestrel | C# | 74056 | 1.69ms | 34.07MB | 3.39s |
| Echo | Go | 73521 | 1.70ms | 38.14MB | 3.40s |
| Chi | Go | 67068 | 1.86ms | 34.77MB | 3.73s |
| Martini | Go | 66955 | 1.86ms | 34.81MB | 3.73s |
| Gin | Go | 60140 | 2.10ms | 24.95MB | 4.20s |
| Koa | Javascript | 52796 | 2.75ms | 20.47MB | 5.51s |
| Express | Javascript | 31982 | 4.27ms | 7.82MB | 8.56s |
| Buffalo | Go | 16548 | 7.55ms | 23.74MB | 15.11s |
📖 Fires 200000 requests with a dynamic parameter of int, sends JSON as request body and receives JSON as response.
| Name | Language | Reqs/sec | Latency | Throughput | Time To Complete |
|---|---|---|---|---|---|
| Iris | Go | 147754 | 843.88us | 40.43MB | 1.35s |
| Chi | Go | 141918 | 0.88ms | 38.15MB | 1.41s |
| Kestrel | C# | 136747 | 0.92ms | 39.68MB | 1.47s |
| Gin | Go | 136480 | 0.91ms | 37.32MB | 1.47s |
| Echo | Go | 134209 | 0.93ms | 36.84MB | 1.49s |
| Martini | Go | 123638 | 1.01ms | 33.22MB | 1.62s |
| Buffalo | Go | 56722 | 2.20ms | 15.56MB | 3.53s |
| Koa | Javascript | 47089 | 2.66ms | 13.91MB | 4.26s |
| Express | Javascript | 41018 | 3.29ms | 13.59MB | 5.28s |
This project is licensed under the MIT License.