This project aims to be a load benchmarking suite, no more, no less
Measuring response times (routing times) for each framework (middleware).
Results are not production-ready yet
- Helping decide between languages, depending on use case
- Learning languages, best practices, devops culture ...
- Having fun ❤️
- Crystal as
built-intools are made in this language - Docker as frameworks are
isolatedinto containers - wrk as benchmarking tool,
>= 4.1.0 - postgresql to store data,
>= 10
ℹ️ℹ️ℹ️ℹ️ℹ️
OSX you need docker-machine to use docker containerization
brew install docker-machine
docker-machine create default
eval $(docker-machine env default)
ℹ️ℹ️ℹ️ℹ️ℹ️
- Install all dependencies
shards install- Build internal tools
shards build- Create and initialize the database
createdb -U postgres benchmark
psql -U postgres -d benchmark < .ci/dump.sqlDocker can be used to set up the database:
docker run -it --rm -d \
-p 5432:5432 \
-e POSTGRES_DB=benchmark \
-e POSTGRES_HOST_AUTH_METHOD=trust \
-v /tmp/pg-data:/var/lib/postgresql/data \
--name pg postgres:12-alpineWait several seconds for the container to start, then inject the dump:
docker exec pg sh -c "echo \"$(cat .ci/dump.sql)\" | psql -U postgres -d benchmark"After creating the database, export its URL:
export DATABASE_URL="postgresql://postgres@localhost/benchmark"- Make configuration
bin/make config- Build containers
jobs are either languages (example : crystal) or frameworks (example : router.cr)
bin/neph [job1] [job2] [job3] ...- Export all results readme
bin/db to_readmeℹ️ Updated on 2020-06-09 ℹ️
Benchmarking with wrk
- Threads : 8
- Timeout : 8
- Duration : 15s (seconds)
ℹ️ Sorted by max req/s on concurrency 64 ℹ️
| Language | Framework | Speed (64) | Speed (256) | Speed (512) | |
|---|---|---|---|---|---|
| 1 | javascript (13.14) | nanoexpress-pro (1.11) | 189 430 | 203 510 | 203 806 |
| 2 | nim (1.2) | httpbeast (0.2) | 188 202 | 201 111 | 202 630 |
| 3 | php (7.4) | simps (1.0) | 182 714 | 193 353 | 195 126 |
| 4 | javascript (13.14) | nanoexpress (2.0) | 178 215 | 191 305 | 191 947 |
| 5 | javascript (13.14) | sifrr (0.0) | 177 592 | 190 717 | 190 183 |
| 6 | kotlin (1.3) | kooby (2.8) | 167 887 | 188 125 | 189 999 |
| 7 | java (8) | jooby (2.8) | 164 958 | 184 422 | 187 137 |
| 8 | go (1.14) | fasthttp (1.14) | 163 996 | 173 869 | 177 604 |
| 9 | go (1.14) | atreugo (11.3) | 160 711 | 169 474 | 171 901 |
| 10 | java (8) | rapidoid (5.5) | 160 630 | 176 516 | 176 459 |
| 11 | go (1.14) | fiber (1.11) | 160 131 | 165 519 | 167 631 |
| 12 | go (1.14) | router (1.1) | 158 643 | 166 114 | 170 884 |
| 13 | php (7.4) | workerman (4.0) | 158 109 | 171 231 | 172 110 |
| 14 | crystal (0.34) | router.cr (0.2) | 158 056 | 165 597 | 162 308 |
| 15 | go (1.14) | gorouter-fasthttp (4.4) | 157 879 | 166 976 | 171 004 |
| 16 | crystal (0.34) | toro (0.4) | 157 664 | 164 912 | 161 384 |
| 17 | crystal (0.34) | spider-gazelle (3.0) | 154 999 | 161 746 | 158 450 |
| 18 | crystal (0.34) | grip (0.28) | 148 479 | 154 450 | 151 062 |
| 19 | crystal (0.34) | kemal (0.26) | 145 452 | 151 361 | 147 772 |
| 20 | nim (1.2) | jester (0.4) | 143 817 | 154 245 | 154 637 |
| 21 | go (1.14) | fasthttprouter (0.1) | 140 603 | 155 054 | 164 274 |
| 22 | crystal (0.34) | amber (0.34) | 139 625 | 144 992 | 141 041 |
| 23 | rust (1.44) | actix (2.0) | 138 199 | 139 565 | 139 160 |
| 24 | crystal (0.34) | orion (2.3) | 133 163 | 137 523 | 132 638 |
| 25 | crystal (0.34) | athena (0.8) | 124 081 | 125 307 | 118 581 |
| 26 | java (8) | act (1.8) | 121 752 | 135 304 | 135 362 |
| 27 | go (1.14) | gin (1.6) | 111 770 | 115 988 | 119 880 |
| 28 | go (1.14) | rte (0.0) | 109 812 | 108 343 | 109 053 |
| 29 | go (1.14) | httprouter (1.3) | 109 405 | 108 393 | 112 120 |
| 30 | go (1.14) | echo (4.1) | 108 842 | 109 434 | 112 784 |
| 31 | go (1.14) | chi (4.1) | 105 003 | 103 094 | 106 642 |
| 32 | go (1.14) | gorilla-mux (1.7) | 103 761 | 100 687 | 106 125 |
| 33 | go (1.14) | gorouter (4.4) | 103 206 | 106 725 | 110 021 |
| 34 | go (1.14) | aero (1.3) | 102 770 | 102 695 | 105 847 |
| 35 | fsharp (4.7) | frank (6.0) | 101 900 | 109 250 | 112 410 |
| 36 | go (1.14) | kami (2.2) | 101 337 | 102 172 | 103 348 |
| 37 | csharp (8.0) | aspnetcore (3.1) | 101 209 | 107 750 | 105 295 |
| 38 | go (1.14) | webgo (4.0) | 100 445 | 99 701 | 102 956 |
| 39 | go (1.14) | goroute (0.0) | 98 953 | 96 279 | 96 652 |
| 40 | c (99) | kore (3.3) | 98 208 | 110 597 | 136 471 |
| 41 | go (1.14) | beego (1.12) | 97 372 | 99 756 | 102 982 |
| 42 | ruby (2.7) | agoo (2.12) | 95 935 | 120 197 | 123 520 |
| 43 | go (1.14) | violetear (7.0) | 92 559 | 87 115 | 92 062 |
| 44 | fsharp (4.7) | falco (1.1) | 92 324 | 101 172 | 103 084 |
| 45 | javascript (13.14) | polkadot (1.0) | 89 858 | 98 680 | 97 916 |
| 46 | cpp (14/17) | drogon (1.0) | 87 819 | 91 765 | 93 877 |
| 47 | fsharp (4.7) | giraffe (4.1) | 85 987 | 84 939 | 82 019 |
| 48 | javascript (13.14) | 0http (2.5) | 85 979 | 95 009 | 94 430 |
| 49 | go (1.14) | air (0.17) | 84 820 | 86 096 | 88 055 |
| 50 | elixir (1.1) | cowboy_stream (2.8) | 80 943 | 80 976 | 78 642 |
| 51 | javascript (13.14) | polka (0.5) | 80 856 | 85 176 | 83 515 |
| 52 | javascript (13.14) | rayo (1.3) | 80 774 | 86 609 | 83 791 |
| 53 | javascript (13.14) | restana (4.6) | 79 817 | 87 938 | 86 889 |
| 54 | scala (2.13) | akkahttp (10.1) | 77 296 | 83 830 | 80 969 |
| 55 | go (1.14) | gf (1.13) | 76 518 | 81 667 | 83 079 |
| 56 | swift (5.2) | perfect (3.1) | 76 458 | 84 424 | 89 853 |
| 57 | java (8) | javalin (3.8) | 74 698 | 79 946 | 79 454 |
| 58 | python (3.8) | falcon (2.0) | 72 868 | 77 248 | 77 142 |
| 59 | javascript (13.14) | muneem (2.4) | 71 727 | 76 807 | 75 170 |
| 60 | kotlin (1.3) | ktor (1.2) | 67 833 | 80 300 | 80 541 |
| 61 | java (8) | spring-boot (2.2) | 67 306 | 73 504 | 72 970 |
| 62 | php (7.4) | siler-swoole (1.7) | 67 159 | 74 275 | 76 947 |
| 63 | php (7.4) | hyperf (1.1) | 66 431 | 69 841 | 70 451 |
| 64 | javascript (13.14) | fastify (2.14) | 65 661 | 69 185 | 66 618 |
| 65 | go (1.14) | mars (1.0) | 65 166 | 64 342 | 67 261 |
| 66 | javascript (13.14) | foxify (0.1) | 64 276 | 69 046 | 68 525 |
| 67 | haskell (8.8) | scotty (0.12) | 63 928 | 65 528 | 69 887 |
| 68 | php (7.4) | one (2.0) | 63 005 | 68 108 | 68 002 |
| 69 | java (8) | micronaut (1.2) | 62 924 | 70 019 | 69 841 |
| 70 | javascript (13.14) | koa (2.12) | 61 072 | 63 648 | 61 845 |
| 71 | javascript (13.14) | nestjs-fastify (7.1) | 59 850 | 64 632 | 62 892 |
| 72 | python (3.8) | bottle (0.12) | 58 403 | 62 402 | 62 444 |
| 73 | javascript (13.14) | iotjs-express (0.0) | 57 655 | 60 118 | 58 356 |
| 74 | java (8) | spring-framework (5.2) | 56 505 | 63 788 | 63 607 |
| 75 | c (11) | agoo-c (0.7) | 56 168 | 182 414 | 172 088 |
| 76 | javascript (13.14) | feathersjs (4.5) | 55 724 | 58 385 | 56 961 |
| 77 | python (3.8) | apidaora (0.26) | 55 655 | 61 570 | 61 845 |
| 78 | swift (5.2) | kitura-nio (2.9) | 55 301 | 56 390 | 55 590 |
| 79 | clojure (1.1) | coast (1.0) | 54 803 | 56 142 | 55 837 |
| 80 | swift (5.2) | kitura (2.9) | 54 658 | 54 498 | 54 170 |
| 81 | rust (1.44) | nickel (0.11) | 53 690 | 54 196 | 53 087 |
| 82 | javascript (13.14) | express (4.17) | 52 490 | 54 566 | 53 583 |
| 83 | rust (1.44) | gotham (0.4) | 51 953 | 58 449 | 60 603 |
| 84 | python (3.8) | asgineer (0.7) | 51 851 | 57 062 | 56 958 |
| 85 | scala (2.13) | http4s (0.21) | 51 443 | 52 752 | 50 695 |
| 86 | elixir (1.1) | cowboy (2.8) | 49 913 | 50 093 | 50 442 |
| 87 | python (3.8) | pyramid (1.1) | 49 432 | 51 532 | 51 289 |
| 88 | swift (5.2) | vapor (4.8) | 48 415 | 50 269 | 49 918 |
| 89 | python (3.8) | blacksheep (0.2) | 48 401 | 53 110 | 53 262 |
| 90 | javascript (13.14) | moleculer (0.14) | 48 288 | 50 113 | 49 232 |
| 91 | python (3.8) | hug (2.6) | 46 960 | 48 197 | 48 470 |
| 92 | php (7.4) | imi (1.2) | 46 335 | 48 580 | 49 310 |
| 93 | python (3.8) | sanic (20.3) | 45 684 | 50 070 | 49 337 |
| 94 | cpp (11) | evhtp (1.2) | 45 337 | 45 691 | 45 431 |
| 95 | python (3.8) | starlette (0.13) | 44 488 | 48 361 | 47 981 |
| 96 | javascript (13.14) | hapi (19.1) | 42 091 | 44 240 | 43 595 |
| 97 | elixir (1.1) | plug (1.10) | 41 272 | 41 916 | 41 852 |
| 98 | javascript (13.14) | nestjs-express (7.1) | 41 097 | 42 135 | 41 527 |
| 99 | javascript (13.14) | restify (8.5) | 39 739 | 41 541 | 41 151 |
| 100 | php (7.4) | sw-fw-less (preview) | 39 059 | 41 161 | 40 948 |
| 101 | scala (2.13) | play (2.8) | 38 884 | 39 932 | 40 138 |
| 102 | python (3.8) | emmett (2.0) | 38 835 | 41 318 | 41 189 |
| 103 | ruby (2.7) | hanami-api (0.1) | 37 292 | 37 646 | 35 449 |
| 104 | ruby (2.7) | syro (3.2) | 36 955 | 38 241 | 36 680 |
| 105 | elixir (1.1) | phoenix (1.5) | 36 307 | 36 638 | 36 911 |
| 106 | ruby (2.7) | roda (3.32) | 36 099 | 36 959 | 35 168 |
| 107 | php (7.4) | swoft (2.0) | 35 961 | 37 135 | 36 789 |
| 108 | php (7.4) | yii-swoole (2.0) | 34 687 | 35 818 | 35 442 |
| 109 | clojure (1.1) | luminus (1.0) | 33 677 | 35 002 | 34 605 |
| 110 | fsharp (4.7) | saturn (0.13) | 32 531 | 28 911 | 26 237 |
| 111 | fsharp (4.7) | suave (2.5) | 31 997 | 28 676 | 31 950 |
| 112 | dart (2.8) | aqueduct (3.3) | 31 700 | 31 498 | 31 161 |
| 113 | ruby (2.7) | cuba (3.9) | 31 531 | 32 247 | 31 079 |
| 114 | haskell (8.8) | servant (0.17) | 30 333 | 29 797 | 28 615 |
| 115 | crystal (0.34) | shivneri (0.15) | 30 190 | 27 501 | 26 282 |
| 116 | python (3.8) | responder (2.0) | 28 959 | 31 164 | 31 086 |
| 117 | python (3.8) | fastapi (0.55) | 28 304 | 31 342 | 31 086 |
| 118 | ruby (2.7) | rack-routing (0.0) | 27 097 | 27 121 | 26 572 |
| 119 | ruby (2.7) | rack_app (7.6) | 27 026 | 27 016 | 26 147 |
| 120 | python (3.8) | molten (1.0) | 26 312 | 26 915 | 26 518 |
| 121 | python (3.8) | clastic (19.9) | 25 451 | 26 243 | 26 107 |
| 122 | python (3.8) | aiohttp (3.6) | 25 443 | 27 469 | 27 429 |
| 123 | go (1.14) | macaron (1.3) | 25 265 | 27 940 | 28 178 |
| 124 | ruby (2.7) | camping (2.1) | 24 262 | 23 820 | 23 206 |
| 125 | rust (1.44) | iron (0.6) | 23 995 | 24 136 | 24 003 |
| 126 | python (3.8) | masonite (2.3) | 23 748 | 24 357 | 24 170 |
| 127 | python (3.8) | flask (1.1) | 23 594 | 24 229 | 24 050 |
| 128 | javascript (13.14) | turbo_polka (0.3) | 22 182 | 21 377 | 20 366 |
| 129 | php (7.4) | spiral (2.4) | 17 878 | 18 279 | 18 350 |
| 130 | java (8) | blade (2.0) | 17 832 | 19 918 | 19 716 |
| 131 | go (1.14) | tango (0.6) | 17 282 | 17 513 | 17 587 |
| 132 | ruby (2.7) | sinatra (2.0) | 15 190 | 15 450 | 15 356 |
| 133 | go (1.14) | gramework (1.7) | 14 588 | 14 921 | 14 880 |
| 134 | ruby (2.7) | grape (1.3) | 14 306 | 14 323 | 14 348 |
| 135 | dart (2.8) | start (0.4) | 13 907 | 13 633 | 12 663 |
| 136 | javascript (13.14) | sails (1.2) | 12 793 | 13 198 | 13 144 |
| 137 | python (3.8) | quart (0.12) | 12 056 | 11 892 | 11 258 |
| 138 | php (7.4) | fastsitephp (1.2) | 11 498 | 11 427 | 11 354 |
| 139 | ruby (2.7) | flame (4.18) | 11 444 | 11 218 | 11 141 |
| 140 | pony (0.35) | jennet (0.1) | 11 294 | 17 581 | 15 969 |
| 141 | swift (5.2) | swifter (1.4) | 11 048 | 11 119 | 11 097 |
| 142 | ruby (2.7) | hanami (1.3) | 10 592 | 10 549 | 10 465 |
| 143 | python (3.8) | tornado (6.0) | 10 016 | 10 159 | 10 010 |
| 144 | php (7.4) | ubiquity (2.3) | 9 345 | 9 372 | 9 366 |
| 145 | python (3.8) | django (3.0) | 9 281 | 9 329 | 9 170 |
| 146 | python (3.8) | cherrypy (18.6) | 9 214 | 9 051 | 8 977 |
| 147 | php (7.4) | one-fpm (2.0) | 8 111 | 8 095 | 8 073 |
| 148 | php (7.4) | siler (1.7) | 7 807 | 7 850 | 7 816 |
| 149 | php (7.4) | phalcon (4.0) | 7 689 | 7 700 | 7 695 |
| 150 | php (7.4) | hamlet (3.2) | 7 612 | 7 697 | 7 707 |
| 151 | php (7.4) | ice (1.5) | 6 387 | 6 427 | 6 522 |
| 152 | php (7.4) | chubbyphp (2.8) | 5 737 | 5 750 | 5 840 |
| 153 | crystal (0.34) | onyx (0.5) | 5 621 | 5 767 | 5 801 |
| 154 | php (7.4) | slim (4.5) | 4 715 | 4 796 | 4 946 |
| 155 | php (7.4) | nette (3.0) | 4 406 | 4 466 | 4 592 |
| 156 | php (7.4) | lumen (7.1) | 4 311 | 4 339 | 4 464 |
| 157 | php (7.4) | yii (2.0) | 4 198 | 4 307 | 4 363 |
| 158 | ruby (2.7) | rails (6.0) | 3 809 | 3 599 | 3 593 |
| 159 | julia (1.5) | merly (0.2) | 3 380 | 7 914 | 6 121 |
| 160 | php (7.4) | symfony (5.1) | 3 202 | 3 245 | 3 322 |
| 161 | php (7.4) | mezzio (3.2) | 2 875 | 2 958 | 2 980 |
| 162 | perl (5.3) | dancer2 (2.0) | 2 537 | 2 702 | 1 122 |
| 163 | python (3.8) | cyclone (1.3) | 2 420 | 2 422 | 2 420 |
| 164 | python (3.8) | klein (20.6) | 1 624 | 1 636 | 1 627 |
| 165 | python (3.8) | nameko (2.12) | 1 557 | 1 507 | 1 502 |
| 166 | php (7.4) | laminas (3.1) | 1 450 | 1 478 | 1 485 |
| 167 | crystal (0.34) | lucky (0.21) | 1 413 | 1 435 | 1 434 |
| 168 | php (7.4) | laravel (7.14) | 586 | 162 | 1 970 |
| 169 | php (7.4) | basicphp (0.9) | 548 | 478 | 1 410 |
In any way you want ...
- Request a framework addition
- Report a bug (on any implementation)
- Suggest an idea
- ...
Any kind of idea is ❤️
- Taichiro Suzuki - Author | Maintainer
- OvermindDL1 - Maintainer
- Marwan Rabbâa - Maintainer