This project aims to be a load benchmarking suite, no more, no less
Measuring response times (routing times) for each framework (middleware).
Results are not production-ready yet
- Helping decide between languages, depending on use case
- Learning languages, best practices, devops culture ...
- Having fun ❤️
- Crystal as
built-intools are made in this language - Docker as frameworks are
isolatedinto containers - wrk as benchmarking tool,
>= 4.1.0 - postgresql to store data,
>= 10
ℹ️ you need wrk stable
git clone --branch 4.1.0 https://github.com/wg/wrkdocker is used for development purpose, production results will be computed on DigitalOcean
- Install all dependencies
shards install- Build internal tools
shards build- Create and initialize the database
createdb -U postgres benchmark
psql -U postgres -d benchmark < .ci/dump.sqlDocker can be used to set up the database:
docker run -it --rm -d \
-p 5432:5432 \
-e POSTGRES_DB=benchmark \
-e POSTGRES_HOST_AUTH_METHOD=trust \
-v /tmp/pg-data:/var/lib/postgresql/data \
--name pg postgres:12-alpineWait several seconds for the container to start, then inject the dump:
docker exec pg sh -c "echo \"$(cat .ci/dump.sql)\" | psql -U postgres -d benchmark"After creating the database, export its URL:
export DATABASE_URL="postgresql://postgres@localhost/benchmark"- Make configuration
bin/make config- Build containers
jobs are either languages (example : crystal) or frameworks (example : router.cr)
bin/neph [job1] [job2] [job3] ...- Export all results readme
bin/db to_readmeℹ️ Updated on 2020-03-21 ℹ️
Benchmarking with wrk
- Threads : 8
- Timeout : 8
- Duration : 15s (seconds)
ℹ️ Sorted by max req/s on concurrency 64 ℹ️
| Language | Framework | Speed (64) | Speed (256) | Speed (512) | Speed (1024) | Speed (2048) | |
|---|---|---|---|---|---|---|---|
| 1 | nim (1.0) | httpbeast (0.2) | 183 311 | 196 756 | 198 580 | 195 282 | 195 309 |
| 2 | javascript (13.7) | sifrr (0.0) | 180 473 | 193 511 | 193 323 | 189 919 | 190 660 |
| 3 | javascript (13.7) | nanoexpress-pro (1.1) | 174 528 | 186 721 | 182 928 | 187 247 | 182 666 |
| 4 | go (1.14) | fasthttp (1.9) | 163 475 | 171 750 | 173 918 | 169 992 | 169 576 |
| 5 | javascript (13.7) | nanoexpress (1.1) | 163 443 | 180 210 | 180 052 | 174 355 | 177 034 |
| 6 | go (1.14) | fasthttprouter (0.1) | 157 613 | 166 134 | 168 271 | 163 778 | 163 430 |
| 7 | go (1.14) | router (0.7) | 157 600 | 165 914 | 168 197 | 164 120 | 163 804 |
| 8 | go (1.14) | atreugo (10.4) | 156 760 | 164 770 | 167 327 | 163 331 | 162 710 |
| 9 | go (1.14) | fiber (1.8) | 156 693 | 166 800 | 164 681 | 155 386 | 154 025 |
| 10 | crystal (0.33) | toro (0.4) | 155 952 | 164 143 | 160 897 | 152 242 | 151 612 |
| 11 | go (1.14) | gorouter-fasthttp (4.4) | 155 382 | 163 425 | 165 682 | 161 294 | 160 839 |
| 12 | java (8) | rapidoid (5.5) | 155 300 | 169 051 | 171 010 | 167 053 | 167 072 |
| 13 | crystal (0.33) | router.cr (0.2) | 155 294 | 162 885 | 160 659 | 151 157 | 151 055 |
| 14 | crystal (0.33) | spider-gazelle (2.3) | 151 418 | 158 300 | 155 352 | 145 977 | 145 891 |
| 15 | php (7.4) | workerman (4.0) | 146 977 | 157 464 | 158 882 | 157 283 | 157 006 |
| 16 | crystal (0.33) | grip (0.28) | 145 763 | 152 517 | 148 582 | 140 198 | 139 117 |
| 17 | crystal (0.33) | kemal (0.26) | 144 654 | 151 945 | 148 037 | 139 799 | 138 193 |
| 18 | nim (1.0) | jester (0.4) | 144 057 | 154 516 | 155 335 | 151 553 | 151 745 |
| 19 | crystal (0.33) | amber (0.33) | 136 702 | 142 972 | 138 890 | 129 404 | 129 430 |
| 20 | crystal (0.33) | lucky (0.18) | 135 172 | 139 221 | 134 528 | 123 838 | 125 426 |
| 21 | crystal (0.33) | orion (2.1) | 132 718 | 136 796 | 131 319 | 120 902 | 120 148 |
| 22 | crystal (0.33) | athena (0.8) | 126 854 | 129 440 | 123 129 | 110 867 | 110 924 |
| 23 | java (8) | act (1.8) | 119 062 | 130 618 | 130 711 | 127 746 | 127 284 |
| 24 | go (1.14) | rte (0.0) | 111 097 | 110 181 | 113 492 | 113 325 | 113 070 |
| 25 | go (1.14) | httprouter (1.3) | 109 322 | 108 482 | 111 757 | 111 644 | 111 172 |
| 26 | go (1.14) | chi (4.0) | 104 669 | 103 122 | 106 196 | 106 042 | 105 482 |
| 27 | go (1.14) | gorouter (4.4) | 104 012 | 107 875 | 110 677 | 109 771 | 109 521 |
| 28 | go (1.14) | aero (1.3) | 103 642 | 103 223 | 105 915 | 104 967 | 104 813 |
| 29 | go (1.14) | violetear (7.0) | 102 144 | 101 539 | 105 017 | 105 104 | 104 802 |
| 30 | go (1.14) | echo (4.1) | 100 257 | 98 774 | 101 913 | 102 211 | 102 004 |
| 31 | go (1.14) | goroute (0.0) | 100 126 | 98 379 | 101 378 | 101 962 | 101 742 |
| 32 | go (1.14) | kami (2.2) | 99 758 | 103 430 | 104 985 | 103 298 | 103 001 |
| 33 | ruby (2.7) | agoo (2.12) | 98 417 | 120 215 | 125 072 | 125 176 | 124 597 |
| 34 | go (1.14) | gorilla-mux (1.7) | 97 314 | 95 604 | 98 713 | 99 235 | 98 902 |
| 35 | go (1.14) | beego (1.12) | 97 016 | 100 360 | 103 549 | 103 221 | 102 790 |
| 36 | csharp (8.0) | aspnetcore (3.1) | 96 821 | 104 852 | 105 447 | 102 601 | 103 044 |
| 37 | go (1.14) | gin (1.5) | 96 127 | 99 270 | 101 595 | 101 793 | 101 530 |
| 38 | go (1.14) | webgo (3.0) | 94 837 | 93 571 | 96 997 | 97 541 | 97 266 |
| 39 | c (99) | kore (3.3) | 91 570 | 157 941 | 157 570 | 156 352 | 147 974 |
| 40 | javascript (13.7) | polkadot (1.0) | 89 819 | 100 738 | 101 326 | 96 351 | 96 820 |
| 41 | cpp (14/17) | drogon (1.0) | 88 286 | 91 531 | 92 457 | 93 056 | 92 279 |
| 42 | javascript (13.7) | 0http (2.2) | 84 696 | 93 695 | 95 814 | 91 279 | 89 801 |
| 43 | go (1.14) | air (0.15) | 83 223 | 87 023 | 89 244 | 89 249 | 89 032 |
| 44 | javascript (13.7) | restana (4.1) | 83 198 | 90 425 | 89 107 | 85 369 | 85 132 |
| 45 | javascript (13.7) | rayo (1.3) | 83 171 | 87 320 | 84 916 | 82 269 | 81 566 |
| 46 | javascript (13.7) | polka (0.5) | 82 224 | 87 898 | 85 434 | 82 197 | 79 916 |
| 47 | go (1.14) | gf (1.11) | 76 779 | 81 949 | 83 564 | 83 615 | 83 386 |
| 48 | java (8) | javalin (3.5) | 74 824 | 78 927 | 78 738 | 75 161 | 76 548 |
| 49 | scala (2.12) | akkahttp (10.1) | 73 386 | 79 793 | 76 646 | 75 118 | 75 791 |
| 50 | python (3.8) | falcon (2.0) | 71 383 | 77 827 | 77 713 | 76 403 | 76 363 |
| 51 | java (8) | spring-boot (2.1) | 71 016 | 77 060 | 74 929 | 69 287 | 70 178 |
| 52 | javascript (13.7) | foxify (0.1) | 66 075 | 67 872 | 63 758 | 63 988 | 64 060 |
| 53 | javascript (13.7) | muneem (2.4) | 65 400 | 70 335 | 68 807 | 66 296 | 66 441 |
| 54 | php (7.4) | one (2.0) | 64 729 | 69 020 | 69 116 | 68 283 | 68 064 |
| 55 | elixir (1.1) | cowboy_stream (2.7) | 64 575 | 65 454 | 64 879 | 63 311 | 63 329 |
| 56 | kotlin (1.3) | ktor (1.2) | 64 476 | 76 911 | 78 410 | 74 786 | 74 168 |
| 57 | javascript (13.7) | fastify (2.13) | 64 457 | 71 281 | 68 226 | 67 351 | 66 360 |
| 58 | swift (5.1) | perfect (3.1) | 63 350 | 71 527 | 76 342 | 76 440 | 78 058 |
| 59 | go (1.14) | mars (1.0) | 61 654 | 64 963 | 67 779 | 67 705 | 67 658 |
| 60 | javascript (13.7) | koa (2.11) | 61 285 | 63 945 | 62 035 | 60 821 | 61 104 |
| 61 | java (8) | micronaut (1.2) | 60 876 | 67 770 | 69 199 | 66 553 | 68 230 |
| 62 | python (3.8) | bottle (0.12) | 59 913 | 62 900 | 62 565 | 61 756 | 61 534 |
| 63 | c (11) | agoo-c (0.7) | 57 393 | 174 634 | 184 324 | 184 173 | 180 275 |
| 64 | python (3.8) | apidaora (0.14) | 57 197 | 63 628 | 63 178 | 60 701 | 60 556 |
| 65 | javascript (13.7) | iotjs-express (0.0) | 57 097 | 61 558 | 58 565 | 57 126 | 56 895 |
| 66 | rust (1.41) | nickel (0.11) | 56 885 | 57 541 | 55 657 | 56 255 | 56 277 |
| 67 | java (8) | spring-framework (5.2) | 56 684 | 63 914 | 63 482 | 61 890 | 61 913 |
| 68 | clojure (1.1) | coast (1.0) | 53 714 | 54 828 | 54 633 | 54 534 | 54 453 |
| 69 | scala (2.12) | http4s (0.18) | 51 511 | 58 460 | 59 343 | 60 945 | 60 666 |
| 70 | javascript (13.7) | express (4.17) | 50 565 | 54 086 | 53 157 | 51 729 | 50 140 |
| 71 | python (3.8) | blacksheep (0.2) | 50 474 | 54 762 | 54 147 | 52 265 | 52 404 |
| 72 | javascript (13.7) | moleculer (0.14) | 50 398 | 51 874 | 50 782 | 50 014 | 49 994 |
| 73 | php (7.4) | hyperf (1.0) | 48 727 | 50 737 | 50 701 | 50 077 | 50 125 |
| 74 | swift (5.1) | kitura (2.8) | 48 422 | 47 788 | 48 416 | 47 470 | 46 984 |
| 75 | python (3.8) | asgineer (0.7) | 48 404 | 52 459 | 51 854 | 50 124 | 50 058 |
| 76 | swift (5.1) | vapor (3.3) | 47 878 | 47 992 | 48 277 | 47 921 | 47 503 |
| 77 | swift (5.1) | kitura-nio (2.8) | 47 606 | 49 185 | 47 492 | 46 829 | 46 688 |
| 78 | python (3.8) | pyramid (1.1) | 46 768 | 52 451 | 51 911 | 51 646 | 51 428 |
| 79 | cpp (11) | evhtp (1.2) | 46 284 | 46 113 | 45 953 | 45 889 | 45 240 |
| 80 | python (3.8) | starlette (0.13) | 45 734 | 49 321 | 48 812 | 47 171 | 47 261 |
| 81 | python (3.8) | hug (2.6) | 44 720 | 46 672 | 46 373 | 45 550 | 45 887 |
| 82 | javascript (13.7) | hapi (19.1) | 43 557 | 45 374 | 44 272 | 44 197 | 43 951 |
| 83 | elixir (1.1) | cowboy (2.7) | 42 503 | 43 536 | 43 083 | 42 197 | 42 043 |
| 84 | php (7.4) | imi (1.0) | 41 478 | 43 033 | 43 406 | 43 153 | 43 198 |
| 85 | javascript (13.7) | restify (8.5) | 41 372 | 43 070 | 41 746 | 41 730 | 41 701 |
| 86 | php (7.4) | sw-fw-less (preview) | 41 250 | 43 279 | 42 975 | 42 389 | 42 317 |
| 87 | rust (1.41) | gotham (0.4) | 41 192 | 49 295 | 50 829 | 52 961 | 52 792 |
| 88 | ruby (2.7) | hanami-api (0.1) | 40 547 | 42 158 | 39 707 | 39 165 | 38 947 |
| 89 | python (3.8) | emmett (2.0.0b1) | 39 108 | 42 006 | 41 665 | 40 296 | 40 464 |
| 90 | ruby (2.7) | syro (3.2) | 38 923 | 43 244 | 41 170 | 40 297 | 40 501 |
| 91 | ruby (2.7) | roda (3.3) | 38 352 | 39 554 | 37 995 | 37 246 | 37 771 |
| 92 | elixir (1.1) | plug (1.8) | 37 651 | 38 396 | 37 850 | 37 076 | 36 999 |
| 93 | php (7.4) | swoft (2.0) | 36 264 | 37 011 | 36 682 | 36 155 | 36 234 |
| 94 | ruby (2.7) | cuba (3.9) | 35 267 | 35 484 | 34 455 | 34 074 | 34 164 |
| 95 | dart (2.7) | aqueduct (3.2) | 33 335 | 32 357 | 32 878 | 31 809 | 31 940 |
| 96 | elixir (1.1) | phoenix (1.4) | 30 789 | 32 803 | 32 406 | 31 796 | 31 751 |
| 97 | python (3.8) | responder (2.0) | 29 727 | 31 424 | 31 028 | 29 822 | 29 869 |
| 98 | python (3.8) | fastapi (0.52) | 29 508 | 31 294 | 31 023 | 29 737 | 29 749 |
| 99 | fsharp (4.7) | suave (2.5) | 29 114 | 27 415 | 33 419 | 32 469 | 31 665 |
| 100 | ruby (2.7) | rack-routing (0.0) | 28 066 | 28 381 | 27 896 | 27 830 | 27 984 |
| 101 | python (3.8) | molten (1.0) | 26 925 | 27 693 | 27 209 | 27 223 | 26 973 |
| 102 | python (3.8) | clastic (19.9) | 26 092 | 26 556 | 26 420 | 26 408 | 26 310 |
| 103 | ruby (2.7) | camping (2.1) | 25 803 | 25 475 | 25 122 | 25 043 | 25 280 |
| 104 | python (3.8) | aiohttp (3.6) | 25 730 | 28 028 | 27 905 | 26 842 | 27 124 |
| 105 | python (3.8) | masonite (2.3) | 24 025 | 24 833 | 24 668 | 24 355 | 23 845 |
| 106 | python (3.8) | flask (1.1) | 23 811 | 24 706 | 24 393 | 24 018 | 24 235 |
| 107 | javascript (13.7) | turbo_polka (0.3) | 23 061 | 21 118 | 20 503 | 20 470 | 20 405 |
| 108 | python (3.8) | sanic (19.12) | 21 529 | 21 840 | 21 195 | 20 542 | 20 394 |
| 109 | rust (1.41) | iron (0.6) | 18 322 | 18 492 | 18 389 | 18 508 | 18 260 |
| 110 | php (7.4) | spiral (2.4) | 17 639 | 18 028 | 18 125 | 17 986 | 18 077 |
| 111 | ruby (2.7) | sinatra (2.0) | 16 170 | 16 094 | 16 052 | 16 030 | 16 010 |
| 112 | go (1.14) | gramework (1.7) | 14 956 | 15 430 | 15 386 | 15 330 | 15 397 |
| 113 | ruby (2.7) | grape (1.3) | 14 870 | 14 746 | 14 698 | 14 641 | 14 657 |
| 114 | ruby (2.7) | flame (4.18) | 13 393 | 13 114 | 13 117 | 13 071 | 13 033 |
| 115 | ruby (2.7) | hanami (1.3) | 12 026 | 11 978 | 11 957 | 11 934 | 11 980 |
| 116 | python (3.8) | quart (0.11) | 11 238 | 12 166 | 11 400 | 10 867 | 10 627 |
| 117 | swift (5.1) | swifter (1.4) | 10 854 | 11 141 | 11 074 | 10 809 | 11 210 |
| 118 | python (3.8) | django (3.0) | 10 204 | 10 348 | 10 289 | 10 238 | 10 250 |
| 119 | php (7.4) | ubiquity (2.3) | 9 104 | 9 010 | 9 127 | 51 769 | 52 415 |
| 120 | python (3.8) | cherrypy (18.5) | 8 042 | 8 353 | 8 254 | 8 202 | 8 219 |
| 121 | php (7.4) | one-fpm (2.0) | 7 868 | 7 844 | 7 774 | 43 856 | 43 118 |
| 122 | python (3.8) | tornado (6.0) | 7 627 | 10 514 | 10 184 | 10 092 | 10 118 |
| 123 | php (7.4) | phalcon (4.0) | 7 463 | 7 471 | 7 524 | 52 154 | 48 362 |
| 124 | php (7.4) | hamlet (3.2) | 7 459 | 7 463 | 7 515 | 43 808 | 44 781 |
| 125 | php (7.4) | chubbyphp (2.8) | 5 605 | 5 570 | 5 591 | 44 556 | 42 802 |
| 126 | crystal (0.33) | onyx (0.5) | 4 794 | 4 898 | 4 917 | 4 950 | 4 918 |
| 127 | php (7.4) | slim (4.4) | 4 612 | 4 622 | 4 736 | 44 003 | 41 856 |
| 128 | php (7.4) | yii (2.0) | 4 326 | 4 362 | 4 416 | 43 618 | 40 804 |
| 129 | php (7.4) | lumen (7.0) | 4 304 | 4 317 | 4 375 | 43 353 | 41 555 |
| 130 | ruby (2.7) | rails (6.0) | 3 981 | 3 783 | 3 774 | 3 748 | 3 730 |
| 131 | php (7.4) | symfony (4.3) | 3 607 | 3 628 | 3 669 | 42 672 | 38 941 |
| 132 | php (7.4) | mezzio (3.2) | 2 812 | 2 832 | 2 899 | 41 735 | 40 543 |
| 133 | python (3.8) | cyclone (1.3) | 2 425 | 2 449 | 2 420 | 2 409 | 2 393 |
| 134 | julia (1.3) | merly (0.2) | 2 131 | 5 475 | 4 352 | 3 467 | 1 783 |
| 135 | python (3.8) | klein (19.6) | 1 641 | 1 653 | 1 624 | 1 608 | 1 620 |
| 136 | python (3.8) | nameko (2.12) | 1 529 | 1 498 | 1 463 | 1 460 | 1 468 |
| 137 | php (7.4) | laminas (3.1) | 1 426 | 1 457 | 1 509 | 39 093 | 38 381 |
| 138 | perl (5.3) | dancer2 (2.0) | 887 | 702 | 1 357 | 1 625 | 486 |
| 139 | php (7.4) | basicphp (0.9) | 482 | 515 | 2 500 | 35 114 | 33 763 |
| 140 | php (7.4) | laravel (7.2) | 274 | 158 | 2 278 | 23 201 | 21 877 |
In any way you want ...
- Request a framework addition
- Report a bug (on any implementation)
- Suggest an idea
- ...
Any kind of idea is ❤️
- Taichiro Suzuki - Author | Maintainer
- OvermindDL1 - Maintainer
- Marwan Rabbâa - Maintainer