Which is the fastest?
This project aims to be a load benchmarking suite, no more, no less
Measuring response times (routing times) for each framework (middleware).
Results are not production-ready yet
Additional purposes :
- Helping decide between languages, depending on use case
- Learning languages, best practices, devops culture ...
- Having fun ❤️
Requirements
- Crystal as
built-intools are made in this language - Docker as frameworks are
isolatedinto containers - wrk as benchmarking tool,
>= 4.1.0 - postgresql to store data,
>= 10
ℹ️ℹ️ℹ️ℹ️ℹ️
OSX you need docker-machine to use docker containerization
brew install docker-machine
docker-machine create default
eval $(docker-machine env default)
ℹ️ℹ️ℹ️ℹ️ℹ️
Usage
- Install all dependencies
shards install- Build internal tools
shards build- Create and initialize the database
createdb -U postgres benchmark
psql -U postgres -d benchmark < .ci/dump.sqlDocker can be used to set up the database:
docker run -it --rm -d \
-p 5432:5432 \
-e POSTGRES_DB=benchmark \
-e POSTGRES_HOST_AUTH_METHOD=trust \
-v /tmp/pg-data:/var/lib/postgresql/data \
--name pg postgres:12-alpineWait several seconds for the container to start, then inject the dump:
docker exec pg sh -c "echo \"$(cat .ci/dump.sql)\" | psql -U postgres -d benchmark"After creating the database, export its URL:
export DATABASE_URL="postgresql://postgres@localhost/benchmark"- Make configuration
bin/make config- Build containers
jobs are either languages (example : crystal) or frameworks (example : router.cr)
bin/neph [job1] [job2] [job3] ...- Export all results readme
bin/db to_readmeResults
ℹ️ Updated on 2020-11-05 ℹ️
Benchmarking with wrk
- Threads : 8
- Timeout : 8
- Duration : 15s (seconds)
ℹ️ Sorted by max req/s on concurrency 64 ℹ️
| Language | Framework | Speed (64) | Speed (256) | Speed (512) | |
|---|---|---|---|---|---|
| 1 | go (1.15) | fiber (2.0) | 169 728 | 181 862 | 183 418 |
| 2 | go (1.15) | fasthttp (1.16) | 169 038 | 183 813 | 184 384 |
| 3 | php (7.4) | mark (1.1) | 168 006 | 172 705 | 172 622 |
| 4 | go (1.15) | gearbox (1.1) | 167 604 | 177 253 | 177 806 |
| 5 | go (1.15) | gorouter-fasthttp (4.4) | 167 276 | 181 708 | 179 071 |
| 6 | go (1.15) | router (1.3) | 166 242 | 180 922 | 179 163 |
| 7 | nim (1.2) | httpbeast (0.2) | 166 147 | 199 087 | 201 467 |
| 8 | go (1.15) | atreugo (11.5) | 165 818 | 180 873 | 180 462 |
| 9 | java (11) | rapidoid (5.5) | 162 464 | 193 489 | 186 348 |
| 10 | nim (1.2) | whip (0.2) | 161 943 | 192 355 | 193 779 |
| 11 | java (11) | jooby (2.8) | 158 858 | 199 473 | 206 854 |
| 12 | javascript (12.18) | sifrr (0.0) | 154 386 | 194 178 | 195 460 |
| 13 | java (11) | light-4j (2.0) | 153 823 | 193 652 | 198 295 |
| 14 | nim (1.2) | jester (0.4) | 150 971 | 178 514 | 181 225 |
| 15 | kotlin (1.4) | kooby (2.8) | 146 865 | 186 795 | 189 715 |
| 16 | crystal (0.35) | toro (0.4) | 144 271 | 169 077 | 167 665 |
| 17 | php (7.4) | workerman (4.0) | 144 193 | 176 854 | 177 918 |
| 18 | crystal (0.35) | router.cr (0.2) | 141 048 | 169 319 | 165 659 |
| 19 | crystal (0.35) | spider-gazelle (3.3) | 140 079 | 166 026 | 165 199 |
| 20 | java (11) | act (1.9) | 134 049 | 164 079 | 169 434 |
| 21 | cpp (14/17) | drogon (1.0) | 132 436 | 140 281 | 141 219 |
| 22 | crystal (0.35) | grip (4.0) | 132 390 | 156 974 | 154 731 |
| 23 | crystal (0.35) | kemal (0.26) | 127 799 | 153 561 | 155 024 |
| 24 | crystal (0.35) | amber (0.35) | 125 367 | 145 804 | 145 287 |
| 25 | c (99) | kore (3.3) | 123 255 | 184 928 | 189 632 |
| 26 | javascript (12.18) | naturaljs-router (0.1.1) | 119 972 | 143 556 | 143 850 |
| 27 | crystal (0.35) | athena (0.9) | 115 794 | 131 125 | 120 210 |
| 28 | go (1.15) | rte (0.0) | 115 700 | 115 575 | 118 922 |
| 29 | go (1.15) | clevergo (0.3) | 114 996 | 115 050 | 119 045 |
| 30 | go (1.15) | httprouter (1.3) | 114 447 | 113 525 | 116 939 |
| 31 | go (1.15) | echo (4.1) | 114 316 | 114 682 | 117 687 |
| 32 | go (1.15) | gorouter (4.5) | 112 304 | 116 445 | 119 310 |
| 33 | go (1.15) | gin (1.6) | 111 748 | 115 464 | 117 430 |
| 34 | rust (1.46) | actix (3.1) | 110 783 | 115 158 | 106 450 |
| 35 | go (1.15) | chi (4.1) | 110 000 | 106 608 | 110 798 |
| 36 | go (1.15) | violetear (7.0) | 109 860 | 109 274 | 113 235 |
| 37 | go (1.15) | webgo (4.1) | 109 245 | 106 991 | 111 010 |
| 38 | php (7.4) | simps (1.0) | 108 668 | 166 150 | 181 713 |
| 39 | go (1.15) | aero (1.3) | 107 303 | 104 847 | 108 944 |
| 40 | go (1.15) | kami (2.2) | 105 662 | 112 320 | 113 686 |
| 41 | go (1.15) | goroute (0.0) | 104 771 | 100 765 | 104 725 |
| 42 | go (1.15) | gorilla-mux (1.8) | 103 807 | 98 155 | 103 369 |
| 43 | go (1.15) | beego (1.12) | 101 686 | 104 852 | 108 363 |
| 44 | csharp (8.0) | aspnetcore (3.1) | 97 248 | 118 985 | 117 064 |
| 45 | fsharp (4.7) | falco (1.2) | 97 192 | 111 765 | 115 443 |
| 46 | fsharp (4.7) | frank (6.1) | 96 423 | 113 437 | 116 546 |
| 47 | go (1.15) | air (0.19) | 89 138 | 87 241 | 91 308 |
| 48 | php (7.4) | nano (0.0.9) | 89 044 | 132 366 | 140 019 |
| 49 | javascript (12.18) | polkadot (1.0) | 88 041 | 97 685 | 98 281 |
| 50 | go (1.15) | goyave (3.1) | 86 588 | 85 359 | 89 957 |
| 51 | java (11) | spring-boot (2.3) | 83 893 | 90 151 | 89 750 |
| 52 | javascript (12.18) | 0http (3.0) | 83 209 | 93 595 | 94 160 |
| 53 | php (7.4) | hyperf (2.0) | 82 731 | 85 087 | 92 148 |
| 54 | javascript (12.18) | polka (0.5) | 81 859 | 88 753 | 87 344 |
| 55 | java (11) | javalin (3.9) | 79 878 | 84 494 | 86 392 |
| 56 | java (11) | spring-framework (5.2) | 79 877 | 88 301 | 89 396 |
| 57 | go (1.15) | gf (1.13) | 79 669 | 85 767 | 88 394 |
| 58 | javascript (12.18) | restana (4.7) | 78 959 | 89 408 | 88 791 |
| 59 | javascript (12.18) | rayo (1.3) | 78 812 | 85 096 | 84 152 |
| 60 | swift (5.3) | perfect (3.1) | 78 294 | 88 235 | 95 031 |
| 61 | javascript (12.18) | fastify (3.7) | 78 039 | 86 384 | 84 820 |
| 62 | elixir (1.1) | cowboy_stream (2.8) | 77 969 | 79 844 | 78 394 |
| 63 | csharp (8.0) | carter (5.1) | 77 102 | 81 970 | 74 482 |
| 64 | php (7.4) | one (2.0) | 76 582 | 103 030 | 105 567 |
| 65 | ruby (2.7) | agoo (2.13) | 76 025 | 107 866 | 118 380 |
| 66 | javascript (12.18) | muneem (2.4) | 74 820 | 81 510 | 79 576 |
| 67 | fsharp (4.7) | saturn (0.14) | 74 303 | 76 460 | 68 728 |
| 68 | scala (2.13) | akkahttp (10.1) | 73 927 | 85 451 | 82 910 |
| 69 | haskell (8.8) | scotty (0.12) | 69 559 | 73 490 | 76 953 |
| 70 | javascript (12.18) | nestjs-fastify (7.4) | 69 195 | 72 603 | 72 333 |
| 71 | kotlin (1.4) | ktor (1.4) | 68 148 | 86 493 | 89 921 |
| 72 | python (3.8) | falcon (2.0) | 66 358 | 77 973 | 77 780 |
| 73 | javascript (12.18) | foxify (0.1) | 64 783 | 69 925 | 68 625 |
| 74 | go (1.15) | mars (1.0) | 64 739 | 63 801 | 68 534 |
| 75 | java (11) | micronaut (1.2) | 64 181 | 71 716 | 71 114 |
| 76 | fsharp (4.7) | websharper (4.6) | 63 569 | 68 235 | 67 000 |
| 77 | javascript (12.18) | iotjs-express (0.0) | 63 344 | 69 584 | 67 521 |
| 78 | php (7.4) | comet (0.8) | 62 521 | 68 366 | 67 816 |
| 79 | clojure (1.1) | coast (1.0) | 61 975 | 64 152 | 64 507 |
| 80 | c (11) | agoo-c (0.7) | 59 629 | 170 443 | 194 965 |
| 81 | php (7.4) | ubiquity (2.3) | 59 189 | 61 033 | 59 159 |
| 82 | javascript (12.18) | koa (2.13) | 58 377 | 62 720 | 60 536 |
| 83 | python (3.8) | bottle (0.12) | 57 945 | 63 723 | 63 815 |
| 84 | rust (1.46) | nickel (0.11) | 54 215 | 56 178 | 53 040 |
| 85 | php (7.4) | siler-swoole (1.7) | 53 231 | 79 265 | 85 067 |
| 86 | swift (5.3) | kitura-nio (2.9) | 52 808 | 53 380 | 53 367 |
| 87 | swift (5.3) | kitura (2.9) | 52 544 | 52 772 | 53 458 |
| 88 | cpp (11) | evhtp (1.2) | 50 565 | 52 933 | 52 484 |
| 89 | elixir (1.1) | cowboy (2.8) | 50 325 | 50 318 | 50 471 |
| 90 | php (7.4) | chubbyphp-workerman (3.1) | 50 132 | 55 504 | 56 474 |
| 91 | javascript (12.18) | moleculer (0.14) | 48 522 | 50 501 | 49 608 |
| 92 | javascript (12.18) | hapi (20.0) | 47 889 | 49 078 | 48 829 |
| 93 | javascript (12.18) | fyrejet-api (2.1) | 47 798 | 53 426 | 51 702 |
| 94 | python (3.8) | pyramid (1.1) | 45 689 | 49 423 | 48 842 |
| 95 | python (3.8) | apidaora (0.27) | 45 235 | 54 951 | 54 850 |
| 96 | javascript (12.18) | fyrejet-uwebsockets (2.1) | 44 593 | 46 667 | 47 551 |
| 97 | rust (1.46) | gotham (0.4) | 43 992 | 49 075 | 50 596 |
| 98 | python (3.8) | hug (2.6) | 43 927 | 46 420 | 47 865 |
| 99 | python (3.8) | blacksheep (0.2) | 43 354 | 49 632 | 49 958 |
| 100 | scala (2.13) | http4s (0.21) | 43 300 | 48 908 | 50 613 |
| 101 | python (3.8) | asgineer (0.8) | 42 858 | 51 416 | 51 553 |
| 102 | python (3.8) | sanic (20.9) | 42 382 | 47 962 | 47 877 |
| 103 | python (3.8) | starlette (0.13) | 41 828 | 46 490 | 46 901 |
| 104 | php (7.4) | imi (1.2) | 41 649 | 48 076 | 47 727 |
| 105 | fsharp (4.7) | suave (2.5) | 39 458 | 39 394 | 39 550 |
| 106 | javascript (12.18) | restify (8.5) | 39 257 | 42 821 | 41 721 |
| 107 | csharp (8.0) | simplify.web (4.1) | 38 500 | 39 979 | 38 764 |
| 108 | javascript (12.18) | fyrejet (2.1) | 37 461 | 40 898 | 40 613 |
| 109 | elixir (1.1) | plug (1.10) | 37 445 | 37 747 | 37 803 |
| 110 | crystal (0.35) | orion (3.0) | 35 669 | 32 897 | 33 568 |
| 111 | scala (2.13) | play (2.8) | 35 184 | 37 799 | 38 003 |
| 112 | python (3.8) | emmett (2.0) | 33 846 | 38 937 | 39 392 |
| 113 | python (3.8) | index.py (0.12) | 32 773 | 36 230 | 35 929 |
| 114 | haskell (8.8) | servant (0.17) | 32 309 | 31 110 | 29 951 |
| 115 | clojure (1.1) | luminus (1.0) | 32 199 | 31 792 | 31 134 |
| 116 | elixir (1.1) | phoenix (1.5) | 31 779 | 32 487 | 32 701 |
| 117 | dart (2.10) | aqueduct (3.3) | 31 767 | 31 122 | 31 609 |
| 118 | nim (1.2) | mike (0.5) | 31 715 | 30 241 | 29 589 |
| 119 | crystal (0.35) | shivneri (0.16) | 31 181 | 29 557 | 27 618 |
| 120 | rust (1.46) | iron (0.6) | 29 297 | 29 435 | 29 972 |
| 121 | javascript (12.18) | turbo_polka (0.3) | 28 955 | 27 504 | 25 451 |
| 122 | php (7.4) | swoft (2.0) | 28 383 | 34 865 | 36 372 |
| 123 | python (3.8) | responder (2.0) | 27 219 | 29 846 | 29 654 |
| 124 | python (3.8) | aiohttp (3.6) | 27 056 | 31 613 | 31 547 |
| 125 | nim (1.2) | akane (0.1) | 27 031 | 24 156 | 22 493 |
| 126 | php (7.4) | yii-swoole (2.0) | 26 727 | 32 858 | 34 835 |
| 127 | python (3.8) | fastapi (0.61) | 26 628 | 30 132 | 29 506 |
| 128 | php (7.4) | chubbyphp-swoole (3.1) | 26 293 | 30 896 | 31 730 |
| 129 | python (3.8) | molten (1.0) | 26 117 | 28 409 | 28 511 |
| 130 | php (7.4) | sw-fw-less (preview) | 25 764 | 31 386 | 32 736 |
| 131 | php (7.4) | slim-swoole (4.5) | 25 565 | 30 794 | 31 543 |
| 132 | ruby (2.7) | hanami-api (0.1) | 24 369 | 24 385 | 23 461 |
| 133 | ruby (2.7) | syro (3.2) | 24 120 | 23 595 | 22 983 |
| 134 | python (3.8) | masonite (2.3) | 23 971 | 26 644 | 26 902 |
| 135 | nim (1.2) | rosencrantz (0.4) | 23 825 | 21 007 | 18 874 |
| 136 | python (3.8) | clastic (19.9) | 23 416 | 25 443 | 24 850 |
| 137 | ruby (2.7) | roda (3.36) | 23 274 | 22 554 | 22 333 |
| 138 | javascript (12.18) | express (4.17) | 22 893 | 23 672 | 24 186 |
| 139 | python (3.8) | flask (1.1) | 22 595 | 21 290 | 22 102 |
| 140 | javascript (12.18) | feathersjs (4.5) | 22 477 | 23 268 | 24 209 |
| 141 | ruby (2.7) | cuba (3.9) | 22 040 | 21 655 | 21 141 |
| 142 | ruby (2.7) | rack-routing (0.0) | 19 784 | 19 903 | 19 421 |
| 143 | ruby (2.7) | rack_app (7.6) | 19 420 | 19 108 | 18 646 |
| 144 | go (1.15) | macaron (1.3) | 18 981 | 20 963 | 20 103 |
| 145 | ruby (2.7) | camping (2.1) | 18 641 | 18 366 | 17 928 |
| 146 | javascript (12.18) | nestjs-express (7.4) | 18 145 | 19 441 | 18 966 |
| 147 | dart (2.10) | start (0.4) | 16 707 | 16 178 | 15 245 |
| 148 | java (11) | blade (2.0) | 15 705 | 19 475 | 19 158 |
| 149 | fsharp (4.7) | giraffe (4.1) | 15 339 | 15 489 | 14 959 |
| 150 | java (11) | struts2 (2.5) | 14 701 | 15 244 | 14 947 |
| 151 | php (7.4) | sunrise-router-roadrunner (2.4) | 14 234 | 15 008 | 15 041 |
| 152 | go (1.15) | tango (0.6) | 14 054 | 14 900 | 14 562 |
| 153 | php (7.4) | chubbyphp-roadrunner (3.1) | 13 921 | 14 422 | 14 347 |
| 154 | php (7.4) | spiral (2.6) | 13 775 | 14 228 | 14 114 |
| 155 | php (7.4) | slim-roadrunner (4.5) | 13 383 | 13 787 | 13 877 |
| 156 | ruby (2.7) | sinatra (2.1) | 13 029 | 12 789 | 12 701 |
| 157 | go (1.15) | gramework (1.7) | 12 472 | 13 142 | 13 208 |
| 158 | ruby (2.7) | grape (1.5) | 12 063 | 11 671 | 11 529 |
| 159 | swift (5.3) | swifter-framework (1.5) | 11 544 | 11 766 | 11 726 |
| 160 | python (3.8) | quart (0.13) | 10 019 | 9 793 | 9 274 |
| 161 | python (3.8) | tornado (6.0) | 9 396 | 9 721 | 9 795 |
| 162 | python (3.8) | cherrypy (18.6) | 9 079 | 9 348 | 8 720 |
| 163 | php (7.4) | basicphp (0.9) | 8 881 | 8 780 | 8 585 |
| 164 | php (7.4) | fastsitephp (1.4) | 8 647 | 8 561 | 8 393 |
| 165 | python (3.8) | django (3.1) | 8 472 | 8 513 | 8 079 |
| 166 | pony (0.38) | jennet (0.1) | 8 174 | 17 384 | 16 912 |
| 167 | python (3.8) | tonberry (0.2) | 7 130 | 6 955 | 6 755 |
| 168 | ruby (2.7) | flame (4.18) | 7 093 | 6 957 | 6 999 |
| 169 | ruby (2.7) | hanami (1.3) | 6 394 | 6 272 | 6 295 |
| 170 | php (7.4) | one-fpm (2.0) | 6 360 | 6 229 | 6 321 |
| 171 | php (7.4) | siler (1.7) | 6 142 | 6 088 | 6 125 |
| 172 | php (7.4) | phalcon (4.0) | 5 844 | 5 863 | 5 848 |
| 173 | php (7.4) | hamlet (3.2) | 5 789 | 5 798 | 5 726 |
| 174 | javascript (12.18) | sails (1.3) | 5 726 | 5 773 | 5 849 |
| 175 | php (7.4) | bearframework (1.2) | 5 071 | 4 996 | 5 038 |
| 176 | php (7.4) | sunrise-router (2.4) | 4 904 | 4 915 | 4 997 |
| 177 | php (7.4) | ice (1.5) | 4 699 | 4 658 | 4 755 |
| 178 | julia (1.5) | merly (0.2) | 4 609 | 9 393 | 7 249 |
| 179 | php (7.4) | chubbyphp (3.1) | 4 236 | 4 246 | 4 296 |
| 180 | ruby (2.7) | rails (6.0) | 3 887 | 3 891 | 3 930 |
| 181 | php (7.4) | slim (4.5) | 3 613 | 3 655 | 3 718 |
| 182 | php (7.4) | nette (3.0) | 3 337 | 3 349 | 3 399 |
| 183 | ruby (2.7) | pakyow (1.0) | 3 255 | 3 415 | 3 385 |
| 184 | php (7.4) | lumen (8.1) | 3 228 | 3 260 | 3 353 |
| 185 | php (7.4) | yii (2.0) | 3 218 | 3 229 | 3 269 |
| 186 | php (7.4) | sunrise-router-annotations (2.4) | 3 036 | 3 080 | 3 149 |
| 187 | crystal (0.35) | runcobo (1.0) | 2 919 | 2 988 | 2 943 |
| 188 | php (7.4) | symfony (5.1) | 2 767 | 2 784 | 2 824 |
| 189 | r (4.0) | restrserve (0.3) | 2 754 | 2 652 | 2 582 |
| 190 | php (7.4) | mezzio (3.2) | 2 421 | 2 456 | 2 482 |
| 191 | python (3.8) | cyclone (1.3) | 2 416 | 2 414 | 2 402 |
| 192 | r (4.0) | httpuv (1.5) | 1 912 | 1 814 | 1 736 |
| 193 | python (3.8) | klein (20.6) | 1 630 | 1 651 | 1 612 |
| 194 | python (3.8) | nameko (2.13) | 1 534 | 1 513 | 1 493 |
| 195 | php (7.4) | laminas (3.1) | 1 352 | 1 378 | 1 439 |
| 196 | v (0.1) | vape (0.3) | 1 165 | 2 025 | 5 365 |
| 197 | php (7.4) | codeigniter4 (4.0) | 1 016 | 1 030 | 1 112 |
| 198 | perl (5.32) | dancer2 (0.3) | 970 | 1 841 | 3 256 |
| 199 | crystal (0.35) | lucky (0.23) | 655 | 666 | 654 |
| 200 | r (4.0) | plumber (1.0) | 429 | 438 | 432 |
| 201 | php (7.4) | laravel (7.27) | 289 | 161 | 2 659 |
How to contribute ?
In any way you want ...
- Request a framework addition
- Report a bug (on any implementation)
- Suggest an idea
- ...
Any kind of idea is ❤️
Contributors
- Taichiro Suzuki - Author | Maintainer
- OvermindDL1 - Maintainer
- Marwan Rabbâa - Maintainer