opium in techempower frameworks benchmarks
mudrz opened this issue · comments
not really an issue, but just letting you know that opium was added to the techempower frameworks benchmarks
https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/OCaml/opium
it didn't catch the last benchmarks run until a planned update of their tools TechEmpower/FrameworkBenchmarks#6064
so the next benchmarks run will be delayed indefinitely
in the meantime please feel free to make changes and improvements
Hi @mudrz,
That's awesome, thanks a lot for doing this!
I'm very curious to see how the next version of Opium (w/ Httpaf) will compare with the one you used for the benchmarks (w/ Cohttp). The benchmarks showing a better performance of Httpaf compared to Cohttp are at least 3 years old and Cohttp evolved a lot since then, so I wonder if they still stand.
in the current implementation opium is pinned to a master commit with httpaf
but it's really easy to add a new dockerfile and use the cohttp version in order to compare
edit: didn't notice that the httpaf version has a new API, so it's it would need more work to create a benchmark with cohttp;
also there are now results:
https://www.techempower.com/benchmarks/#section=test&runid=bf3eceff-da94-46a8-87ab-6fc3ef39c12c&hw=ph&test=json&l=ziiku7-v&a=2&f=2-0-10-e8-74-9hc-6nq9ds-2-e8-hra0hs-f0g
@tmattio poster about the results here:
https://discuss.ocaml.org/t/techempower-benchmarks-for-opium-running-httpaf-are-ready-how-to-improve-the-results/6627
There is something wrong with the plaintext results, not sure if it is in my implementation, or perhaps there's something in opium?
Hi @mudrz,
There's definitely something going on with the plain text serialization. I'll look into it as soon as I have some spare time!
Thanks again for doing this, it's great to see where Opium positions itself in comparison with other frameworks 🙂
the plaintext issue is related to httpaf and should be fixed:
https://discuss.ocaml.org/t/techempower-benchmarks-for-opium-running-httpaf-are-ready-how-to-improve-the-results/6627/18?u=mudrz
(but there doesn't seem to be a new release)