Collect and publish Benchmarks somewhere / Provide Benchmark quick start
gitmalong opened this issue · comments
Hi!
I just discovered your wonderful crate and naturally I'd love to see some benchmarks.
- I would greatly appreciate it if the README.md elaborates on how to run benchmarks on the own machine. Ideally without creating a new rust program with help of one or two commands
☺️ . - I propose to collect some benchmarks in the repo itself to eliminate the need for 1) for certain users.
Thank you!
Hello
I'm not really inclined to do either.
- README is very secondary kind of documentation, the primary one is on docs.rs.
- The benchmarks are run the usual Rust way through
cargo bench
. I don't think each repo with some Rust code should give a manual tocargo
.
As to providing information about the performance. The raw numbers are not provided on purpose, because:
- That's not the most important performance characteristic of this crate, the lock freedom is.
- Microbenchmarks are a very misleading thing. People naturally want to see some numbers and interpret them without really doing the effort of understanding them.
- Collecting them in CI doesn't work (because that's very noisy, the instances differ from each other and the load differs), running it manually would be forgotten.
Nevertheless, there's a whole extensive page about performance in the documentation.