Improve benchmark
GravisZro opened this issue · comments
Time is a lousy benchmark, especially with a low numbers of elements running on an OS. There a lot of way of profiling but I would suggest using an LLVM based profiling solution like this one because it measures the number of intermediate language instructions executed. This avoids compiler/microcode optimizations and gives you the most representative output for all platforms.
Wouldn't that fail to measure things like cache misses and branch mispredictions? It would be a useful metric to keep track of though.
What I have currently is within 1% accurate by measuring the best run out of 100. For low numbers of elements the number of repetitions can be increased to reduce the relative overhead of gettimeofday().
The idea is to measure the speed of the algorithm without regard to machine it runs on, specifically because not all machines are created equal. It's fine to have "real world" results as well (though I would include more platforms) but nothing about them are empirical.