Choose performance targets for SLVM
StevenLove opened this issue · comments
Performance is not a central focus of this project, but it is necessary to decide on some metrics to determine if the performance is acceptable or not acceptable. Otherwise it is unclear how to evaluate proposed changes that affect performance. Additionally, it helps to guide our development priorities toward or away from performance depending on how close we are to our performance cutoff.
I propose that we choose
- some common algorithms to measure performance
- some existing language(s) to compare with SLVM
- some threshold like '10x'
At which point we can easily decide that SLVM performance is acceptable IFF its performance on those chosen algorithms is within '10x' of the chosen language(s)
https://github.com/attractivechaos/plb2 has some helpful discussion on the topic of comparing performance across languages. We might choose to use some of the algorithms from this project, which would make some language comparisons quite easy.
but also performance isn't not not a focus. the vm exists because the original tree walking interpreter was too slow. so to be true to that intent it would be nice to have benchmarks proving it.
invert a matrix
calculate variance
other stats things.
plenty of other languages have implementations for these we could check against.