Benchmark VMB tests
bitjson opened this issue · comments
It would be good to have a set of VMB tests that implementations can use to benchmark VM performance for all the known worst-case scenarios. Implementations should use the performance benchmark tooling for their stack to judge the relative performance of each test (over a series of runs) to verify that there aren't any significant outliers.
The start of a wallet template including all the benchmarks:
https://ide.bitauth.com/import-gist/6be7fbca3b3f77d60bc8953439e42612
Here's some discussion of the last few months of work, and I'm continuing to build out test vectors and benchmarks.
Ultimately, this CHIP should also significantly increase our set of cross-implementation test vectors to thoroughly cover both the behavior and benchmarked performance of every operation. I'm just going to leave this issue open until we've thoroughly covered everything and migrated script_tests.json.
I'm going to close this now that we've directly committed the trimmed down set of test vectors/benchmarks and completed the Tests & Benchmarks appendix.
I still plan to expand BCH's test coverage and deprecate script_tests.json, but that's significantly beyond the scope of this CHIP. I'll report on progress here (even after this CHIP is archived): bitauth/libauth#31