Restore CI benchmarks
Robbepop opened this issue · comments
With the transition of Wasmi from the paritytech GitHub organization to the wasmi-labs GitHub organization we also (expectedly) lost the GitLab based CI benchmarking for Wasmi which immensely helped the Wasmi developers keep track of the performance implications of their PRs.
This was done by automated benchmarks for each commit of a PR to master and a comment that summarizes the results in detail, such as this: #928 (comment)
Unfortunately with the transition we no longer have access to Parity's GitLab instance and runners and our current CI setup no longer works. Affected files are:
Ideally the new CI uses GitHub Actions and self-hosted runners compatible with GitHub actions that are similar in hardware specs (or better) than the original ones we had with our GitLab instance. For that purpose we might want to rent some proper hardware to do the job in the future. But for experimental purposes it should be fine to build initial GitHub Actions based CI benchmarks with default GitHub action runners.
So far we did the parsing of the benchmark results with the benchmark-report.sh file but maybe we could use this crate and a proper internal Rust crate for the job in the future: https://crates.io/crates/criterion-table
Removed GitLab benchmarking CI related files here: #951