📎 Pin bench libs
Sec-ant opened this issue · comments
Description
Many of the libraries we use for benchmarking are not pinned to a specific version but rather a branch that are actively updated, and we also don't compare their content hash to decide whether to update them if we already have their local copies in the target path.
We should pin those libs to specific commit hashes or concrete version numbers, and I think we should also use CDNs like jsDelivr to replace those raw.githubusercontent.com links for the benefit of the speed and accessibility across the world (yes, in some places the GitHub urls are blocked :( )
The links that need to be updated and pinned are in https://github.com/biomejs/biome/tree/main/xtask/bench/benches
We can use this page to convert a raw.githubusercontent.com url into a https://cdn.jsdelivr.net/gh/ url. We can also check this page to understand the different forms of the url.
Additionally, we should use the pattern [name].[content-hash].[extension] to save and validate lib files. The code is in https://github.com/biomejs/biome/blob/main/xtask/bench/src/test_case.rs