Stable-only benchmarks don't run locally
Nadrieril opened this issue · comments
Some benchmarks like encoding are category: stable instead of category: primary/secondary. When I run collector bench_local --include encoding,..., encoding is not run. Is there a way to make it run, short of locally changing its category?
It's not currently possible because of this. We can probably just remove that line though.
Out of curiousity, what is your use-case for running a stable benchmark?
Removing this line would fix my problem :D Maybe --include primary,secondary could filter just these for anyone who needs that? I only ever run a selected set of benchmarks myself since I only touch match checking.
My use-case is testing whether we can remove an old hack that specifically regressed encoding (here).
@nnethercote Did you have any specific motivation for disabling stable benchmarks in bench_local?
Because running the stable benchmarks locally is almost never what you want to do. More detail here.