wg-grammar doc full bench slightly noisy
klensy opened this issue · comments
As can be seen on https://perf.rust-lang.org/index.html?kind=percentrelative&stat=size%3Adoc_bytes there some persistent few byte noise on graph, which not exist for other benches, which makes me think that there something interesting going on here.
Can't locally reproduce this: tried few nightly's but no luck, is there way to download bench artifacts or they erased?
Ok, idea: doc_bytes converted u64 -> f64 -> db and back, loosing precision somewhere between. Postgres have extra_float_digits(https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-EXTRA-FLOAT-DIGITS) option that have effect on precision.
Or just js fun?
Good idea, but I doubt this is the reason. The maximum value of doc_bytes stored in the DB is ~150 000 000. 64-bit floats should be able to represent all numbers up to 2^53 (https://stackoverflow.com/questions/1848700/biggest-integer-that-can-be-stored-in-a-double), and that's was larger than the max. value of doc_bytes. For 32-bit floats it's more interesting, since it's just 2^24, and that's 16 million, so smaller than the largest value that we have.
But we don't use f32 pretty much anywhere in the collector or our DB code, and Postgre's double precision should be a 64-bit float.
We also use much larger values for instruction counts, so I doubt that it would be ok there, but it would be an issue for doc_bytes specifically. Something like this would probably affect everything.
The changes in the doc_bytes are very small (0.05% etc.). It's probably some non-reproducible output, like including the compiler's version in the HTML files or something like that.
like including the compiler's version in the HTML files or something like that.
In that case all tests will be affected by that, not only few of them.
I did a diff of the doc directories for cargo-0.60 using Meld, for two artifacts that had a small blip on perf.RLO (be00c5a9b89161b7f45ba80340f709e8e41122f9 and 5876c8cdfd3df742c334d6447d44d760c77103b6).
There are a few differences. The version of the compiler is embedded in the HTML files, but its byte size should be mostly the same. Hashes of various JS files also differ, but again, the size should be the same. Then I saw that some output (HTML tags) is being emitted with different anchors:
[-href="#method.from-1"-]{+href="#method.from"+}
which causes small diffs. It seems to somehow oscillate between different sections on the same page:
It also seemed like the order of some things was printed in a non-deterministic order (maybe some HashMap iteration going on in rustdoc)?
Anyway, it looks quite harmless, but pinging @GuillaumeGomez, just in case this wasn't expected behavior.
Some order thing should be fixed by rust-lang/rust#120641