Improve wyrand a bit
rurban opened this issue · comments
I started testing now also all rng functions.
See https://github.com/rurban/dieharder/blob/pcg/QUALITY.md
wyrand is the fastest wo far, but has a single weakness in the rgb_lagged_sum with ntup=17 test.
So it is currently not the very best, but could easily be.
ah, this is not weak, but due to "multiple testing problem". https://en.wikipedia.org/wiki/Multiple_comparisons_problem
because you make 1000 statistical tests, some of them will show a p-value like 0.001, it is natural.
you can run your program with different seed. "weakness" due to random chance will disappear with different seed while systematic fail will persist.
You are right.
I either
- have to eliminate expected outliers as in smhasher (needs lot of time and memory, which is harder with that many prng results).
- Or substract the alpha (family-wise error rate). (with this I loose some raw data, but should be done).
- Or I use a different testing strategy (-Y1 in dieharder), which checks in weak results again with different seeds to resolve potential ambiguities in bad p-values. (this is already builtin)
This brought now perfect results for wyrand.