Random number generation
patnr opened this issue · comments
I'm guessing the tests that just failed are due to some change in recent Python patches affecting the random numbers generated. But it's difficult to debug when I cannot seem to access the most recent Python's locally...
DAPPER is/was using numpy's legacy/global RandomState system, accessed via numpy.random.seed/randn/rand/choice/.... Maybe upgrading to the Generator local stream would fix it, and just be a good forward-looking choice?
Update: I wrote random numbers to file and uploaded it as artefact, and confirmed that it is the random numbers that have changed. Strangely though, the first several (hundreds?) of numbers remain the same.
I should have tried a new install locally first. The problem was not the platform or python version, but simply the dependency 🤦 Specifically, upgrading scipy to 1.11 causes the error. Probably some part of scipy is using the global state differently than before. This type of problems are bound to happen with global random state, unless we pin everything, which we dont want, and therefore seem like a good reason to use new random stream/generator system.
BUT, the new system seems problematic for exact reproduction of pseudo-random results (e.g. tests_example_2.py), which appear liable to change quite often and depending on platform. From docs:
Because of the change in algorithms, it is not possible to reproduce the exact random values using
Generatorfor these distributions or any distribution method that relies on them.
More info at stackoverflow.
The upgrade of scipy also revealed another weakness: Computing matrix square roots has a degree of freedom to it: the sign of each row/column. And for some reason scipy.linalg.svd sometimes returned different signs from version 1.10 to 1.11. So if you use the svd to compute rectangular square roots then you get different random samples when they're colored by that square root. Fixed in deb07f7