Random failures in test suite
tillea opened this issue · comments
Hi,
the Debian package of garli received a bug report about random failures in the build time test suite. The bug reporter did about 100 builds in a row and reported a failure rate might be around 10%. He linked three example build logs (no idea how longs these logs will persist).
My solution was for the moment to substitute randseed of -1 to 1 in all config files of the test suite to have reproducible results. I noticed that even than the output of different runs is not identical - no idea how the test can be made fully reproducible). This will solve the Fails-To-Build-From-Source issue in Debian but I think it is worth reporting the issue here for further inspection.
Kind regards, Andreas.
Hi Andreas,
Thanks for the feedback. I'm not the maintainer of this project. Best to report upstream.
Thanks.