Measure the spread in Elo performance by random seed
nodchip opened this issue · comments
In #152, we have measured Elo performance between nets generated by different commits and random seeds. Random seeds should affect the spread in Elo performance. We should measure the spread in Elo performance by setting random seed to several values.
There are some combinations:
- Setting random seed to a constant value in training data generation, and setting random seed to a constant value.
- Setting random seed to a constant value in training data generation, and setting random seed to several values in training.
- Setting random seed to several values in training data generation, and setting random seed to a constant value.
- Setting random seed to several values in training data generation, and setting random seed to several values in training.
We should set the number of the threads to 1 to avoid the effects of multi-threading. We should also be better to measure the effects of single-threading vs multi-threading.