Code versions as a hyperparameter
bclarkson-code opened this issue · comments
It is common practice to record model hyperparameters when experimenting. But I think this misses a lot of extra information. To experiment properly, I think that one should also record specific implementations of different components. you could simply record the git commit that was used, but I think it would be more helpful to record the specific version of each component used.
I think that a good place to start would be recording a hash (or similar) for each layer in the model.
Closing for now as it is off topic