Model selection via validation loss
kylebgorman opened this issue · comments
To support experiments on whether our model selection strategy (maximal validation accuracy), we should provide a flag which enables model selection based on minimal validation loss instead.
The interface I would suggest is something like this: a flag --model_selection_mode
with a default value accuracy
and an alternative value loss
. (We already have something similar in place for the reduce-on-plateau scheduling: see here).