Remove the need to look at Tensorboard for LR
erikrose opened this issue · comments
It's a drag for people to have to install tensorboard, run the server, surf to the page, then learn to interpret it. And then they have to remember to check it again if the rules they add change the learning rate they need. It seems like we ought to be able to pick a LR automatically, perhaps by watching for oscillations or something even more clever. (The Adam optimizer does dial in the LRs (one per input, in fact) automatically, but it can take several iterations. In the meantime, early stopping fires off prematurely, leaving you with an incorrect result.)
It should be sufficient to watch for deviations from monotonicity in the training loss or maybe the validation loss.