What's the specific parameters of learning rate annealing?
fourierer opened this issue · comments
Fourier commented
Hello, I set the learning rate annealing parameter like this: for every 5 epochs passed, the learning rate was multiplied by 0.5, from 0.4 to 0. I'm currently training up to 40 epochs, top-1 accuracy is 67.010%, top-5 accuracy is 86.918%, and now the increase is very small, do I need to continue training? Could you tell me your specific parameters of linear LR annealing, which's accuracy is 72.318%/90.670%?
Duo Li commented
I think you're using the step annealing method rather than linear. At each iteration, the learning rate should linearly adapt like this
lr = init_lr * (1 - (current_iter - warmup_iter) / (max_iter - warmup_iter))