Warmup scheduler seems not working?
feiyuhuahuo opened this issue · comments
Aizen commented
Kang Kim commented
Hi, the constant warmup assigns a smaller LR (1/3 of the original LR) for the first 500 iterations. I think the linear warmup should also be working well. I don't think the choice between constant and linear warmup will make a big difference here.