optimizer params restore does not work
zhengzx-nlp opened this issue · comments
learning rate is not yet restorable.
optim.optim.load_state_dict(defaultdict(dict, **optimizer_params))
I think this line could retore the learning rate ?
emmm. So bad things must happen to optim params restore itself