vballoli / nfnets-pytorch

NFNets and Adaptive Gradient Clipping for SGD implemented in PyTorch. Find explanation at tourdeml.github.io/blog/

https://nfnets-pytorch.readthedocs.io/en/latest/

Incorrect params initialization in AGC

haigh1510 opened this issue 3 years ago · comments

haigh1510 commented 3 years ago

Probably parameters should be renamed to params here

nfnets-pytorch/nfnets/agc.py

Line 44 in 1513cea

parameters = [{"params": module.parameters()} for name,

It's not used otherwise

Vaibhav Balloli commented 3 years ago

Thanks for bringing it up, not sure how I missed that.