mT5-base model return NaN when I use mixed precision
asartipi13 opened this issue · comments
Amir Sartipi commented
Hi everyone,
I use torch.autocast for mT5 model in order to use float16, but the model return NaN.
I also checked #40497, but I couldn't find a solution.
Thanks in advance for your answers.
qibao77 commented
@asartipi13 Hi, How did you solve it? Thank you!