pytorch / examples

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.

Home Page:https://pytorch.org/examples

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

mT5-base model return NaN when I use mixed precision

asartipi13 opened this issue · comments

Hi everyone,

I use torch.autocast for mT5 model in order to use float16, but the model return NaN.
I also checked #40497, but I couldn't find a solution.

Thanks in advance for your answers.

@asartipi13 Hi, How did you solve it? Thank you!