Using Triton with PyTorch 2.0 for AMP training results in tensors containing inf values.

[Ryan]

Hi, thx for your great work!

I set use_triton=True, and turned on automatic mixed precision training, but inf appeared in the results.
Does the lion_pytorch/triton.py need to consider bf16 or fp16?

It seems this issue #20 (comment) is similar to mine