support amp with fused optimizer
wangraying opened this issue · comments
torch native amp + apex amp
Hi! Any updates on this issue? Is it possible to use fuse_optimizer with torch.cuda.amp.GradScaler?
Not yet. We may solve this issue in Q2 this year, hopefully. There is a previous draft related to this issue master...fuse-opt-fp16.