mixed precision for lnms
Matze99 opened this issue · comments
The one thing is to make sure your network output is fp32 - you can add a linear activation to each output as they show in the TF page
outputs = layers.Activation('softmax', dtype='float32', name='predictions')(x)
change 'softmax' to linear and it's basically just a casting operation
https://docs.nvidia.com/deeplearning/performance/mixed-precision-training/index.html