cudnn scaling parameters are incorrect for f16/bf16
coreylowman opened this issue · comments
According to https://docs.nvidia.com/deeplearning/cudnn/developer-guide/index.html#scaling-parameters, scaling parameters for both f16 and f32 need to be f32 values.
Currently cudarc uses f16 scalar parameters