cuDNN
jafioti opened this issue · comments
Currently we use cublas through the cudarc library for matmuls, which is very good for generic matmul performance. cuDNN has many more NN specific ops that can be leveraged specifically for stuff like convs and mlps. Not sure if cudarc has this or if we'll need another library for it