[FEA] Allow Real * Complex GEMMs

[Cliff Burdick]

Currently BLAS only supports GEMMs with both inputs of the same type (real/complex). We should support a mix of real/complex inputs even if it takes a conversion internally.

[Justin Luitjens]

Can this be done at the API level with a simple constexpr branch checking types and dispatching the compex operator?

void gemm(...)
if constexp( is_complex_v(Atype) && !is_complex_v(Btype)) {
gemm(C, A, complex(B), stream);
} else if constexpr (!is_complex_v(Atype) && is_complex_v(Btype){
gemm(C, complex(A), B, stream);
} else {
// existing launch code.
}

[Cliff Burdick]

we need to async allocate a new tensor and upconvert to the required type. For example, if it's complex * real, we need to allocate space for a complex, assign the real to the complex, then do the gemm. I think what you're suggesting would only work if the GEMM libraries could take operators.

[Justin Luitjens]

I think i wrote it so gemm does this automatically if one of the inputs is an operator.

[Justin Luitjens]

https://github.com/NVIDIA/MatX/blob/main/include/matx/transforms/matmul.h#L1072-L1087

[Cliff Burdick]

Great, so then we just need to extend that to promote any types to the output type

[Kshiteej K]

@cliffburdick @luitjens I would like to give it a shot. Will try to send a PR soon :)