[FEA] cuTENSOR 2.0 support

[Gabriel Wu]

Now that cuTENSOR 2.0 has been released for some time, I think it is natural for MatX to follow this update and enjoy the performance gains brought about by this new version.

[Gabriel Wu]

I just realized that since cuTensorNet is used in MatX, this update is blocked until cuTensorNet adopts cuTENSOR 2.0.

[Leo Fang]

cuQuantum 24.03 will move to use cuTENSOR 2.

[Gabriel Wu]

cuQuantum 24.03 will move to use cuTENSOR 2.

Thanks for the information.

[Cliff Burdick]

@lucifer1004 you just have read my mind because we were just talking about this. I'll try to get to it next week. Do you use einsum right now?

[Gabriel Wu]

I need to use einsum. See #597.

[Cliff Burdick]

@lucifer1004 we will wait until the new cuquantum is released so that it's compatible with cutensor 2.x.

[Cliff Burdick]

@lucifer1004 the existing (older) cutensor + einsum is useable now and we will integrate the new cutensornet once ready

[Leo Fang]

FYI cuQuantum 24.03 is out.

[Cliff Burdick]

Updated to cuQuantum 24.03 and verified fix.