COO to GCXS without relying on linear indices
LivelyLiz opened this issue · comments
Problem
I have tensors that are so large that the linear indices are larger than what an integer value can hold but during conversion from COO to GCXS, the indices are converted to linear an therefore conversion fails. The error I get is ValueError: invalid dims: array size defined by dims is larger than the maximum possible size. which is thrown by np.ravel_multi_index.
Describe the solution you'd like
Ideally the conversion would not rely on linear indices but I'm not entirely sure if this is even possible or if GCXS is using them internally. I skimmed the paper related to it and I think it's not necessary.
Unfortunately, it's used a lot internally, which is why you see this message.
I see, too bad for me :/ I guess I would need to work with COO then
COO won't work either, sadly.
Ah, damn, you are right. I have the construction of the COO and conversion to GCXS in one line and thought the problem occurs during the conversion but it already happens during construction of the COO object. Darn these way too large tensors!