pydata / sparse

Sparse multi-dimensional arrays for the PyData ecosystem

Home Page:https://sparse.pydata.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Save multiple arrays to the same npz file

pedroaugustosmribeiro opened this issue · comments

It would be nice to save multiple arrays to a single npz file, just like in numpy.

Is this actually possible with scipy.sparse matrices? In the sparse context, save_npz already needs to save multiple arrays for each sparse matrix npz

e.g. saving a scipy.sparse.csr_matrix saves to the following arrays:

['indices', 'indptr', 'format', 'shape', 'data']

I could imagine doing that multiple times to save more than one array (indices_0 indptr_0 ... indices_1 ...) but I don't think scipy does that at the moment.

Sorry, i meant like numpy.savez or numpy.savez_compressed

While an NPZ file can hold many NumPy arrays, one sparse array contains multiple NumPy arrays, so it can only hold one sparse array unless tricks like ones mentioned in #549 (comment) are used.

Sorry, i meant like numpy.savez or numpy.savez_compressed

NB: if you try to use these methods on a scipy.sparse matrix it will save it as a pickle, which tends to take up more space and be slower to read. That's why scipy.sparse.save_npz is a separate function.

Closing as this is complicated and out of scope.