Write Index into memory or Read Index from memory
ava57r opened this issue · comments
Hello.
C++ Faiss supports some methods for saving a index into memory.
write into IOWriter
https://github.com/facebookresearch/faiss/blob/main/faiss/index_io.h#L40
and impl VectorIOWriter https://github.com/facebookresearch/faiss/blob/main/faiss/impl/io.h#L59
read from IOReader
https://github.com/facebookresearch/faiss/blob/main/faiss/index_io.h#L61
and impl VectorIOReader
https://github.com/facebookresearch/faiss/blob/main/faiss/impl/io.h#L53
Could We add these impls into rust library?
We would be well served with standard Rust APIs for writers and readers, so that an index can be written to anything implementing std::io::Write
and an index can be read from anything implementing std::io::Read
. I am not fully familiar with how these data structures operate, but so long as they can work with preallocated input/output buffers, it should be possible to interoperate them.