Support for other HSNW engines
mingmasplace opened this issue · comments
Does pgvector has its own HNSW implementation or it support pluggable HNSW engines? Per https://github.com/erikbern/ann-benchmarks, there are several fast HNSW implementations such as nmslib. It will be useful if they can be integrated into pgvector.
Hi @mingmasplace, pgvector has its own implementation that's designed for Postgres. I don't think it makes sense to support other implementations (which don't use Postgres storage or write-ahead logging).
Adding a bit to @ankane - why this is advantageous for PostgreSQL is how the pgvector implementation can target the PostgreSQL page layout specifically. Other providers, while effective for in-memory and other domains, don't have to worry about on-disk layout or grouping data together in 8KB chunks. By creating an implementation that's specific to PostgreSQL's storage layout, we can both optimize the algorithms towards this while getting the other benefits of using PostgreSQL (e.g. ACID).