neondatabase / pg_embedding

Hierarchical Navigable Small World (HNSW) algorithm for vector similarity search in PostgreSQL

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Efficient HNSW + filtering

Palmik opened this issue · comments

Is there any plan to implement efficient HNSW based ANN with filtering? Imagine you have a table:

CREATE TABLE document (
  id UUID PRIMARY KEY,
  text TEXT NOT NULL,
  embedding REAL[] NOT NULL,
  category TEXT NOT NULL
)

And then you want to do a query like:

SELECT id, text
FROM document
WHERE category = $1
ORDER BY embedding <-> $2
LIMIT 10;

To do this efficiently is not straightforward and will likely need some work, see e.g.: https://qdrant.tech/articles/filtrable-hnsw/
(In this scenario, you could probably achieve similar effect by partitioning the table -- but that's not possible for more complex filters)