pgvector / pgvector-php

pgvector support for PHP

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add Laravel Command to rebuild an index

C0chett0 opened this issue · comments

I'll add to my project an artisan command which will drop and rebuild my index on my embeddings table. Which will be triggered when new embeddings will be added to the table.

If interested I can make a generic version to be added to this package.

At which point I would start wondering if it wouldn't make sense to have two separate packages, one for native PHP and one for Laravel specifically.

Thanks for another suggestion. Indexes shouldn't need to be rebuilt when rows are added/updated/deleted in most cases, so I'd prefer to hold off on adding this for now (and if lists needs to be adjusted, it should happen through a migration).

Re separate packages: I'd prefer to keep it as one for now, unless it somehow limits what can be added.

Perhaps we can create a job or similar which can be called whenever necessary. I also had some issues with the index (pgvector/pgvector#109) and @ankane advice is clear and worked: Use the right values for lists and probes.

My current solution is a job which runs after every batch of inserting new data. It drops the index and recreates it based on the suggested variable values (https://github.com/pgvector/pgvector#indexing).

I would open a pull request with my suggested solution if you agree.

Hi @kainiklas, recreating the index after inserts isn't needed in most cases, so it's not something I'd like to include in the library, but someone could always create a separate package for it.