[JS] Add support for Vertex Vector Search to the Vertex AI plugin
chrisraygill opened this issue · comments
Is your feature request related to a problem? Please describe.
Developers who want to use Vertex Vector Search as a scalable, production-ready solution don't have a way to do so through Genkit.
Describe the solution you'd like
Make Vertex Vector Search available through the Vertex AI plugin as indexer and retriever components.
Describe alternatives you've considered
Developers can define their own indexer and retriever components, but it's more convenient to have it exposed through the plugin as a first class experience.
Additional context
More info on Vertex Vector Search available here: https://cloud.google.com/vertex-ai/docs/vector-search/overview
@chrisraygill I thought I'd discuss this here, let me know if it should be somewhere else:
Unless I've misunderstood. a Vertex AI index will store an id and the embedding, and users are expected to keep the content of the document and any metadata which corresponds to this id somewhere else (In another database, e.g BQ, to retrieve via the id)
In Genkit we have a Document which has content and metadata, and a retriever responds with a list of documents.
Should we enforce BigQuery as the database in this feature?
adding for reference #519