A modular semantic search stack.
It can be fully cloud-based with Cloudflare KV or Upstash Redis, OpenAI or Cohere Text Embedding, and Pinecone or Qdrant Cloud Vector Database.
It can also be self-hosted with CouchDB, Qdrant on Docker.
- Support for different item types: KVec supports text, web page, and image items out of the box, and you can easily extend it to support other types.
- Modular architecture: KVec allows you to easily change components like the encoder, object store, and vector store to fit your specific use case.
- Authentication: KVec supports authentication using JWT tokens.
- Dashboard GUI: KVec comes with a simple dashboard that allows you to manage your items and issue authentication tokens.
- RESTful API: KVec provides a simple API for creating, reading, deleting, and searching items.
To use Cloudflare Pages to host your KVec.
You should set up the following environment variables:
APP_SECRET
: Secret for signing JWT tokensPINECONE_API_KEY
: To enablePineconeVecStore
(see Structure)PINECONE_ENDPOINT
: The endpoint of your Pinecone index, nessary forPineconeVecStore
OPENAI_API_KEY
: To enableOpenAIEncoder
(see Structure)HF_API_TOKEN
: To enableBaseTextAdapter
's image feature
And bind a KV namespace:
KV
: KV namespace for storing items, nessary forCloudflareKVObjStore
(see Structure)
You can simply:
- Fork this repo.
- Create a new Cloudflare Pages project, connect to your forked repo, and setup the environment variables.
- Create a KV namespace and bind it to the project.
To use KVec with Docker, you'll need to setup environment variables in the .env
file.
You may want something like this:
PORT="8080"
ORIGIN="http://localhost:8080"
APP_SECRET="my-kvec-secret"
# Adapter
HF_API_TOKEN="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
# Encoder
COHERE_API_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
# VecStore
QDRANT_SERVER="http://qdrant:6333"
QDRANT_COLLECTION="kvec"
# ObjStore
COUCHDB_URL="http://couchdb:5984"
COUCHDB_DB="kvec"
COUCHDB_USER="admin"
COUCHDB_PASSWORD="password"
Then, you can run the following command to start:
docker compose up -d
To setup the Qdrant collection at the first time, you can run the following command:
curl -X 'PUT' 'http://localhost:6333/collections/kvec' -H 'accept: application/json' -H 'content-type: application/json' -d '
{
"vectors": {
"size": 4096,
"distance": "Cosine"
}
}'
# The vector size for Cohere's embedding is 4096, if you are using OpenAI's embedding, you should change it to 1536.
To setup the CouchDB database at the first time, you can run the following command:
curl -X PUT http://admin:password@localhost:5984/kvec
Now, you should be able to access the dashboard from http://localhost:8080/
.
You can use the dashboard GUI to manage items and issue tokens.
You can access the dashboard from https://kvec.yourdomain.com/
.
You will need to get a JWT token to make requests to the item API.
One way to do this is to use the dashboard GUI.
However, you can also issue tokens using the API itself:
curl -X POST \
-H "Content-Type: application/json" \
-d '{ "secret": "your-app-secret", "exp": 3600, "perm": { "read": true, "write": false } }' \
https://kvec.yourdomain.com/api/auth
# Creates a token that expires in 1 hour and only allows read access
{
"token": "ISSUED_JWT"
}
The token can be passed in the
Authorization
header of the request or thekvec_token
cookie.
The item API allows you to create, read, delete, and search items.
write
permission is required.
curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: YOUR_TOKEN" \
-d '{ "data": { text: "the content of the item" } }' \
https://kvec.yourdomain.com/api/item
{
"id": "ITEM_ID"
}
This will create a new item with the text
the content of the item
.
Examples for other types of items:
Web page. The
BaseTextAdapter
will automatically fetch and use the page title and description as the page feature.
{
"data": {
"page": "https://github.com/JacobLinCool/kvec"
}
}
Image.
HF_API_TOKEN
environment variable is required. It uses Hugging Face's inference API to transform the image into text as the image feature. The default model isnlpconnect/vit-gpt2-image-captioning
, but you can specify a different model by setting theHF_IMGCAP_MODEL
environment variable.http://
,https://
, anddata:
are supported.
{
"data": {
"img": "https://kvec.pages.dev/icon.png"
}
}
read
permission is required.
curl -X GET \
-H "Content-Type: application/json" \
-H "Authorization: YOUR_TOKEN" \
https://kvec.yourdomain.com/api/item/<ITEM_ID>
{
"item": {
"id": "ITEM_ID",
"data": {
"text": "the content of the item"
},
"meta": {
"type": "text"
}
}
}
write
permission is required.
curl -X DELETE \
-H "Content-Type: application/json" \
-H "Authorization: YOUR_TOKEN" \
https://kvec.yourdomain.com/api/item/<ITEM_ID>
{
"deleted": true,
"item": {
"id": "ITEM_ID",
"data": {
"text": "the content of the item"
},
"meta": {
"type": "text"
}
}
}
It performs a semantic search to find items that are similar to the query.
read
permission is required.
curl -X GET \
-H "Content-Type: application/json" \
-H "Authorization: YOUR_TOKEN" \
https://kvec.yourdomain.com/api/item?q=<QUERY>
{
"items": [
{
"id": "ITEM_ID_1",
"data": {
"text": "the content of item 1"
},
"meta": {
"type": "text"
}
},
{
"id": "ITEM_ID_2",
"data": {
"text": "the content of item 2",
"page": "https://example.com/item2"
},
"meta": {
"type": "page"
}
},
{
"id": "ITEM_ID_3",
"data": {
"text": "the content of item 2",
"img": "https://example.com/item3.png"
},
"meta": {
"type": "img"
}
}
]
}
The KVec structure is mainly based on 6 components:
- The API and GUI layer, which allows other services to interact with KVec easily and manage the authorizations.
- The Adapter layer, which is responsible for adapting the data from the API layer to the encoder layer.
- The Encoder layer, which is responsible for encoding the data into vectors (embeddings).
- The ObjStore layer, which is responsible for storing the items itself.
- The VecStore layer, which is responsible for storing the vectors, and performing the search.
- The Cache layer, which is responsible for caching the search results.
The Adapter, Encoder, ObjStore, VecStore, and Cache layers are all pluggable, so you can easily customize them to fit your needs.
Currently, the following implementations are available:
- Adapter
-
BaseTextAdapter
: Support text, web page, and image (envHF_API_TOKEN
required) items.
-
- Encoder
-
OpenAIEncoder
: Use OpenAI'stext-embedding-ada-002
to create embeddings -
CohereEncoder
: Use Cohere as the embedding service -
JustEncoder
: Only for local development
-
- ObjStore
-
CloudflareKVObjStore
: Use Cloudflare KV as the object store backend -
UpstashRedisObjStore
: Use Upstash Redis as the object store backend -
CouchDBObjStore
: Use CouchDBObjStore as the object store backend -
MemoryObjStore
: Only for local development
-
- VecStore
- Cache
-
CloudflareCache
: Use Cloudflare's Cache API -
MemoryCache
: Only for local development, it just "don't cache anything"
-
The auto module will automatically load the correct implementation based on the environment variables. See src/lib/server/auto/index.ts You can also take a look at the README of each module to see how to configure them.