OCA99/vearch-workers

Overview

Vearch is a scalable distributed system for efficient similarity search of deep learning vectors.

Quick start

Quickly compile the source codes to build a distributed vector search system with RESTful API, please see docs/SourceCompileDeployment.md.
Vearch can be leveraged to build a complete visual search system to index billions of images. The image retrieval plugin for object detection and feature extraction is also required. For more information, please refer to docs/Quickstart.md.
Vearch Python SDK enables vearch to use locally. Vearch python sdk can be installed easily by pip install vearch. For more information, please refer to docs/APIPythonSDK.md.
One-click binary installation, please see docs/BinaryInstallation.md(中文版).

APIs and Use Cases

Document

Components

Vearch Architecture

Master

Responsible for schema mananagement, cluster-level metadata, and resource coordination.

Router

Provides RESTful API: `create` , `delete` `search` and `update` ; request routing, and result merging.

PartitionServer (PS)

Hosts document partitions with raft-based replication.

Gamma is the core vector search engine implemented based on faiss. It provides the ability of storing, indexing and retrieving the vectors and scalars.

Benchmarks

benchmarks

Reference

Reference to cite when you use Vearch in a research paper:

@misc{li2019design,
      title={The Design and Implementation of a Real Time Visual Search System on JD E-commerce Platform}, 
      author={Jie Li and Haifeng Liu and Chuanghua Gui and Jianyu Chen and Zhenyun Ni and Ning Wang},
      year={2019},
      eprint={1908.07389},
      archivePrefix={arXiv},
      primaryClass={cs.IR}
}