Cross ranker transformers module
The inference container for Weaviate's reranker-transformers module
π Documentation
Documentation for this module can be found here.
π¦ Requirements
- Create a new virtual environment
python3 -m venv .venv
source .venv/bin/activate
- Install all module and test dependencies
pip3 install -r requirements.txt
pip3 install -r requirements-test.txt
- Download the model locally
MODEL_NAME=cross-encoder/ms-marco-MiniLM-L-6-v2 ./download.py
- Run the inference server
uvicorn app:app --host 0.0.0.0 --port 8000
π‘ Testing
For sanity checks that to check that all works properly you can run our smoke tests against your server
python3 smoke_tests.py
π³ Docker support
In order to build locally a docker image one can run this command in project's root folder
LOCAL_REPO="local-reranker" MODEL_NAME="cross-encoder/ms-marco-MiniLM-L-6-v2" ./cicd/build.sh
In order to test the built docker image run this command in project's root folder
LOCAL_REPO="local-reranker" ./cicd/test.sh