Simple fastapi app cloning openai completion api
Download the model from https://huggingface.co/piratos/ct2fast-docsgpt-14b
Make sure an environment variable MODEL_PATH
is accessible to the container runtime
containing the path to the model folder
docker build -t openai_backend
docker run -v <modelpath>:/app/ct2fast-docsgpt-14b -d openai_backend
Adjust mounting volumes
docker compose build openai
docker compose up -d