Truss is the simplest way to serve AI/ML models in production.
To get you started with Truss, this repository has dozens of example models, each ready to deploy as-is or adapt to your needs. Check out models like Falcon-40B, Starcoder, and Stable Diffusion for inspiration.
Get the repository with:
git clone https://github.com/basetenlabs/truss-examples
Install Truss with:
pip install --upgrade truss baseten
Log in with a Baseten API key:
baseten login
Pick a model to deploy by passing a path to that model.
import truss
import baseten
# Use these variables to pick and name a model
model_path = "wizardlm-truss"
model_name = "WizardLM"
model = truss.load(model_path)
baseten.deploy(model, model_name=model_name)Invocation depends on the model's input and output specifications. See individual model READMEs for invocation details.