OpenRouter Runner
OpenRouter Runner is a monolith inference engine, built with Modal, used for lots of the open source models hosted in a fallback capacity on openrouter.ai.
Engines
- vLLM
- HF Transformers
Getting Started
cd modal- Select a modal app, like the runner
- Follow the steps in the project README.
Contributions
Interested in contributing? Please read our contributing guide and follow our code of conduct.