llm-gateway
There are 6 repositories under llm-gateway topic.
- Language:Lua 42141
BerriAI / litellm
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
Language:Python 30796- gateway
Portkey-AI / gateway
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
Language:TypeScript 9813 - coai
coaidev / coai
🚀 Next Generation AI One-Stop Internationalization Solution. 🚀 下一代 AI 一站式 B/C 端解决方案，支持 OpenAI，Midjourney，Claude，讯飞星火，Stable Diffusion，DALL·E，ChatGLM，通义千问，腾讯混元，360 智脑，百川 AI，火山方舟，新必应，Gemini，Moonshot 等模型，支持对话分享，自定义预设，云端同步，模型市场，支持弹性计费和订阅计划模式，支持图片解析，支持联网搜索，支持模型缓存，丰富美观的后台管理与仪表盘数据统计。
Language:TypeScript 8800 katanemo / archgw
The data plane for agents. Arch is a models-native proxy server that handles the plumbing work in AI: agent routing & hand off, guardrails, zero-code logs and traces, unified access to LLMs from OpenAI, Anthropic, Ollama, etc. Build agents faster, and scale them reliably.
Language:Rust 4303APIParkLab / APIPark
Cloud native, ultra-high performance AI&API gateway, LLM API management, distribution system, open platform, supporting all AI APIs.🦄云原生、超高性能 AI&API网关，LLM API 管理、分发系统、开放平台，支持所有AI API，不限于OpenAI、Azure、Anthropic Claude、Google Gemini、DeepSeek、字节豆包、ChatGLM、文心一言、讯飞星火、通义千问、360 智脑、腾讯混元等主流模型，统一 API 请求和返回，API申请与审批，调用统计、负载均衡、多模型灾备。一键部署，开箱即用。
Language:TypeScript 1458maximhq / bifrost
Fastest LLM gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.
Language:Go 1018langdb / ai-gateway
Govern, Secure, and Optimize your AI Traffic. AI Gateway provides unified interface to all LLMs using OpenAI API format with a focus on performance and reliability. Built in Rust.
Language:Rust 676LianjiaTech / bella-openapi
Bella OpenAPI是一个提供了丰富的AI调用能力的API网关，可类比openrouter，与之不同的是除了提供聊天补全(chat-completion)能力外，还提供了文本向量化(text-embedding)、语音识别(ASR)、语音合成(TTS)、文生图、图生图等多种AI能力，同时集成了计费、限流和资源管理功能。且集成的所有能力都经过了大规模生产环境的验证。
Language:Java 169- oss-llmops-stack
langfuse / oss-llmops-stack
Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability, prompt versioning, and performance evaluations.
starbased-co / ccproxy
Customize and Extend Claude Code with ccproxy: Route to OpenAI, Gemini, Qwen, OpenRouter, and Ollama. Gain full control of your Claude Max/Pro Subscription with your own router.
Language:Python 86bluewave-labs / LangRoute
This is a robust and configurable LLM proxy server built with Node.js, Express, and PostgreSQL. It acts as an intermediary between your applications and various Large Language Model (LLM) providers
Language:TypeScript 28teilomillet / hapax
The reliability layer between your code and LLM providers.
Language:Go 22fabiojbg / LLMApiGateway
A personal LLM gateway with fault-tolerant capabilities for calls to LLM models from any provider with OpenAI-compatible APIs. Advanced features like retry, model sequencing, and body parameter injection are also available. Especially useful to work with AI coders like Cline and RooCode and providers like OpenRouter.
Language:Python 19hypersigilhq / hypersigil
Prompt management gateway with a UI for AI-powered applications. Enables non-technical users to test, refine, and deploy prompts seamlessly across multiple AI providers.
Language:Vue 18shengyanli1982 / llmproxy
🧭🧭 An intelligent load balancer with smart scheduling that unifies diverse LLMs.
Language:Rust 16cloud-apim / otoroshi-llm-extension
Connect, setup, secure and seamlessly manage LLM models using an Universal/OpenAI compatible API
Language:Scala 11ozanunal0 / Prometheus-Gateway
An open-source, security-first LLM Gateway designed to provide a unified, secure, and observable entry point to any Large Language Model.
Language:Python 9- gateway
burgonet-eu / gateway
Burgonet Gateway is an enterprise LLM gateway that provides secure access and compliance controls for AI systems
Language:Rust 6 - Language:C# 5
Tanantor / langgate
Lightweight AI inference gateway - local model registry & parameter transformer (Python SDK) - with optional Envoy proxy processor and FastAPI registry server deployment options.
Language:Python 4charmitro / ai-proxy
A lightweight proxy for LLM API calls with guardrails, metrics, and monitoring. A vibe coding experiment.
Language:Go 3- Language:Rust 2
billebel / splunk-community-ai
A secure, governable AI gateway for Splunk with operational guardrails. An alternative to Splunk AI Assistant focused on safety, compliance, and predictable results using a 'Configuration as Code' approach."
Language:Python 1claymore666 / MCPConnectionHub
MCP Connection Hub - A unified Model Context Protocol Gateway
Language:Python 1- frontllm-sdk
frontllm / frontllm-sdk
FrontLLM is your public gateway to LLMs. Request LLM directly from your front-end code. No backend needed.
Language:TypeScript 1 jonthanliu / gemini-gateway
An intelligent AI gateway that provides a unified API for Google's Gemini models, featuring load balancing, failover, and a real-time management dashboard. Seamlessly integrate Gemini into any application with support for OpenAI or Anthropic APIs. 智能AI网关，为Google Gemini模型提供统一的API接口，支持负载均衡、故障转移和实时管理仪表盘。将Gemini集成到任何支持OpenAI或Anthropic API的客户端。
Language:TypeScript 1nazdridoy / llm-proxy
A robust, transparent REST API proxy service for LLM APIs with profile-based configuration and comprehensive logging.
Language:JavaScript 1oxtx / hapax
The reliability layer between your code and LLM providers. Hapax is a production-ready AI infrastructure layer that ensures uninterrupted AI operations through intelligent provider management and automatic failover. It is designed to address common challenges in managing AI infrastructure
Language:Go 1harshita-d / Conductor_LLM_Platform
Enterprise LLM Orchestration Platform - Intelligently route requests across multiple AI providers with cost optimisation and real-time monitoring
Language:Python 0scampion / burgonet-gateway
Burgonet Gateway is an enterprise LLM gateway that provides secure access and compliance controls for AI systems
Language:Rust 0viniViado / LLMSecOps
LLMSecOps focuses on integrating security practices within the lifecycle of machine learning models. It ensures that models are robust against threats while maintaining compliance and performance standards.
0AICoreDirector / AICoreDirector
AICoreDirector - an enterprise AI governance & Orchestration platform. it unifies all LLM calls, AI components, and traditional ML services behind a single interface. It provides intelligent routing and scheduling, fallback, and integrated management—serving as the enterprise AI capability bus and central hub for production.
Language:Python- Language:Python
