llmops

There are 122 repositories under llmops topic.

vllm-project / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
amd blackwell cuda deepseek deepseek-v3 gpt gpt-oss inference kimi llama llm llm-serving model-serving moe openai pytorch qwen qwen3 tpu transformer
Language:Python 62461
llm-app
pathwaycom / llm-app
Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.
chatbot hugging-face llm llm-local llm-prompting llm-security llmops machine-learning open-ai pathway rag real-time retrieval-augmented-generation vector-database vector-index
Language:Jupyter Notebook 46565
BerriAI / litellm
Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
ai-gateway anthropic azure-openai bedrock gateway langchain litellm llm llm-gateway llmops mcp-gateway openai openai-proxy vertex-ai
Language:Python 30795
ComposioHQ / composio
Composio equips your AI agents & LLMs with 100+ high-quality integrations via function calling
ai python agents aiagents function-calling developer-tools gpt-4 llm llmops typescript javascript js ai-agents mcp remote-mcp-server sse agentic-ai
Language:TypeScript 25884
mlflow
mlflow / mlflow
The open source developer platform to build AI/LLM applications and models with confidence. Enhance your AI applications with end-to-end tracking, observability, and evaluations, all in one integrated platform.
agentops agents ai ai-governance apache-spark evaluation langchain llm-evaluation llmops machine-learning ml mlflow mlops model-management observability open-source openai prompt-engineering
Language:Python 22849
serve
jina-ai / serve
☁️ Build multimodal AI applications with cloud-native stack
cloud-native cncf deep-learning docker fastapi framework generative-ai grpc jaeger kubernetes llmops machine-learning microservice mlops multimodal neural-search opentelemetry orchestration pipeline prometheus
Language:Python 21779
liguodongiot / llm-action
本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）
llm llm-inference llm-serving llm-training llmops
Language:HTML 21732
langfuse
langfuse / langfuse
🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
analytics autogen evaluation langchain large-language-models llama-index llm llm-evaluation llm-observability llmops monitoring observability open-source openai playground prompt-engineering prompt-management self-hosted ycombinator
Language:TypeScript 18062
SuperAGI
TransformerOptimus / SuperAGI
<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
agents agi ai artificial-general-intelligence artificial-intelligence autonomous-agents gpt-4 hacktoberfest llm llmops nextjs openai pinecone python superagi
Language:Python 16840
raga-ai-hub / RagaAI-Catalyst
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
agentneo agents ai-performance-optimization llm-testing llmops agentic-ai-development ai-agent-monitoring ai-application-debugging ai-evaluation-tools ai-tool-interaction-monitoring llm-tracing agentic-ai
Language:Python 16046
comet-ml / opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
hacktoberfest hacktoberfest2025 langchain llama-index llm llm-evaluation llm-observability llmops open-source openai playground prompt-engineering
Language:Python 15502
bentoml / OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
llm llmops model-inference fine-tuning llm-serving llama vicuna bentoml llama2 llm-inference llm-ops open-source-llm openllm mistral mlops llama3-1 llama3-2 llama3-2-vision
Language:Python 11921
explodinggradients / ragas
Supercharge Your LLM Application Evaluations 🚀
evaluation llm llmops
Language:Python 11356
tensorzero
tensorzero / tensorzero
TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluation, and experimentation.
ai artificial-intelligence deep-learning gpt llm llmops llms machine-learning rust ml mlops anthropic llama openai generative-ai ai-engineering python ml-engineering large-language-models genai
Language:Rust 10519
dataelement / bisheng
BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.
agent ai chatbot enterprise finetune genai gpt langchian llama llm llmdevops llmops ocr openai orchestration python rag react sft workflow
Language:TypeScript 9887
gateway
Portkey-AI / gateway
A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
ai-gateway gateway generative-ai hacktoberfest langchain llm llm-gateway llmops llms mcp mcp-client mcp-gateway mcp-servers model-router openai
Language:TypeScript 9813
metaflow
Netflix / metaflow
Build, Manage and Deploy AI/ML Systems
agents ai aws azure cost-optimization datascience distributed-training gcp generative-ai high-performance-computing kubernetes llm llmops machine-learning ml ml-infrastructure ml-platform mlops model-management python
Language:Python 9610
promptfoo / promptfoo
Test your prompts, agents, and RAGs. AI Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.
ci ci-cd cicd evaluation evaluation-framework llm llm-eval llm-evaluation llm-evaluation-framework llmops pentesting prompt-engineering prompt-testing prompts rag red-teaming testing vulnerability-scanners
Language:TypeScript 8998
BentoML
bentoml / BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
ai-inference deep-learning generative-ai inference-platform llm llm-inference llm-serving llmops machine-learning ml-engineering mlops model-inference-service model-serving multimodal python
Language:Python 8187
phoenix
Arize-ai / phoenix
AI Observability & Evaluation
agents ai-monitoring ai-observability aiengineering anthropic datasets evals langchain llamaindex llm-eval llm-evaluation llmops llms openai prompt-engineering smolagents
Language:Jupyter Notebook 7628
evidentlyai / evidently
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
data-drift jupyter-notebook pandas-dataframe machine-learning model-monitoring html-report mlops data-science hacktoberfest data-quality data-validation generative-ai llm llmops
Language:Jupyter Notebook 6793
traceloop / openllmetry
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
artifical-intelligence datascience generative-ai good-first-issue good-first-issues help-wanted llm llmops metrics ml model-monitoring monitoring observability open-source open-telemetry opentelemetry opentelemetry-python python
Language:Python 6562
tensorchord / Awesome-LLMOps
An awesome & curated list of best LLMOps tools for developers
ai-development-tools awesome-list llmops mlops
Language:Shell 5404
superduper
superduper-io / superduper
Superduper: End-to-end framework for building custom AI applications and agents.
ai chatbot data database distributed-ml inference llm-inference llm-serving llmops ml mlops mongodb pretrained-models python pytorch rag semantic-search torch transformers vector-search
Language:Python 5222
coze-dev / coze-loop
Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring.
agent agent-evaluation agent-observability agentops ai coze eino evaluation langchain llm-observability llmops monitoring observability open-source openai playground prompt-management
Language:Go 5070
zenml
zenml-io / zenml
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
agentops agents ai automl data-science deep-learning devops-tools genai llm llmops machine-learning metadata-tracking ml mlops pipelines production-ready pytorch tensorflow workflow zenml
Language:Python 4997
giskard-oss
Giskard-AI / giskard-oss
🐢 Open-Source Evaluation & Testing library for LLM Agents
agent-evaluation ai-red-team ai-security ai-testing fairness-ai llm llm-eval llm-evaluation llm-security llmops ml-testing ml-validation mlops rag-evaluation red-team-tools responsible-ai trustworthy-ai
Language:Python 4964
0xPlaygrounds / rig
⚙️🦀 Build modular and scalable LLM Applications in Rust
ai llm agent artificial-intelligence automation large-language-model rust scalable-ai generative-ai llmops
Language:Rust 4853
Helicone / helicone
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
large-language-models prompt-engineering agent-monitoring analytics evaluation gpt langchain llama-index llm llm-cost llm-evaluation llm-observability llmops monitoring open-source openai playground prompt-management ycombinator
Language:TypeScript 4699
cube-studio
tencentmusic / cube-studio
cube studio开源云原生一站式机器学习/深度学习/大模型AI平台，mlops算法链路全流程，算力租赁平台，notebook在线开发，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务VGPU虚拟化，边缘计算，标注平台自动化标注，deepseek等大模型sft微调/奖励模型/强化学习训练，vllm/ollama/mindie大模型多机推理，私有知识库，AI模型市场，支持国产cpu/gpu/npu 昇腾生态，支持RDMA，支持pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano等分布式
ai aihub argo automl deepseek gpt inference kubeflow kubernetes llmops mlops notebook pipeline pytorch spark vgpu workflow
Language:Python 4668
PacktPublishing / LLM-Engineers-Handbook
The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices
aws fine-tuning-llm genai llm llm-evaluation llmops ml-system-design mlops rag
Language:Python 4349
katanemo / archgw
The data plane for agents. Arch is a models-native proxy server that handles the plumbing work in AI: agent routing & hand off, guardrails, zero-code logs and traces, unified access to LLMs from OpenAI, Anthropic, Ollama, etc. Build agents faster, and scale them reliably.
ai-gateway ai-gateway-support envoy envoyproxy gateway generative-ai llm-gateway llm-inference llm-proxy llm-routing llmops llms openai prompt proxy proxy-server routing
Language:Rust 4300
cognita
truefoundry / cognita
RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry
agent ai application data deep-learning fine-tuning framework generative-ai llm llm-ops llmops machine-learning mlops model-deployment python rag retrieval-augmented-generation typescript
Language:Python 4276
decodingml / llm-twin-course
🤖 𝗟𝗲𝗮𝗿𝗻 for 𝗳𝗿𝗲𝗲 how to 𝗯𝘂𝗶𝗹𝗱 an end-to-end 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻-𝗿𝗲𝗮𝗱𝘆 𝗟𝗟𝗠 & 𝗥𝗔𝗚 𝘀𝘆𝘀𝘁𝗲𝗺 using 𝗟𝗟𝗠𝗢𝗽𝘀 best practices: ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 12 𝘩𝘢𝘯𝘥𝘴-𝘰𝘯 𝘭𝘦𝘴𝘴𝘰𝘯𝘴
aws bytewax comet-ml course docker generative-ai infrastructure-as-code large-language-models llmops machine-learning-engineering ml-system-design mlops pulumi qdrant qwak rag superlinked
Language:Python 4158
predibase / lorax
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
fine-tuning gpt llama llm llm-inference llm-serving llmops lora model-serving pytorch transformers
Language:Python 3528
iusztinpaul / hands-on-llms
🦖 𝗟𝗲𝗮𝗿𝗻 about 𝗟𝗟𝗠𝘀, 𝗟𝗟𝗠𝗢𝗽𝘀, and 𝘃𝗲𝗰𝘁𝗼𝗿 𝗗𝗕𝘀 for free by designing, training, and deploying a real-time financial advisor LLM system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 𝘷𝘪𝘥𝘦𝘰 & 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴
3-pipeline-design aws beam bytewax cicd comet-ml docker fine-tuning generative-ai huggingface langchain llmops llms mlops qdrant qlora streaming transformers
Language:Jupyter Notebook 3364

llmops

vllm-project / vllm

pathwaycom / llm-app

BerriAI / litellm

ComposioHQ / composio

mlflow / mlflow

jina-ai / serve

liguodongiot / llm-action

langfuse / langfuse

TransformerOptimus / SuperAGI

raga-ai-hub / RagaAI-Catalyst

comet-ml / opik

bentoml / OpenLLM

explodinggradients / ragas

tensorzero / tensorzero

dataelement / bisheng

Portkey-AI / gateway

Netflix / metaflow

promptfoo / promptfoo

bentoml / BentoML

Arize-ai / phoenix

evidentlyai / evidently

traceloop / openllmetry

tensorchord / Awesome-LLMOps

superduper-io / superduper

coze-dev / coze-loop

zenml-io / zenml

Giskard-AI / giskard-oss

0xPlaygrounds / rig

Helicone / helicone

tencentmusic / cube-studio

PacktPublishing / LLM-Engineers-Handbook

katanemo / archgw

truefoundry / cognita

decodingml / llm-twin-course

predibase / lorax

iusztinpaul / hands-on-llms