llmops

There are 58 repositories under llmops topic.

microsoft / autogen
A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap
agent-based-framework agent-oriented-programming agentic agentic-agi chat chat-application chatbot chatgpt gpt gpt-35-turbo gpt-4 llm-agent llm-framework llm-inference llmops
Language:Jupyter Notebook 25879
jina
jina-ai / jina
☁️ Build multimodal AI applications with cloud-native stack
neural-search cloud-native deep-learning machine-learning framework grpc kubernetes multimodal mlops pipeline fastapi generative-ai docker jaeger llmops opentelemetry cncf microservice orchestration prometheus
Language:Python 20150
vllm-project / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt llm pytorch llmops mlops model-serving transformer llm-serving inference llama amd rocm cuda inferentia trainium
Language:Python 19531
SuperAGI
TransformerOptimus / SuperAGI
<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
agi ai autonomous-agents python agents artificial-intelligence gpt-4 openai artificial-general-intelligence superagi llm llmops nextjs pinecone hacktoberfest
Language:Python 14598
bentoml / OpenLLM
Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.
ai bentoml falcon fine-tuning llama llama2 llm llm-inference llm-ops llm-serving llmops mistral ml mlops model-inference mpt open-source-llm openllm stablelm vicuna
Language:Python 8926
BerriAI / litellm
Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)
anthropic langchain langchain-python llm llmops openai
Language:Python 8890
infiniflow / ragflow
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.
document-understanding llm ocr rag table-structure-recognition data-pipelines deep-learning document-parser information-retrieval llmops machine-learning nlp orchestration pdf-to-text preprocessing retrieval-augmented-generation
Language:Python 7529
phidatahq / phidata
Build AI Assistants with memory, knowledge and tools.
ai aws developer-tools gpt-4 llm llmops python
Language:Python 6842
BentoML
bentoml / BentoML
The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!
model-serving mlops llmops generative-ai llm-inference model-inference-service inference-platform deep-learning llm-serving machine-learning python multimodal ml-engineering llm
Language:Python 6600
liguodongiot / llm-action
本项目旨在分享大模型相关技术原理以及实战经验。
llm llm-inference llm-serving llm-training llmops
Language:HTML 6558
explodinggradients / ragas
Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines
llm llmops
Language:Python 4913
gateway
Portkey-AI / gateway
A Blazing Fast AI Gateway. Route to 100+ LLMs with 1 fast & friendly API.
ai-gateway gateway generative-ai langchain llama-index llmops llms openai prompt-engineering router
Language:TypeScript 4719
superduperdb
SuperDuperDB / superduperdb
🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalable model training and vector search.
ai mlops torch transformers mongodb python pytorch ml database data inference distributed-ml llm-inference pretrained-models chatbot semantic-search llm-serving llmops vector-search rag
Language:Python 4406
langfuse
langfuse / langfuse
🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
analytics llm llmops gpt large-language-models openai self-hosted ycombinator monitoring observability open-source langchain llama-index evaluation prompt-engineering prompt-management playground evals llm-evaluation
Language:TypeScript 3695
zenml
zenml-io / zenml
ZenML 🙏: Build portable, production-ready MLOps pipelines. https://zenml.io.
mlops machine-learning data-science production-ready devops-tools zenml pipelines metadata-tracking deep-learning pytorch tensorflow ml ai automl hacktoberfest workflow llm llmops
Language:Python 3686
giskard
Giskard-AI / giskard
🐢 Open-Source Evaluation & Testing for LLMs and ML models
mlops ml-validation ml-testing ai-testing ai-safety ml-safety llmops ethical-artificial-intelligence responsible-ai fairness-ai trustworthy-ai llm-eval llm-evaluation rag-evaluation ai-security llm-security ai-red-team red-team-tools model-monitoring llm
Language:Python 3191
tensorchord / Awesome-LLMOps
An awesome & curated list of best LLMOps tools for developers
ai-development-tools awesome-list llmops mlops
Language:Shell 3130
promptfoo / promptfoo
Test your prompts, models, and RAGs. Catch regressions and improve prompt quality. LLM evals for OpenAI, Azure, Anthropic, Gemini, Mistral, Llama, Bedrock, Ollama, and other local & private models with CI/CD integration.
ci ci-cd cicd evaluation evaluation-framework llm llm-eval llm-evaluation llm-evaluation-framework llmops prompt-engineering prompt-testing prompts rag testing
Language:TypeScript 2919
phoenix
Arize-ai / phoenix
AI Observability & Evaluation
ml-observability model-observability ai-roi llmops mlops ai-monitoring ai-observability ml-monitoring model-monitoring umap clustering llm-eval
Language:Jupyter Notebook 2745
cube-studio
tencentmusic / cube-studio
cube studio开源云原生一站式机器学习/深度学习AI平台，支持sso登录，多租户/多项目组，大数据平台对接，notebook在线开发，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务VGPU，边缘计算，serverless，标注平台，自动化标注，数据集管理，大模型微调，vllm大模型推理，llmops，私有知识库，AI模型应用商店，支持模型一键开发/推理/微调，支持国产cpu/gpu/npu芯片，支持RDMA，支持pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/spark/ray/volcano分布式
kubernetes inference mlops workflow ai pytorch spark argo kubeflow automl aihub gpt llmops notebook pipeline vgpu
Language:Jupyter Notebook 2573
llm-app
pathwaycom / llm-app
LLM App templates for RAG, knowledge mining, and stream analytics. Ready to run with Docker,⚡in sync with your data sources.
chatbot hugging-face llm llm-local llm-prompting llm-security llmops machine-learning open-ai pathway rag real-time retrieval-augmented-generation vector-database vector-index
Language:Python 2524
Josh-XT / AGiXT
AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.
ai automation artificial chromadb intelligence llama llamacpp openai python agi llm llmops agent-llm agixt
Language:Python 2473
iusztinpaul / hands-on-llms
🦖 𝗟𝗲𝗮𝗿𝗻 about 𝗟𝗟𝗠𝘀, 𝗟𝗟𝗠𝗢𝗽𝘀, and 𝘃𝗲𝗰𝘁𝗼𝗿 𝗗𝗕𝘀 for free by designing, training, and deploying a real-time financial advisor LLM system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 𝘷𝘪𝘥𝘦𝘰 & 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴
bytewax comet-ml huggingface mlops qdrant transformers beam langchain generative-ai llms 3-pipeline-design aws cicd fine-tuning llmops qlora streaming docker
Language:Jupyter Notebook 2431
OpenPipe / OpenPipe
Turn expensive prompts into cheap fine-tuned models
ai llm llmops prompt-engineering
Language:TypeScript 2385
dot-agent / nextpy
🤖Self-Modifying Framework from the Future 🔮 World's First AMS
agent agi ai autogpt gpt llmops mlops ai-agents fastapi fullstack-development llm openai python fastapi-framework fastapi-template pydantic sqlmodel streamlit webdev webdevelopment
Language:Python 2122
ianarawjo / ChainForge
An open-source visual programming environment for battle-testing prompts to LLMs.
ai evaluation large-language-models llmops llms prompt-engineering
Language:TypeScript 2021
uptrain-ai / uptrain
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
autoevaluation evaluation experimentation hallucination-detection jailbreak-detection llm-eval llm-prompting llm-test llmops machine-learning monitoring openai-evals prompt-engineering root-cause-analysis
Language:Python 2016
tensorchord / envd
🏕️ Reproducible development environment
developer-tools development-environment docker buildkit hacktoberfest llmops mlops mlops-workflow model-serving
Language:Go 1922
pezzo
pezzolabs / pezzo
🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.
ai gpt-3 openai devtools gpt-4 prompt-engineering nodejs platform langchain monitoring observability prompt prompt-management javascript nestjs python typescript llm llmops hacktoberfest
Language:TypeScript 1831
microsoft / aici
AICI: Prompts as (Wasm) Programs
ai rust wasm wasmtime inference language-model llm llm-framework llm-inference llm-serving llmops model-serving transformer
Language:Rust 1758
truera / trulens
Evaluation and Tracking for LLM Experiments
explainable-ml llm llmops machine-learning neural-networks
Language:Jupyter Notebook 1665
bionic-gpt / bionic-gpt
BionicGPT is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality
architecture full-stack llmops llms rust
Language:Rust 1623
predibase / lorax
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
fine-tuning gpt llama llm llm-inference llm-serving llmops lora model-serving pytorch transformers
Language:Python 1611
DAGWorks-Inc / hamilton
Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.
data-science python dag data-engineering dataframe etl etl-framework etl-pipeline feature-engineering featurization machine-learning numpy pandas software-engineering data-analysis lineage llmops mlops orchestration hacktoberfest
Language:Jupyter Notebook 1384
traceloop / openllmetry
Open-source observability for your LLM application, based on OpenTelemetry
llmops observability open-telemetry metrics monitoring opentelemetry datascience ml model-monitoring opentelemetry-python python good-first-issue help-wanted artifical-intelligence generative-ai good-first-issues llm open-source
Language:Python 1328
cognita
truefoundry / cognita
RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry
agent ai application data deep-learning fine-tuning framework generative-ai llm llm-ops llmops machine-learning mlops model-deployment python rag retrieval-augmented-generation typescript
Language:Python 1326

llmops

microsoft / autogen

jina-ai / jina

vllm-project / vllm

TransformerOptimus / SuperAGI

bentoml / OpenLLM

BerriAI / litellm

infiniflow / ragflow

phidatahq / phidata

bentoml / BentoML

liguodongiot / llm-action

explodinggradients / ragas

Portkey-AI / gateway

SuperDuperDB / superduperdb

langfuse / langfuse

zenml-io / zenml

Giskard-AI / giskard

tensorchord / Awesome-LLMOps

promptfoo / promptfoo

Arize-ai / phoenix

tencentmusic / cube-studio

pathwaycom / llm-app

Josh-XT / AGiXT

iusztinpaul / hands-on-llms

OpenPipe / OpenPipe

dot-agent / nextpy

ianarawjo / ChainForge

uptrain-ai / uptrain

tensorchord / envd

pezzolabs / pezzo

microsoft / aici

truera / trulens

bionic-gpt / bionic-gpt

predibase / lorax

DAGWorks-Inc / hamilton

traceloop / openllmetry

truefoundry / cognita