mlops

There are 355 repositories under mlops topic.

Made-With-ML
GokuMohandas / Made-With-ML
Learn how to design, develop, deploy and iterate on production-grade ML applications.
machine-learning deep-learning pytorch natural-language-processing data-science python mlops data-engineering data-quality distributed-ml llms ray distributed-training
Language:Jupyter Notebook 35994
airflow
apache / airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
airflow apache apache-airflow python scheduler workflow automation dag data-engineering data-integration data-orchestrator data-pipelines data-science elt etl machine-learning mlops orchestration workflow-engine workflow-orchestration
Language:Python 34694
jina
jina-ai / jina
☁️ Build multimodal AI applications with cloud-native stack
neural-search cloud-native deep-learning machine-learning framework grpc kubernetes multimodal mlops pipeline fastapi generative-ai docker jaeger llmops opentelemetry cncf microservice orchestration prometheus
Language:Python 20151
vllm-project / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
gpt llm pytorch llmops mlops model-serving transformer llm-serving inference llama amd rocm cuda inferentia trainium
Language:Python 19546
qdrant
qdrant / qdrant
Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/
approximate-nearest-neighbor-search embeddings-similarity hacktoberfest hnsw image-search knn-algorithm machine-learning matching mlops nearest-neighbor-search neural-network neural-search recommender-system search search-engine search-engines similarity-search vector-database vector-search vector-search-engine
Language:Rust 18167
label-studio
HumanSignal / label-studio
Label Studio is a multi-type data labeling and annotation tool with standardized output format
computer-vision deep-learning image-annotation annotation-tool annotation labeling labeling-tool image-labeling image-labelling-tool boundingbox image-classification annotations semantic-segmentation dataset datasets label-studio data-labeling text-annotation yolo mlops
Language:JavaScript 16718
EthicalML / awesome-production-machine-learning
A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning
machine-learning mlops interpretability explainability responsible-ai deep-learning machine-learning-operations ml-ops ml-operations privacy-preserving privacy-preserving-ml privacy-preserving-machine-learning data-mining large-scale-ml production-ml large-scale-machine-learning production-machine-learning awesome awesome-list
16178
argoproj / argo-workflows
Workflow Engine for Kubernetes
workflow kubernetes argo dag knative airflow machine-learning argo-workflows workflow-engine hacktoberfest cloud-native cncf k8s gitops mlops batch-processing data-engineering pipelines
Language:Go 14374
nni
microsoft / nni
An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
automl deep-learning neural-architecture-search hyperparameter-optimization distributed bayesian-optimization automated-machine-learning machine-learning machine-learning-algorithms data-science tensorflow pytorch neural-network deep-neural-network model-compression feature-engineering nas python hyperparameter-tuning mlops
Language:Python 13793
visenger / awesome-mlops
A curated list of references for MLOps
machine-learning mlops data-science engineering federated-learning devops software-engineering ai ml
11829
dagster-io / dagster
An orchestration platform for the development, production, and observation of data assets.
analytics dagster data-engineering data-integration data-orchestrator data-pipelines data-science etl metadata mlops orchestration python scheduler workflow workflow-automation
Language:Python 10367
stas00 / ml-engineering
Machine Learning Engineering Open Book
pytorch slurm large-language-models llm machine-learning scalability transformers machine-learning-engineering mlops ai
Language:Python 9907
mlops-zoomcamp
DataTalksClub / mlops-zoomcamp
Free MLOps course from DataTalks.Club
machine-learning mlops model-deployment model-monitoring workflow-orchestration
Language:Jupyter Notebook 9807
weaviate
weaviate / weaviate
Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database.
search-engine semantic-search semantic-search-engine vector-search vector-search-engine vector-database approximate-nearest-neighbor-search image-search hnsw information-retrieval mlops nearest-neighbor-search neural-search recommender-system similarity-search vectors generative-search hybrid-search weaviate grpc
Language:Go 9692
aws / amazon-sagemaker-examples
Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
sagemaker aws reinforcement-learning machine-learning deep-learning examples jupyter-notebook mlops data-science training inference
Language:Jupyter Notebook 9552
great-expectations / great_expectations
Always know what to expect from your data.
pipeline-tests dataquality datacleaning datacleaner data-science data-profiling pipeline pipeline-testing cleandata dataunittest data-unit-tests eda exploratory-data-analysis exploratory-analysis exploratorydataanalysis data-quality data-engineering pipeline-debt data-profilers mlops
Language:Python 9521
kedro
kedro-org / kedro
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
experiment-tracking hacktoberfest kedro machine-learning machine-learning-engineering mlops pipeline python
Language:Python 9390
bentoml / OpenLLM
Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.
ai bentoml falcon fine-tuning llama llama2 llm llm-inference llm-ops llm-serving llmops mistral ml mlops model-inference mpt open-source-llm openllm stablelm vicuna
Language:Python 8928
Avaiga / taipy
Turns Data and AI algorithms into production-ready web applications in no time.
automation data-engineering data-integration data-ops data-visualization datascience developer-tools hacktoberfest hacktoberfest2023 job-scheduler mlops orchestration pipeline pipelines python scenario scenario-analysis taipy-core taipy-gui workflow
Language:Python 8824
chiphuyen / machine-learning-systems-design
A booklet on machine learning systems design with exercises. NOT the repo for the book "Designing Machine Learning Systems"
data-science machine-learning-production mlops
Language:HTML 8553
wandb
wandb / wandb
🔥 A tool for visualizing and tracking your machine learning experiments. This repo contains the CLI and Python API.
machine-learning experiment-track deep-learning keras tensorflow pytorch hyperparameter-search reinforcement-learning mlops data-science collaboration hyperparameter-optimization reproducibility hyperparameter-tuning data-versioning model-versioning ml-platform jax
Language:Python 8311
activeloopai / deeplake
Database for AI. Store Vectors, Images, Texts, Videos, etc. Use with LLMs/LangChain. Store, query, version, & visualize any AI data. Stream data in real-time to PyTorch/TensorFlow. https://activeloop.ai
datasets deep-learning machine-learning data-science pytorch tensorflow data-version-control python ai ml mlops computer-vision cv image-processing datalake langchain llm large-language-models vector-database vector-search
Language:Python 7750
metaflow
Netflix / metaflow
:rocket: Build and manage real-life ML, AI, and data science projects with ease!
ai aws azure data-science datascience gcp high-performance-computing kubernetes machine-learning ml ml-infrastructure ml-platform mlops model-management productivity python r r-package reproducible-research rstats
Language:Python 7644
BentoML
bentoml / BentoML
The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!
model-serving mlops llmops generative-ai llm-inference model-inference-service inference-platform deep-learning llm-serving machine-learning python multimodal ml-engineering llm
Language:Python 6600
clearml
allegroai / clearml
ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
version-control experiment-manager version control experiment deeplearning deep-learning machine-learning machinelearning ai trains trainsai clearml k8s devops mlops
Language:Python 5294
feast-dev / feast
The Open Source Feature Store for Machine Learning
machine-learning features ml big-data feature-store python mlops data-engineering data-science data-quality
Language:Python 5293
flyte
flyteorg / flyte
Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
flyte machine-learning golang scale workflow data-science data-analysis data kubernetes-operator kubernetes orchestration-engine mlops dataops grpc python production production-grade declarative fine-tuning llm
Language:Go 4840
aimhubio / aim
Aim 💫 — An easy-to-use & supercharged open-source experiment tracker.
ai data-science data-visualization experiment-tracking machine-learning metadata metadata-tracking ml mlflow mlops prompt-engineering python pytorch tensorboard tensorflow visualization
Language:Python 4832
evidentlyai / evidently
Evaluate and monitor ML models from validation to production. Join our Discord: https://discord.com/invite/xZjKRaNp8b
data-drift jupyter-notebook pandas-dataframe machine-learning model-monitoring html-report production-machine-learning machine-learning-operations mlops data-science hacktoberfest
Language:Jupyter Notebook 4699
SkalskiP / courses
This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)
computer-vision deep-learning deep-neural-networks machine-learning mlops multimodal transformers tutorial natural-language-processing nlp generative-model stable-diffusion
Language:Python 4581
superduperdb
SuperDuperDB / superduperdb
🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalable model training and vector search.
ai mlops torch transformers mongodb python pytorch ml database data inference distributed-ml llm-inference pretrained-models chatbot semantic-search llm-serving llmops vector-search rag
Language:Python 4407
SeldonIO / seldon-core
An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
kubernetes machine-learning deployment serving mlops aiops machine-learning-operations production-machine-learning
Language:HTML 4237
FedML-AI / FedML
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.
ai-agent deep-learning distributed-training edge-ai federated-learning inference-engine machine-learning mlops model-deployment model-serving on-device-training
Language:Python 4073
pytorch / serve
Serve, optimize and scale PyTorch models in production
cpu deep-learning docker gpu kubernetes machine-learning metrics mlops optimization pytorch serving
Language:Java 3983
ashleve / lightning-hydra-template
PyTorch Lightning + Hydra. A very user-friendly template for ML experimentation. ⚡🔥⚡
pytorch-lightning project-structure config deep-learning hydra template pytorch reproducibility best-practices mlops
Language:Python 3716
zenml
zenml-io / zenml
ZenML 🙏: Build portable, production-ready MLOps pipelines. https://zenml.io.
mlops machine-learning data-science production-ready devops-tools zenml pipelines metadata-tracking deep-learning pytorch tensorflow ml ai automl hacktoberfest workflow llm llmops
Language:Python 3686