mlops

There are 347 repositories under mlops topic.

Made-With-ML
GokuMohandas / Made-With-ML
Learn how to design, develop, deploy and iterate on production-grade ML applications.
machine-learning deep-learning pytorch natural-language-processing data-science python mlops data-engineering data-quality distributed-ml llms ray distributed-training
Language:Jupyter Notebook 35610
airflow
apache / airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
airflow apache apache-airflow python scheduler workflow automation dag data-engineering data-integration data-orchestrator data-pipelines data-science elt etl machine-learning mlops orchestration workflow-engine workflow-orchestration
Language:Python 34405
jina
jina-ai / jina
☁️ Build multimodal AI applications with cloud-native stack
cloud-native cncf deep-learning docker fastapi framework generative-ai grpc jaeger kubernetes llmops machine-learning microservice mlops multimodal neural-search opentelemetry orchestration pipeline prometheus
Language:Python 19999
vllm-project / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
amd cuda gpt inference inferentia llama llm llm-serving llmops mlops model-serving pytorch rocm trainium transformer
Language:Python 17931
qdrant
qdrant / qdrant
Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/
approximate-nearest-neighbor-search embeddings-similarity hacktoberfest hnsw image-search knn-algorithm machine-learning matching mlops nearest-neighbor-search neural-network neural-search recommender-system search search-engine search-engines similarity-search vector-database vector-search vector-search-engine
Language:Rust 17795
label-studio
HumanSignal / label-studio
Label Studio is a multi-type data labeling and annotation tool with standardized output format
computer-vision deep-learning image-annotation annotation-tool annotation labeling labeling-tool image-labeling image-labelling-tool boundingbox image-classification annotations semantic-segmentation dataset datasets label-studio data-labeling text-annotation yolo mlops
Language:JavaScript 16404
EthicalML / awesome-production-machine-learning
A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning
awesome awesome-list data-mining deep-learning explainability interpretability large-scale-machine-learning large-scale-ml machine-learning machine-learning-operations ml-operations ml-ops mlops privacy-preserving privacy-preserving-machine-learning privacy-preserving-ml production-machine-learning production-ml responsible-ai
15919
argoproj / argo-workflows
Workflow Engine for Kubernetes
workflow kubernetes argo dag knative airflow machine-learning argo-workflows workflow-engine hacktoberfest cloud-native cncf k8s gitops mlops batch-processing data-engineering pipelines
Language:Go 14264
nni
microsoft / nni
An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
automl deep-learning neural-architecture-search hyperparameter-optimization distributed bayesian-optimization automated-machine-learning machine-learning machine-learning-algorithms data-science tensorflow pytorch neural-network deep-neural-network model-compression feature-engineering nas python hyperparameter-tuning mlops
Language:Python 13725
visenger / awesome-mlops
A curated list of references for MLOps
ai data-science devops engineering federated-learning machine-learning ml mlops software-engineering
11707
dagster-io / dagster
An orchestration platform for the development, production, and observation of data assets.
analytics dagster data-engineering data-integration data-orchestrator data-pipelines data-science etl metadata mlops orchestration python scheduler workflow workflow-automation
Language:Python 10166
stas00 / ml-engineering
Machine Learning Engineering Open Book
pytorch slurm large-language-models llm machine-learning scalability transformers machine-learning-engineering mlops ai
Language:Python 9712
aws / amazon-sagemaker-examples
Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
sagemaker aws reinforcement-learning machine-learning deep-learning examples jupyter-notebook mlops data-science training inference
Language:Jupyter Notebook 9489
great-expectations / great_expectations
Always know what to expect from your data.
cleandata data-engineering data-profilers data-profiling data-quality data-science data-unit-tests datacleaner datacleaning dataquality dataunittest eda exploratory-analysis exploratory-data-analysis exploratorydataanalysis mlops pipeline pipeline-debt pipeline-testing pipeline-tests
Language:Python 9437
weaviate
weaviate / weaviate
Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database.
approximate-nearest-neighbor-search generative-search grpc hnsw hybrid-search image-search information-retrieval mlops nearest-neighbor-search neural-search recommender-system search-engine semantic-search semantic-search-engine similarity-search vector-database vector-search vector-search-engine vectors weaviate
Language:Go 9436
kedro
kedro-org / kedro
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
experiment-tracking hacktoberfest kedro machine-learning machine-learning-engineering mlops pipeline python
Language:Python 9347
mlops-zoomcamp
DataTalksClub / mlops-zoomcamp
Free MLOps course from DataTalks.Club
machine-learning mlops model-deployment model-monitoring workflow-orchestration
Language:Jupyter Notebook 8731
bentoml / OpenLLM
Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint, locally and in the cloud.
llm llmops model-inference falcon fine-tuning stablelm llm-serving llama mpt vicuna bentoml llama2 llm-inference llm-ops open-source-llm openllm ai mistral ml mlops
Language:Python 8697
chiphuyen / machine-learning-systems-design
A booklet on machine learning systems design with exercises. NOT the repo for the book "Designing Machine Learning Systems"
data-science machine-learning-production mlops
Language:HTML 8357
Avaiga / taipy
Turns Data and AI algorithms into production-ready web applications in no time.
automation data-engineering data-integration data-ops data-visualization datascience developer-tools hacktoberfest hacktoberfest2023 job-scheduler mlops orchestration pipeline pipelines python scenario scenario-analysis taipy-core taipy-gui workflow
Language:Python 8308
wandb
wandb / wandb
🔥 A tool for visualizing and tracking your machine learning experiments. This repo contains the CLI and Python API.
collaboration data-science data-versioning deep-learning experiment-track hyperparameter-optimization hyperparameter-search hyperparameter-tuning jax keras machine-learning ml-platform mlops model-versioning pytorch reinforcement-learning reproducibility tensorflow
Language:Python 8181
activeloopai / deeplake
Database for AI. Store Vectors, Images, Texts, Videos, etc. Use with LLMs/LangChain. Store, query, version, & visualize any AI data. Stream data in real-time to PyTorch/TensorFlow. https://activeloop.ai
datasets deep-learning machine-learning data-science pytorch tensorflow data-version-control python ai ml mlops computer-vision cv image-processing datalake langchain llm large-language-models vector-database vector-search
Language:Python 7686
metaflow
Netflix / metaflow
:rocket: Build and manage real-life ML, AI, and data science projects with ease!
machine-learning data-science productivity model-management ai ml ml-platform ml-infrastructure python r rstats reproducible-research r-package mlops datascience high-performance-computing kubernetes aws azure gcp
Language:Python 7566
BentoML
bentoml / BentoML
The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!
model-serving model-deployment model-management ml-platform ai machine-learning mlops bentoml kubernetes deep-learning microservices llmops lmops generative-ai ai-infra inference-api model-inference multimodal-deep-learning
Language:Python 6522
feast-dev / feast
Feature Store for Machine Learning
big-data data-engineering data-quality data-science feature-store features machine-learning ml mlops python
Language:Python 5250
clearml
allegroai / clearml
ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution
version-control experiment-manager version control experiment deeplearning deep-learning machine-learning machinelearning ai trains trainsai clearml k8s devops mlops
Language:Python 5224
aimhubio / aim
Aim 💫 — An easy-to-use & supercharged open-source experiment tracker.
python ai data-science data-visualization experiment-tracking machine-learning metadata metadata-tracking ml mlflow mlops pytorch tensorboard tensorflow visualization prompt-engineering
Language:Python 4762
flyte
flyteorg / flyte
Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
flyte machine-learning golang scale workflow data-science data-analysis data kubernetes-operator kubernetes orchestration-engine mlops dataops grpc python production production-grade declarative fine-tuning llm
Language:Go 4727
evidentlyai / evidently
Evaluate and monitor ML models from validation to production. Join our Discord: https://discord.com/invite/xZjKRaNp8b
data-drift data-science hacktoberfest html-report jupyter-notebook machine-learning machine-learning-operations mlops model-monitoring pandas-dataframe production-machine-learning
Language:Jupyter Notebook 4619
SkalskiP / courses
This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)
computer-vision deep-learning deep-neural-networks generative-model machine-learning mlops multimodal natural-language-processing nlp stable-diffusion transformers tutorial
Language:Python 4439
superduperdb
SuperDuperDB / superduperdb
🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalable model training and vector search.
ai mlops torch transformers mongodb python pytorch ml database data inference distributed-ml llm-inference pretrained-models chatbot semantic-search llm-serving llmops vector-search rag
Language:Python 4325
SeldonIO / seldon-core
An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
aiops deployment kubernetes machine-learning machine-learning-operations mlops production-machine-learning serving
Language:HTML 4206
FedML-AI / FedML
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, FEDML Nexus AI (https://fedml.ai) is your generative AI platform at scale.
federated-learning deep-learning distributed-training edge-ai machine-learning on-device-training inference-engine mlops model-deployment model-serving ai-agent
Language:Python 4055
pytorch / serve
Serve, optimize and scale PyTorch models in production
pytorch machine-learning mlops serving docker kubernetes optimization cpu gpu metrics deep-learning
Language:Java 3947
ashleve / lightning-hydra-template
PyTorch Lightning + Hydra. A very user-friendly template for ML experimentation. ⚡🔥⚡
pytorch-lightning project-structure config deep-learning hydra template pytorch reproducibility best-practices mlops
Language:Python 3652
zenml
zenml-io / zenml
ZenML 🙏: Build portable, production-ready MLOps pipelines. https://zenml.io.
mlops machine-learning data-science production-ready devops-tools zenml pipelines metadata-tracking deep-learning pytorch tensorflow ml ai automl hacktoberfest workflow llm llmops
Language:Python 3647