workflow-orchestration

There are 3 repositories under workflow-orchestration topic.

airflow
apache / airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
airflow apache apache-airflow python scheduler workflow automation dag data-engineering data-integration data-orchestrator data-pipelines data-science elt etl machine-learning mlops orchestration workflow-engine workflow-orchestration
Language:Python 34721
apache / dolphinscheduler
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
workflow-schedule azkaban airflow task-scheduler job-scheduler cloud-native data-pipelines orchestration workflow workflow-orchestration powerful-data-pipelines
Language:Java 12198
mlops-zoomcamp
DataTalksClub / mlops-zoomcamp
Free MLOps course from DataTalks.Club
machine-learning mlops model-deployment model-monitoring workflow-orchestration
Language:Jupyter Notebook 9899
awesome-argo
akuity / awesome-argo
A curated list of awesome projects and resources related to Argo (a CNCF graduated project)
awesome awesome-list awesome-lists argocd argo-workflows argo-events argo-rollouts machine-learning workflow-engine workflow-management infrastructure-as-code continuous-delivery cloud-native kubernetes cncf gitops workflow-orchestration devops mlops argo
1819
polaris
harshadmanglani / polaris
Polaris: High performance workflow orchestrator for Golang
go golang orchestrator workflow workflow-engine workflow-orchestration workflow-orchestrator
Language:Go 249
awesome-kubeflow
terrytangyuan / awesome-kubeflow
A curated list of awesome projects and resources related to Kubeflow (a CNCF incubating project)
awesome awesome-list awesome-lists cloud-native cncf deep-learning devops kubeflow kubeflow-pipelines kubernetes machine-learning mlops workflow-orchestration data-pipelines data-science mle-pipelines
182
iobruno / data-engineering-zoomcamp
Data Engineering examples covering Airflow and Mage for workflows; dbt for BigQuery, Redshift, ClickHouse; Spark and Kafka for Batch/Streaming Processing
pyspark spark spark-sql kafka kafka-streams ksqldb workflow-orchestration dbt-bigquery dbt-postgres dbt-clickhouse dbt-redshift
Language:Python 47
sartography / spiff-arena
SpiffWorkflow is a software development platform for building, running, and monitoring executable diagrams
bpmn spiffworkflow bpm business-process flowchart orchestration-framework process-engine python workflow workflow-engine workflow-orchestration
Language:Python 44
zetane / ZetaForge
Open source AI platform for rapid development of advanced AI and AGI pipelines.
agi ai data-science developer-tools kubernetes llm machine-learning ml ml-pipelines mlops python workflow zetaforge workflow-orchestration claude gpt
Language:JavaScript 40
sarahmk125 / airflow-docker-metrics
dataengineering airflow-metrics statsd-format grafana airflow-containers statsd-metrics data orchestration workflow-orchestration
Language:Python 39
hadron-project / hadron
The Kubernetes native and CloudEvents native distributed event streaming, event orchestration & messaging platform
events event-driven messaging streams pipelines dag rpc kubernetes orchestration workflows workflow-orchestration cloudevents rust pubsub
Language:Rust 35
zzstoatzz / prefect-memory-profiling
memory-profiling prefect python workflow-orchestration
Language:Python 22
Nerdward / Mlops_project
An end to end ML project. Using MLflow for experiment tracking and model registry. Prefect for workflow orchestration. S3 for artifacts storage. AWS Lambda/ ECR for serverless model serving. AWS REST API gateway as endpoint to lambda function. GitHub Actions for CI/CD.
aws continuous-delivery continuous-integration devops experiment-tracking infrastructure-as-code machine-learning model-registry workflow-orchestration
Language:Jupyter Notebook 11
Node-M2M / m2m
m2m is a lightweight communication library for developing client-server applications using the machine-to-machine framework node-m2m.
iot m2m telematics telemetry automation machine-control workflow-orchestration distributed-systems iiot industry-4 javascript microservices nodejs orchestration-framework raspberrypi smart-manufacturing cloud-computing edge edge-computing
Language:JavaScript 10
clamp-orchestrator / clamp-core
A microservices flow orchestrator and workflow manager.
clamp microservices orchestration orchestrator workflow-management workflow-orchestration workflow-orchestrator
Language:Go 9
cmigpereira / mlops-zoomcamp-project
Repo for my MLOps Zoomcamp project
machine-learning workflow-orchestration mlops model-deployment model-monitoring
Language:Python 6
anna-geller / kestra-terraform-examples
Bring Infrastructure as Code best practices to your data workflows with Kestra and Terraform
automation aws data-engineering data-infrastructure data-orchestration infrastructure-as-code kestra platform-engineering terraform workflow-as-code workflow-orchestration
Language:HCL 5
functionless / eventual-eda-patterns
examples of Event-Driven Architectures (EDA) and Domain Driven Design (DDD) patterns implemented using eventual
aws-cdk domain-driven-design eda event-driven event-driven-architecture micro-services pulumi serverless workflow-engine workflow-orchestration
Language:TypeScript 2
ashkan-mokarian / mlops-zoomcamp
Notes, homeworks, and projects for MLOps Zoomcamp @ datatalks.club
learning mlops model-deployment model-monitoring workflow-orchestration
Language:Jupyter Notebook 1
beam-engine / core
State Machine Core Package
orchestration state-machine workflow-engine workflow-orchestration
Language:Go 0
ManthanShettigar / mlops_Zoomcamp_2023
mlops zoomcamp 2023 solutions
ai data-science machine-learning mlflow mlops experiment-tracking prefect weights-and-biases workflow-orchestration deployment docker python docker-compose evidently grafana-dashboard model-monitoring monitoring postgresql batch-deployment
Language:Jupyter Notebook 0
AbhishekRS4 / Data_Ingestion_Prefect
Data Ingestion pipeline orchestration with Prefect
data-ingestion prefect workflow-orchestration
Language:Python
topahande / data-engineering-project
Video games sales
elt etl google-cloud-bigquery google-cloud-platform google-cloud-storage prefect terraform workflow-orchestration looker-studio
Language:Python