What is Argo Workflows?
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).
- Define workflows where each step in the workflow is a container.
- Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a graph (DAG).
- Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes.
- Run CI/CD pipelines natively on Kubernetes without configuring complex software development products.
Argo is a Cloud Native Computing Foundation (CNCF) hosted project.
Why Argo Workflows?
- Designed from the ground up for containers without the overhead and limitations of legacy VM and server-based environments.
- Cloud agnostic and can run on any Kubernetes cluster.
- Easily orchestrate highly parallel jobs on Kubernetes.
- Argo Workflows puts a cloud-scale supercomputer at your fingertips!
Quickstart
kubectl create namespace argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo/stable/manifests/install.yamlWho uses Argo Workflows?
Official Argo Workflows user list
Documentation
Features
- UI to visualize and manage Workflows
- Artifact support (S3, Artifactory, HTTP, Git, GCS, raw)
- Workflow templating to store commonly used Workflows in the cluster
- Archiving Workflows after executing for later access
- Scheduled workflows
- Server interface with REST API
- DAG or Steps based declaration of workflows
- Step level input & outputs (artifacts/parameters)
- Loops
- Parameterization
- Conditionals
- Timeouts (step & workflow level)
- Retry (step & workflow level)
- Resubmit (memoized)
- Suspend & Resume
- Cancellation
- K8s resource orchestration
- Exit Hooks (notifications, cleanup)
- Garbage collection of completed workflow
- Scheduling (affinity/tolerations/node selectors)
- Volumes (ephemeral/existing)
- Parallelism limits
- Daemoned steps
- DinD (docker-in-docker)
- Script steps
- Event emission
- Prometheus metrics
- Multiple executors
- Multiple pod and workflow garbage collection strategies
- Automatically calculated resource usage per step
- Pod Disruption Budget support
Community Blogs and Presentations
- Argo Ansible role: Provisioning Argo Workflows on OpenShift
- Argo Workflows vs Apache Airflow
- CI/CD with Argo on Kubernetes
- Running Argo Workflows Across Multiple Kubernetes Clusters
- Open Source Model Management Roundup: Polyaxon, Argo, and Seldon
- Producing 200 OpenStreetMap extracts in 35 minutes using a scalable data workflow
- Argo integration review
- TGI Kubernetes with Joe Beda: Argo workflow system
- Community meeting minutes and recordings
Project Resources
- Argo GitHub: https://github.com/argoproj
- Argo Website: https://argoproj.github.io/
- Argo Slack: click here to join