Argo - The Workflow Engine for Kubernetes
News
We are thrilled that BlackRock has developed an eventing framework for Argo and has decided to contribute it to the Argo Community. Please check out the new project and try Argo Events!
If you actively use Argo in your organization and believe that your organization may be interested in actively participating in the Argo Community, please ask a representative to contact saradhi_sreegiriraju@intuit.com for additional information.
What is Argo?
Argo is an open source container-native workflow engine for getting work done on Kubernetes. Argo is implemented as a Kubernetes CRD (Custom Resource Definition).
- Define workflows where each step in the workflow is a container.
- Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a graph (DAG).
- Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo workflows on Kubernetes.
- Run CI/CD pipelines natively on Kubernetes without configuring complex software development products.
Why Argo?
- Argo is designed from the ground up for containers without the overhead and limitations of legacy VM and server-based environments.
- Argo is cloud agnostic and can run on any kubernetes cluster.
- Argo with Kubernetes puts a cloud-scale supercomputer at your fingertips.
Documentation
Features
- DAG or Steps based declaration of workflows
- Artifact support (S3, Artifactory, HTTP, Git, raw)
- Step level input & outputs (artifacts/parameters)
- Loops
- Parameterization
- Conditionals
- Timeouts (step & workflow level)
- Retry (step & workflow level)
- Resubmit (memoized)
- Suspend & Resume
- Cancellation
- K8s resource orchestration
- Exit Hooks (notifications, cleanup)
- Garbage collection of completed workflow
- Scheduling (affinity/tolerations/node selectors)
- Volumes (ephemeral/existing)
- Parallelism limits
- Daemoned steps
- DinD (docker-in-docker)
- Script steps
Who uses Argo?
As the Argo Community grows, we'd like to keep track of our users. Please send a PR with your organization name.
Currently officially using Argo:
- Adobe
- BlackRock
- CoreFiling
- Cratejoy
- Cyrus Biotechnology
- Datadog
- Gladly
- Interline Technologies
- Intuit
- KintoHub
- Localytics
- NVIDIA
- SAP Hybris
- Styra
Community Blogs and Presentations
- Open Source Model Management Roundup: Polyaxon, Argo, and Seldon
- Producing 200 OpenStreetMap extracts in 35 minutes using a scalable data workflow
- Argo integration review
- TGI Kubernetes with Joe Beda: Argo workflow system
- Community meeting minutes and recordings
Project Resources
- Argo GitHub: https://github.com/argoproj
- Argo Slack: click here to join
- Argo website: https://argoproj.github.io/
- Argo forum: https://groups.google.com/forum/#!forum/argoproj