"Click on the image to get started in 4 minutes with Kestra."
Try Kestra using our live demo.
Kestra is a universal open-source orchestrator that makes both scheduled and event-driven workflows easy. By bringing Infrastructure as Code best practices to data, process, and microservice orchestration, you can build reliable workflows and manage them with confidence.
In just a few lines of code, you can create a flow directly from the UI. Thanks to the declarative YAML interface for defining orchestration logic, business stakeholders can participate in the workflow creation process.
Kestra offers a versatile set of language-agnostic developer tools while simultaneously providing an intuitive user interface tailored for business professionals. The YAML definition gets automatically adjusted any time you make changes to a workflow from the UI or via an API call. Therefore, the orchestration logic is always managed declaratively in code, even if some workflow components are modified in other ways (UI, CI/CD, Terraform, API calls).
Flowis the main component in Kestra. It's a container for your tasks and orchestration logic.Namespaceis used to provide logical isolation, e.g., to separate development and production environments. Namespaces are like folders on your file system — they organize flows into logical categories and can be nested to provide a hierarchical structure.Tasksare atomic actions in a flow. By default, all tasks in the list will be executed sequentially, with additional customization options, a.o. to run tasks in parallel or allow a failure of specific tasks when needed.Triggersdefine when a flow should run. In Kestra, flows are triggered based on events. Examples of such events include:- a regular time-based schedule
- an API call (webhook trigger)
- ad-hoc execution from the UI
- a flow trigger - flows can be triggered from other flows using a flow trigger or a subflow, enabling highly modular workflows.
- custom events, including a new file arrival (file detection event), a new message in a message bus, query completion, and more.
Inputsallow you to pass runtime-specific variables to a flow. They are strongly typed, and allow additional validation rules.
Most tasks in Kestra are available as plugins, but many type of tasks are available in the core library, including a.o. script tasks supporting various programming languages (e.g., Python, Node, Bash) and the ability to orchestrate your business logic packaged into Docker container images.
To create your own plugins, check the plugin developer guide.
Kestra provides a variety of tasks to handle both simple and complex business logic, including:
- subflows
- retries
- timeout
- error handling
- conditional branching
- dynamic tasks
- sequential and parallel tasks
- skipping tasks or triggers when needed by setting the flag
disabledtotrue. - configuring dependencies between tasks, flows and triggers
- advanced scheduling and trigger conditions
- backfills
- blueprints
- documenting your flows, tasks and triggers by adding a markdown description to any component
- adding labels to add additional metadata to your flows such as the flow owner or team:
id: getting_started
namespace: dev
description: |
# Getting Started
Let's `write` some **markdown** - [first flow](https://t.ly/Vemr0) 🚀
labels:
owner: rick.astley
project: never-gonna-give-you-up
tasks:
- id: hello
type: io.kestra.core.tasks.log.Log
message: Hello world!
description: a *very* important task
disabled: false
timeout: PT10M
retry:
type: constant # type: string
interval: PT15M # type: Duration
maxDuration: PT1H # type: Duration
maxAttempt: 5 # type: int
warningOnRetry: true # type: boolean, default is false
- id: parallel
type: io.kestra.core.tasks.flows.Parallel
concurrent: 3
tasks:
- id: task1
type: io.kestra.plugin.scripts.shell.Commands
commands:
- 'echo "running {{task.id}}"'
- 'sleep 2'
- id: task2
type: io.kestra.plugin.scripts.shell.Commands
commands:
- 'echo "running {{task.id}}"'
- 'sleep 1'
- id: task3
type: io.kestra.plugin.scripts.shell.Commands
commands:
- 'echo "running {{task.id}}"'
- 'sleep 3'
triggers:
- id: schedule
type: io.kestra.core.models.triggers.types.Schedule
cron: "*/15 * * * *"
backfill:
start: 2023-10-05T14:00:00ZYou can write workflows directly from the UI. When writing your workflows, the UI provides:
- autocompletion
- syntax validation
- embedded plugin documentation
- example flows provided as blueprints
- topology view (view of your dependencies in a Directed Acyclic Graph) that get updated live as you modify and add new tasks.
We release new versions every month. Give the repository a star to stay up to date with the latest releases and get notified about future updates.
Follow the steps below to start local development.
Make sure that Docker is installed and running on your system. The default installation requires the following:
Download the Docker Compose file:
curl -o docker-compose.yml https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.ymlAlternatively, you can use wget https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml.
Start Kestra:
docker compose up -dOpen http://localhost:8080 in your browser and create your first flow.
Here is a simple example logging hello world message to the terminal:
id: getting_started
namespace: dev
tasks:
- id: hello_world
type: io.kestra.core.tasks.log.Log
message: Hello World!For more information:
- Follow the getting started tutorial.
- Read the documentation to understand how to:
- Develop your flows
- Deploy Kestra
- Use our Terraform provider to deploy your flows
- Develop your own plugins.
Kestra is built on a plugin system. You can find your plugin to interact with your provider; alternatively, you can follow these steps to develop your own plugin.
For a full list of plugins, check the plugins page.
Here are some examples of the available plugins:
This list is growing quickly and we welcome contributions.
If you need help or have any questions, reach out using one of the following channels:
- Slack - join the community and get the latest updates.
- GitHub discussions - useful to start a conversation that is not a bug or feature request.
- Twitter - to follow up with the latest updates.
We love contributions, big or small. Check out our contributor guide for details on how to contribute to Kestra.
See our Plugin Developer Guide for details on developing and publishing Kestra plugins.
Apache 2.0 © Kestra Technologies