sapk / armada

A multi-cluster batch queuing system for high-throughput workloads on Kubernetes.

Armada is an application to achieve high throughput of run-to-completion jobs on multiple Kubernetes clusters.

It stores queues for users/projects with pod specifications and creates these pods once there is available resource in one of the connected Kubernetes clusters.

Documentation

• Quickstart
• User Guide
• Installation in Production
• Design Documentation
• Development Guide
• Api Documentation

Key features

• Armada maintains fair resource share over time (inspired by HTCondor priority)
• It can handle large amounts of queued jobs (million+)
• It allows adding and removing clusters from the system without disruption
• By utilizing multiple Kubernetes clusters the system can scale beyond the limits of a single Kubernetes cluster

Key concepts

Queue: Represents a user or project, used to maintain fair share over time, with a priority factor

Job: Unit of work to be run (described as a Kubernetes PodSpec)

Job Set: Group of related jobs. The API allows observing progress of a job set together

Try it out locally

Follow the quickstart guide to get Armada up and running locally.