Helios is a Docker orchestration platform for deploying and managing containers across an entire fleet of servers. Helios provides a HTTP API as well as a command line client to interact with servers running your containers. It also keeps history of events in your cluster including information such as deploys, restarts and version changes.
# Create an nginx job using the nginx container image, exposing it on the host on port 8080
$ helios create nginx:v1 nginx:1.7.1 -p http=80:8080
# Check that the job is listed
$ helios jobs
# List helios hosts
$ helios hosts
# Deploy the nginx job on one of the hosts
$ helios deploy nginx:v1 <host>
# Check the job status
$ helios status
# Curl the nginx container when it's started running
$ curl <host>:8080
# Undeploy the nginx job
$ helios undeploy -a nginx:v1
# Remove the nginx job
$ helios remove nginx:v1The helios command line tool connects to your helios master via HTTP. The
Helios master is connected to a Zookeeper cluster that is used both as
persistent storage and as a communications channel to the agents. The
helios agent is a java process that typically lives on the same host as
the Docker daemon, connecting to it via a Unix socket or optionally TCP
socket.
Helios is designed for high availability, with execution state being confined to a potentially highly available Zookeeper cluster. This means that several helios-master services can respond to HTTP requests concurrently, removing any single point of failure in the helios setup using straight forward HTTP load balancing strategies.
We at Spotify are running this in production now (as of early July 2014) with a money-generating service, so we trust it. Whether you should trust it to not cause smoking holes in your infrastructure is up to you.
There are a number of Docker orchestration systems, why should you choose Helios?
-
Helios is pragmatic. We're not trying to solve everything today, but what we have, we try hard to ensure is rock-solid. So we don't have things like resource limits or dynamic scheduling yet. Today, for us, it has been more important to get the CI/CD use cases, and surrounding tooling solid first. That said, we eventually want to do dynamic scheduling, composite jobs, etc. (see below for more). But what we provide, we use (i.e. we eat our own dogfood), so you can have reasonable assurances that anything that's been in the codebase for more than a week or two is pretty solid as we release frequently (usually, at least weekly) into production here at Spotify.
-
We do not to pretend to have features we don't have.
-
Helios should be able to fit in the way you already do ops. Of the popular Docker Orchestration frameworks, Helios is the only one we're aware of that doesn't have anything much in the way of system dependencies. That is, we don't require that you run in AWS or GCE, etc. We don't require a specific network topology. We don't require you run a specific operating system. We don't require that you're using Mesos. Our only requirement is that you have a ZooKeeper cluster somewhere and a Java 7 JVM on the machines which Helios runs on. So if you're using Puppet, Chef, etc. to manage the rest of the OS install and configuration, you can still continue to do so with whatever Linux OS you're using.
-
Don't have to drink all the Kool-Aid. Generally, we try to make it so you only have to take the features you want to use, and should be able to ignore the rest. For example, Helios doesn't prescribe a discovery service: we happen to provide a plugin for SkyDNS, and we hear that someone else is working on one for another service, but if you don't want to even use a discovery service, you don't have to.
-
Scalability. We're already at well over a hundred machines in production, but we're nowhere near the limit before the existing architecture would need to be revisited. Helios can also scale down well in that you can run a single machine instance if you want to run it all locally.
If you're looking for how to use Helios, see the docs directory. Most probably the User Manual is what you're looking for.
If you're looking for how to download, build, install and run Helios, keep reading.
The binary release of Helios is built for Ubuntu 14.04.1 LTS, but Helios should be buildable on any platform with at least Java 7 and a recent Maven 3 available.
Other components are that are required for a helios installation are:
- Docker 1.0 or newer
- Zookeeper 3.4.0 or newer
You can download Ubuntu packages from our releases page. Though if you want to build it yourself, the process described below is the same process that generates those packages.
First, make sure you have Docker installed locally. If you're using OS X, you can use the included Vagrantfile to bring up Docker inside of a VM:
$ vagrant up
# set DOCKER_HOST to use the Docker instance inside the VM
$ export DOCKER_HOST=tcp://192.168.33.10:2375Actually building Helios and running its tests should be a simple matter of running:
$ mvn clean test
If you have Vagrant installed locally, just run:
$ vagrant up
This will start a virtual machine, install the prereqs, build the project, and install and run the Helios agent and master. You can then use the Vagrant environment:
$ vagrant ssh
vagrant@ubuntu-14:~$ helios hosts
HOST STATUS DEPLOYED RUNNING CPUS MEM LOAD AVG MEM USAGE OS VERSION
192.168.33.10 Down 0 0 2 0 gb 0.09 0.81 Linux 3.13.0-24-generic
Note that the included Vagrantfile doesn't run the test suite when building Helios. This is so
we can get you up and running with a VM as quickly as possible. You should run mvn package
yourself to run the test suite.
The launcher scripts are in bin/.
After you've run mvn package, you should be able to start the agent and master:
$ bin/helios-master &
$ bin/helios-agent &
If you see any issues, make sure you have the prerequisites (Docker and Zookeeper) installed.
You can install the downloaded deb packages as mentioned above, or you can build them yourself.
Running mvn package generates deb packages that you can install on
your servers. For example:
$ mvn package
...
$ find . -name "*.deb"
./helios-services/target/helios-agent_0.0.27-SNAPSHOT_all.deb
./helios-services/target/helios-master_0.0.27-SNAPSHOT_all.deb
./helios-services/target/helios-services_0.0.27-SNAPSHOT_all.deb
./helios-tools/target/helios_0.0.27-SNAPSHOT_all.deb
The deb package in the helios-tools directory contains the Helios CLI tool.
The helios-services deb package is a shared dependency of both the agent
and the master. Install it and either the helios-agent or helios-master
deb package on each agent and master, respectively.
Here are a few other things you probably want to consider using alongside Helios:
- docker-gc Garbage collects dead containers and removes unused images.
- syslog-redirector Can be used by Helios agents to redirect the standard out/err of containers to syslog.
- helios-skydns Makes it so you can auto register services in SkyDNS. If you use leading underscores in your SRV record names, let us know, we have a patch for etcd which disables the "hidden" node feature which makes this use case break.
- skygc When using SkyDNS, especially if you're using the Helios Testing Framework, can leave garbage in the skydns tree within etcd. This will clean out dead stuff.
- docker-maven-plugin Simplifies the building of Docker containers if you're using Maven (and most likely Java).
To run findbugs on the helios codebase, do
mvn clean compile site. This will build helios and then run an analysis,
emitting reports in helios-*/target/site/findbugs.html.
To silence an irrelevant warning, add a filter match along with a justification
in findbugs-exclude.xml.
The sources for the Helios master and agent are under helios-services. The CLI source is under helios-tools. The Helios Java client is under helios-client.
The main meat of the Helios agent is in Supervisor.java, which revolves around the lifecycle of managing individual running Docker containers.
For the master, the HTTP response handlers are in src/main/java/com/spotify/helios/master/resources.
Interactions with ZooKeeper for the agent and master are mainly in ZookeeperAgentModel.java and ZooKeeperMasterModel.java, respectively.
The Helios services use Dropwizard which is a bundle of Jetty, Jersey, Jackson, Yammer Metrics, Guava, Logback and other Java libraries.
These are things we want, but haven't gotten to. If you feel inspired, we'd love to talk to you about these (in no particular order):
- Host groups
- ACLs - on jobs, hosts, and deployments
- Composite jobs -- be able to deploy related containers as a unit on a machine
- Run once jobs -- for batch jobs
- Resource specification and enforcement -- That is: restrict my container to X MB of RAM, X CPUs, and X MB disk and perhaps other things like IOPs, network bandwidth, etc.
- Dynamic scheduling of jobs -- either within Helios itself or as a layer on top
- Packaging/Config for other Linux distributions such as RedHat, CoreOS, etc.