spindlelabs / otto

You got your platform in my application management! You got your application in my platform management!

Otto

For an introduction to Otto, read our blog post: Otto: unifying deployment, application management, and platform management

Otto is tested with Puppet 2.7.19 on Ubuntu 12.04 LTS.

Getting started: deploying helloworld

helloworld is a simple Java application that reads its configuration, logs several messages, and then exits after 30 seconds (simulating an application crash.) We'll demonstrate deploying helloworld from a build server using Otto. (Otto is agnostic to how your artifact lands on the machine; you could use BitTorrent, for example.)

To deploy helloworld, Otto will:

1. Create an unprivileged user, helloworld (examples/helloworld/modules/helloworld/manifests/init.pp)
2. Install the Java 7 runtime package, openjdk-7-jre (examples/helloworld/modules/java/manifests/init.pp)
3. Create the /opt/otto hierarchy (modules/otto/manifests/init.pp)
4. Install daemontools, a public domain collection of tools for managing services
5. Ensure that svscan, the daemontools service scanner, is running
6. Download the helloworld build artifact from the (simulated, Jenkins-like) build server into /opt/otto/build/helloworld/helloworld-master-checkin@1 (examples/helloworld/modules/otto_java_app_from_jenkins/manifests/init.pp)
7. Deploy the helloworld configuration files into /opt/otto/conf/helloworld (examples/helloworld/files/conf)
8. Run the application using /opt/otto/run/helloworld (examples/helloworld/templates/run.erb)
9. Ensure that the service is started at boot and automatically restarted if it fails

To deploy helloworld on Ubuntu 12.04 LTS, run:

sudo apt-get install git puppet
git clone https://github.com/spindlelabs/otto.git
cd otto

# This command will make changes to your system state as described above; consider using a virtual machine
sudo puppet apply --modulepath modules:examples/helloworld/modules --debug examples/helloworld/manifests/site.pp

helloworld will be installed into /opt/otto and started automatically.

Try a few experiments (as root) to understand Otto:

• Run svstat /etc/service/helloworld to show the application's PID and uptime
• Run pstree -paul to show the process hierarchy; observe that java is running as the unprivileged helloworld user
• Run svc -t /etc/service/helloworld to send SIGTERM to the application; it will automatically restart
• Restart the machine; helloworld will automatically start
• View the application logs in /opt/otto/data/helloworld/log. Observe the logged configuration values and environment variables.
• Examine /opt/otto/service/helloworld/run and /opt/otto/run/helloworld to understand how Otto invokes helloworld. Note that the logging configuration in /opt/otto/conf/helloworld/logback.xml uses an environment variable supplied by Otto to locate the application data directory without hard-coding paths.
• Note the permissions for /opt/otto/build/helloworld/helloworld-master-checkin@1, /opt/otto/conf/helloworld, and /opt/otto/data/helloworld: an application can modify its data directory but cannot modify its build artifact or its configuration

Next, try changing the application configuration. After making each change below, rerun the puppet apply command above; Otto will restart the application with its new configuration.

• Deploy a different build (1, 2 or 3) by changing jenkinsBuildID in examples/helloworld/manifests/site.pp. Otto will download the new build, stop the existing build, and then start the new build. Try rolling back to a previous build; Otto will avoid redownloading a build that it has already retrieved. Because the build is configured using the Puppet DSL, you can use features like generate(), extlookup(), and external node classifiers to dynamically determine the build running on each node. You could use these features to implement continuous deployment, scheduled deployment ("deploy this build when traffic drops below k QPS"), or automatic rollback.
• Change the log level in examples/helloworld/modules/helloworld/files/conf/logback.xml or the configuration value value1 in examples/helloworld/modules/files/conf/application.conf
• Change the configuration value value2 in examples/helloworld/manifests/site.pp
• Change appRunService to false in examples/helloworld/manifests/site.pp to stop the application; ensure that the change persists after restarting the machine. Revert the change to continue experimenting.
• Change the configuration value value3 in examples/helloworld/modules/templates/run.erb by changing the amount of swap space on the machine (dd if=/dev/zero of=/swapfile bs=1024 count=65536; mkswap /swapfile; swapon /swapfile). The amount of swap space on the machine is provided by Facter.
• As root, modify /opt/otto/conf/helloworld/logback.xml. Otto will revert the change to prevent configuration drift.
• Simulate a build server failure by changing jenkinsBuildID in examples/helloworld/manifests/site.pp and appBuildArtifactUrl in examples/helloworld/modules/otto_java_app_from_jenkins/manifests/init.pp. Otto will abort the upgrade, log an error, and continue running the previous build. Otto will retry the upgrade the next time Puppet runs.

Next steps

Now that you've seen a working example, try deploying your own application with Otto. If you're not familiar with Puppet, read the Puppet introduction. Copy modules/otto into your Puppet modules directory, and modify the examples to fit your environment. Be sure to follow the application notes in modules/otto/manifests/{app,init}.pp.

Known issues

• Otto does not provide a mechanism for removing deployed applications, but it's easy to do manually: remove the application definition from the node manifest, stop the application and its supervise process, and then remove its state from /opt/otto. You could also simply set appRunService to false.