Roadrunner =============================== Roadrunner is a tool to run commands on hosts in a flow based manner. The user simply specifies a job flow in the form of a JSON file and submits it to Roadrunner. Roadrunner handles the synchronizing of jobs. How is the job flow organized? =============================== The job flow is divided into slots. All the jobs in one slot are run in parallel. For example, if slot 1 contains job A and job B, both of them will be run in parallel. You can set success constraints on a per job basis. For example, if job A runs a command on lets say 10 hosts, you can set the success constraint as 80%. That means, the job will be deemed successful if the command succeeds on atleast 80% (in this case 8) hosts. One can also set the degree of parallelism per job. The degree of parallelism means the number of hosts on which the command can be run in parallel. The jobs in the next slot are executed only when all the jobs in a previous slot execute successfully. Example use cases =============================== Consider a deployment scenario where we want to install pkg1 on hosts db[1-10] and pkg2 on api[1-6]. If both the previous jobs succeed, we will restart apache on fe[1-100]. In our scenario, we can organize the job flow as follows: slot1: [pkg1 on db[1-10], pkg2 on api[1-6]] slot2: [restart apache on fe[1-100] with parallelism set to 50%] The reason we are setting parallelism on the FE box to 50% is because we do not want to restart the webserver on all hosts at once. That will make the site unavailable for the time apache is restarting. Will Roadrunner scale? =============================== Roadrunner uses Gearman as the job distribution mechanism. Gearman is a proven job distribution engine used by many big companies. Roadrunner also uses Fabric to run ssh commands on the hosts. Dependencies =============================== 1. Gearman [http://gearman.org/] 2. Python Wrapper for Gearman [http://pypi.python.org/pypi/gearman/2.0.2] 3. Fabric [http://pypi.python.org/pypi/Fabric]