NicholasRasi / SlurmCluster

:flashlight: Ansible playbooks to setup a multinode cluster with Slurm + MPI benchmarks + HPC containers

Slurm Cluster

Ansible playbooks to setup a multinode cluster with Slurm.

3 playbooks for the cluster initialization:

• controllers (slurmctdl)
• workers (slurmd)
• NAS (for network storage)

+ 1 extra playbook for benchmarks initialization (http://mvapich.cse.ohio-state.edu/benchmarks/, mpiBench)

+ 1 extra playbook for HPC containers initialization (Singularity, CharlieCloud, Sarus, + DockerCE (for convenience))

---

The playbooks (controllers, workers, nas) will setup:

• hostname
• ssh connection
• ntp synch
• nfs server and client
• Munge and key
• Slurm (controller and worker)
• helpers

It can be used on Azure with the CentOS-based HPC images (Rogue Wave Software / OpenLogic) to quickly deploy a cluster with MPI and pre-installed drivers for Infiniband (e.g. on HB60).

Setup Local

• Install Ansible and deps:

virtualenv env
source env/bin/activate
pip install -r requirements.txt

• Edit inventory as needed
• Edit conf/hosts as needed the init controllers, nas and workers

Setup Single Components

Setup Controllers

• Edit conf/slurm.conf as needed (info)
• Edit conf/cgroup.conf as needed (info)

ansible-playbook -i inventory init_controllers.yml

Setup NAS

A folder called /home/user/workspace is created and exported. It is automounted on workers

If the NAS and the controller are on the same node, then run

ansible-playbook -i inventory init_nas.yml

otherwise run

ansible-playbook -i inventory init_nas.yml --extra-vars "external_nas=Y"

Setup Workers

ansible-playbook -i inventory init_workers.yml

Setup Container

The playbook will install Docker, Singularity, Charliecloud and Sarus

• Edit init_containers.yml vars to set the containers release versions

ansible-playbook -i inventory init_containers.yml

Setup Benchmarks

• Edit init_containers.yml vars to set the containers release versions

ansible-playbook -i inventory init_benchmarks.yml

Setup All

Execute all the playbooks with:

ansible-playbook -i inventory init_all.yml