EXPERIMENTAL Note that QHub OnPrem is currently experimental
QHub OnPrem is an opinionated open source deployment of jupyterhub based on an HPC jobscheduler. QHub OnPrem is a "distribution" of these packages much like Debian and Ubuntu are distributions of Linux. The high level goal of this distribution is to form a cohesive set of tools that enable:
- environment management via conda and conda-store
- monitoring of compute infrastructure and services
- scalable and efficient compute via jupyterlab and dask
- deployment of jupyterhub on prem without requiring deep devops knowledge of the Slurm/HPC and jupyter ecosystem
Install ansible dependencies
ansible-galaxy collection install -r requirements.yamlThere are tests for deploying QHub OnPrem on a virtual machine provisioner and in the cloud.
Vagrant is a tool responsible for creating and provisioning vms. It
has convenient integration with ansible which allows for easy
effective control over configuration. Currently the Vagrantfile only
has support for libvirt.
cd tests/ubuntu1804
# cd tests/ubuntu2004
vagrant up --provider=<provider-name>Notebook for testing functionality
tests/assets/notebook/test-dask-gateway.ipynb
Current testing environment spins up four nodes:
- all nodes :: node_exporter for node metrics
- master node :: slurm scheduler, munge, mysql, jupyterhub, grafana, prometheus
- worker node :: slurm daemon, munge
Jupyterhub is accessible via <master node ip>:8000
Grafana is accessible via <master node ip>:3000
Contributions are welcome!