Plan B

A service for deploying containerized jobs locally or on remote clusters.

Setup

Create database

cat db.sql | sqlite3 db.sqlite3

Copy config.json.default to config.json and edit accordingly.
Make sure SSH public key is installed on remote systems.
Configure IRODS for remote user on remote systems.

Running

For development:

npm install nodemon -g
npm run dev

For production:

sudo npm install pm2@latest -g
pm2 start --name plan-b server.js
sudo pm2 startup systemd

System Configuration

Add an entry to systems.json:

hostname        Hostname of the remote (or local) system.
type            Type of system: "server", "hpc", or "hadoop".
username        User on remote system.
stagingPath     Data staging directory on remote system.
ssh             (optional) Custom ssh command and parameters.  
                For example to ssh proxy use "ssh -J myuser@hpc.arizona.edu login.ocelote.hpc.arizona.edu".
env             (optional) Environment variables to set on remote system. 
                For example:
                  {
                    "IRODS_ENVIRONMENT_FILE": "/home/myuser/irods_environment.json",
                    "PATH": "/opt/irods/4.2.2/usr/bin:/home/myuser/bin"
                  }
targetHdfsPath  (type="hadoop") Data staging directory in HDFS.
hadoopJavaOpts  (type="hadoop", optional) Java command-line options for setting variables or memory limits.

App Configuration

Add a TAPIS app description file to the apps/ directory. Set the executionSystem and deploymentPath fields to correspond to the target system specified in systems.json.

See the TAPIS documentation for more info on the app description file:
https://tacc-cloud.readthedocs.io/projects/agave/en/latest/agave/guides/apps/introduction.html

REST API

The API closely resembles the TAPIS Jobs API:
https://tacc-cloud.readthedocs.io/projects/agave/en/latest/agave/guides/jobs/introduction.html

GET /apps/[id]              Fetch app by ID
GET /jobs                   Fetch all jobs
GET /jobs/[id]              Fetch job by ID
GET /jobs/[id]/history      Fetch job history by ID
POST /jobs                  Submit job

All endpoints require a TAPIS authentication bearer token.

hurwitzlab / plan-b