This module is me looking at ways to talk to the Grid Engine Scheduler with Python. It's basically a sketch of how things might work at this point and you probably shouldn't touch it.
The idea is that some future service will present JupyterHub/Lab as a front end to HPC at UCL and some future well designed version of this thing should allow users to do job control through Python.
Hopefully there'd also be tools for inspecting the status of the user's jobs and retrieving the output etc.
At the moment, there is some very basic job script creation code:
>>> import gepy
>>> test_job = gepy.job(name="test")
>>> test_job.workload.append(gepy.serial_command())
>>> test_job.workload.append(gepy.user_script('echo HELLO'))
>>> test_job.workload.append(gepy.parallel_command(binary='lmp_any', args=['-in', 'in.lammps']))
>>> test_job.modules.append('lammps')
>>> print(test_job.get_job_script())
#!/bin/bash -l
# Job script generated by GEPY (https://github.com/owainkenwayucl/gepy)
#$ -N test
#$ -l mem=1024
#$ -l h_rt=1:00:00
#$ -cwd
module load lammps
# GEPY serial command.
/usr/bin/hostname
# User supplied shell code.
echo HELLO
# End user supplied shell code.
# GEPY Parallel command.
gerun lmp_any -in in.lammps
>>> There's also now a submodule called executor which is starting to provide Grid Engine functionality, specifically, it can get a list of jobs from qstat, and submit the job scripts created by the script creation code.
In the summer of 2021, Bowen Zheng, a CDT MSc student completed an MSc project, supervised by me, exploring this idea. You can see his code and efforts here: https://github.com/SC2020-zbw/MSc_project