Requested parallel environment does not exist

[Filomeno Sanchez]

System information

• PyJob version: 0.4
• Python version: 3.7

Problem description

Running task on a SGE HPC without any mpi environment set-up prevents job submission (this is common on small HPC with a single node with a big thread count). Can be solved by checking the list of available environments before creating submission script.

Example

Python call:

from pyjob import TaskFactory, Script

my_script=Script(directory="/data2/filo/test", prefix="mytest", stem='', suffix='.sh')
my_script.append("echo $(date)")

with TaskFactory("sge", my_script, max_array_size=1, shell="/bin/bash", cwd="/data2/filo/test") as task:
        task.run()

SGE submission script:

#!/bin/bash
#$ -V
#$ -w e
#$ -j yes
#$ -N mytest
#$ -S /bin/bash
#$ -wd /data2/filo/test
#$ -pe mpi 1
#$ -o /data2/filo/test/mytest.log

echo $(date)

Traceback

Unable to run job: job rejected: the requested parallel environment "mpi" does not exist
Exiting.

Task never gets submitted.