sh Illegal option -o pipefail

[bfr42]

Describe the bug
Error with "-o pipefail" option wile running the test data via "./nextDenovo test_data/run.cfg".

Error message
Main task log

[70899 INFO] 2023-01-20 15:21:35 NextDenovo start...
[70899 INFO] 2023-01-20 15:21:35 version:v2.5.0 logfile:pid70899.log.info
[70899 WARNING] 2023-01-20 15:21:35 Re-write workdir
[70899 INFO] 2023-01-20 15:21:35 mkdir: /mnt/nextDenovo_assembly/cent/NextDenovo/test_data/./01_rundir
[70899 INFO] 2023-01-20 15:21:35 mkdir: /mnt/nextDenovo_assembly/cent/NextDenovo/test_data/./01_rundir/01.raw_align
[70899 INFO] 2023-01-20 15:21:35 mkdir: /mnt/nextDenovo_assembly/cent/NextDenovo/test_data/./01_rundir/02.cns_align
[70899 INFO] 2023-01-20 15:21:35 mkdir: /mnt/nextDenovo_assembly/cent/NextDenovo/test_data/./01_rundir/03.ctg_graph
[70899 INFO] 2023-01-20 15:21:40 Total jobs: 1
[70899 INFO] 2023-01-20 15:21:40 Submitted jobID:[70919] jobCmd:[/mnt/nextDenovo_assembly/cent/NextDenovo/test_data/01_rundir/01.raw_align/01.db_stat.sh.work/db_stat1/nextDenovo.sh] in the local_cycle.
[70919 CRITICAL] 2023-01-20 15:21:40 Command '/bin/sh /mnt/nextDenovo_assembly/cent/NextDenovo/test_data/01_rundir/01.raw_align/01.db_stat.sh.work/db_stat1/nextDenovo.sh > /mnt/nextDenovo_assembly/cent/NextDenovo/test_data/01_rundir/01.raw_align/01.db_stat.sh.work/db_stat1/nextDenovo.sh.o 2> /mnt/nextDenovo_assembly/cent/NextDenovo/test_data/01_rundir/01.raw_align/01.db_stat.sh.work/db_stat1/nextDenovo.sh.e' returned non-zero exit status 2, error info: .
[70899 ERROR] 2023-01-20 15:21:45 db_stat failed: please check the following logs:
[70899 ERROR] 2023-01-20 15:21:45 /mnt/nextDenovo_assembly/cent/NextDenovo/test_data/01_rundir/01.raw_align/01.db_stat.sh.work/db_stat1/nextDenovo.sh.e

Failed subtask log:

/mnt/nextDenovo_assembly/cent/NextDenovo/test_data/01_rundir/01.raw_align/01.db_stat.sh.work/db_stat1/nextDenovo.sh: 2: set: Illegal option -o pipefail

Genome characteristics
Your test data

Input data
Your test data

Config file
[General]
job_type = local
job_prefix = nextDenovo
task = all # 'all', 'correct', 'assemble'
rewrite = yes # yes/no
deltmp = yes
rerun = 3
parallel_jobs = 2
input_type = raw
read_type = clr
input_fofn = ./input.fofn
workdir = ./01_rundir

[correct_option]
read_cutoff = 1k
genome_size = 308161
pa_correction = 2
sort_options = -m 1g -t 2
minimap2_options_raw = -t 8
correction_options = -p 15

[assemble_option]
minimap2_options_cns = -t 8
nextgraph_options = -a 1

Operating system
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.01 LTS
Release: 22.04
Codename: jammy

GCC
gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu1~22.04)

Python
Python 3.9.15

NextDenovo
nextDenovo v2.5.0

Additional context (Optional)
The sh scripts your program creates start with "#!/bin/sh". If changing it to "#!/bin/bash" or executing them with "bash /mnt/nextDenovo_assembly/cent/NextDenovo/test_data/01_rundir/01.raw_align/01.db_stat.sh.work/db_stat1/nextDenovo.sh" works. However, there are too many steps/ sh scripts to do it all manually.

Thanks

[Hu Jiang]

Hi，I have fixed this bug, try pip install --upgrade paralleltask to update paralleltask

[Daniel Fabian]

I have the same issue. I think it might be caused by the "set -xveo pipefail" in 01.raw_align/01.db_stat.sh.work/db_stat1/nextDenovo_input_reads_filtered.sh - this is what the log suggests.

When I comment "set -xveo pipefail", and then run the script again, this is printed in the terminal "d8181d3c042d" and a new file named nextDenovo_input_reads_filtered.sh.done is created.

This is with paralleltask 0.2.0, nextDenovo 2.5.0 and Python 3.10.6 on Ubuntu 22.04.1 LTS.

[Hu Jiang]

Yes, and try to upgrade paralleltask.