raw align error

[Jianz9293]

Describe the bug
Hello, the following problem occurred when running NextDenovo(v2.4.0)

Error message
Paste the complete log message, include the main task log and failed subtask log.
The main task log is usually located in your working directory and is named pidXXX.log.info and the main task log will tell you the failed subtask log in the last few lines, such as:

[INFO] 2021-07-14 04:09:19,490 Submit jobID:[21476] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align57/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 04:09:50,986 Submit jobID:[21509] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align58/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 04:10:01,419 Submit jobID:[21526] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align59/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 05:15:45,364 52 raw_align jobs failed, and rerun for the 3 time
[INFO] 2021-07-14 05:15:45,370 Total jobs: 52
[INFO] 2021-07-14 05:15:45,372 Submit jobID:[25394] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 05:15:45,875 Submit jobID:[25399] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align01/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 05:15:46,378 Submit jobID:[25405] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align02/nextDenovo.sh] in the local_cycle.
[INFO] 2021-07-14 05:15:46,881 Submit jobID:[25410] jobCmd:[/home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align03/nextDenovo.sh] in the local_cycle.

Genome characteristics
`genome size 200M

Input data
Types Count (#) Bases (bp) Depth (X)
Raw 2842058 18982058099 94.91
Clean 2842058 18982058099 94.91
Count (#) Length (bp)
N50 955354 6868

Config file
[General]
job_type = local # local, slurm, sge, pbs, lsf
job_prefix = nextDenovo
task = all # all, correct, assemble
rewrite = yes # yes/no
deltmp = yes
parallel_jobs = 20 # number of tasks used to run in parallel
input_type = raw # raw, corrected
read_type = ont # clr, ont, hifi
input_fofn = input.fofn
workdir = 01_rundir

[correct_option]
read_cutoff = 1k
genome_size = 200m # estimated genome size
sort_options = -m 20g -t 15
seed_depth = 25
minimap2_options_raw = -x ava-ont -t 8
pa_correction = 3 # number of corrected tasks used to run in parallel, each corrected task requires ~TOTAL_INPUT_BASES/4 bytes of memory usage.
correction_options = -p 10

[assemble_option]
random_round = 20
minimap2_options_cns = -t 8
nextgraph_options = -a 1

Operating system
no found, linux

GCC
gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC)

Python
Python 2.7.18

NextDenovo
v2.4.0

To Reproduce (Optional)
Steps to reproduce the behavior. Providing a minimal test dataset on which we can reproduce the behavior will generally lead to quicker turnaround time!

Additional context (Optional)
Add any other context about the problem here.

[Hu Jiang]

Could you paste the content of /home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh.e to here?

[Jianz9293]

Thank you for your reply,After I read through other similar problems, I thought it was due to lack of memory, so I ran the same program again on a disk with enough memory, but the same problem still occurred. Now I write out the information of the wrong file
here is /home/lanlina/opt/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh.e
hostname

• hostname
  cd /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00
• cd /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00
  time /data1/software/biosoft/NextDenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 8 /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.seed.003.2bit /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.part.014.2bit -o input.seed.003.2bit.0.ovl;
• /data1/software/biosoft/NextDenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 8 /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.seed.003.2bit /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.part.014.2bit -o input.seed.003.2bit.0.ovl
  /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh: line 5: 363 Killed /data1/software/biosoft/NextDenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 8 /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.seed.003.2bit /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.part.014.2bit -o input.seed.003.2bit.0.ovl

And /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh: line 5 is here
#!/bin/bash
set -xve
hostname
cd /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00
time /data1/software/biosoft/NextDenovo/bin/minimap2-nd --step 1 --dual=yes -x ava-ont -t 8 /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.seed.003.2bit /data1/software/biosoft/NEXT/01_rundir/01.raw_align/input.part.014.2bit -o input.seed.003.2bit.0.ovl;
touch /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh.done

[Hu Jiang]

The system killed subtaskes (see /data1/software/biosoft/NEXT/01_rundir/01.raw_align/03.raw_align.sh.work/raw_align00/nextDenovo.sh: line 5: 363 Killed), which usually caused by lack of memory, see here to optimize parallel computing parameters.

[Jianz9293]

OK, I will adjust the parameters and try again. Thank you very much for your reply

[Hu Jiang]

enen... I will close this issue but feel free to re-open if you still have this issue.