No seq_split in new download
Amarea1 opened this issue · comments
Question or Expected behavior
I have attempted to update my copy of NextPolish from 1.3 to the newest edition. On downloading the file from github, the test data has run fine but when I run my own data I get an error saying that there is no seq_split file. Re-downloading does not fix the issue. It also appears to be generating a backup file containing only the "00.score_chain" directory as soon as it gives me the Traceback.
Operating system
Ubuntu 18.04 server
GCC
GCC v. 7.5.0
Python
3.8.12
NextPolish
1.4.0
/home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh.e
hostname
- hostname
cd /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1 - cd /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1
time /home/stxsk44/NextPolish/bin/seq_split -d /home/stxsk44/NextPolish -m 6921750.0 -n 6 -t 5 -i 1 -s 41530500 -p inpu$+ time /home/stxsk44/NextPolish/bin/seq_split -d /home/stxsk44/NextPolish -m 6921750.0 -n 6 -t 5 -i 1 -s 41530500 -p in$time: cannot run /home/stxsk44/NextPolish/bin/seq_split: No such file or directory
Command exited with non-zero status 127
0.00user 0.00system 0:00.00elapsed ?%CPU (0avgtext+0avgdata 1120maxresident)k
0inputs+0outputs (0major+27minor)pagefaults 0swaps
I'm not the most proficient person with Linux or software like this so hopefully it's an easy issue to fix. Thanks very much!
Hi, you need to compile after downloading, then you will find seq_split in the bin directory.
pip install paralleltask
tar -vxzf NextPolish.tgz && cd NextPolish && make
Hi again
I tried re-installing from scratch and made sure to use both of the commands you said but now it's giving a different issue
Output:
[20784 INFO] 2022-04-27 12:21:13 NextPolish start...
[20784 INFO] 2022-04-27 12:21:13 version:v1.4.0 logfile:pid20784.log.info
[20784 WARNING] 2022-04-27 12:21:13 Delete task: 5 due to missing lgs_fofn.
[20784 WARNING] 2022-04-27 12:21:13 Delete task: 5 due to missing lgs_fofn.
[20784 WARNING] 2022-04-27 12:21:13 Delete task: 6 due to missing hifi_fofn.
[20784 WARNING] 2022-04-27 12:21:13 Delete task: 6 due to missing hifi_fofn.
[20784 INFO] 2022-04-27 12:21:13 scheduled tasks:
[1, 2, 1, 2]
[20784 INFO] 2022-04-27 12:21:13 options:
[20784 INFO] 2022-04-27 12:21:13
rerun: 3
rewrite: 0
kill: None
cleantmp: 0
use_drmaa: 0
submit: None
job_type: local
sgs_unpaired: 0
sgs_rm_nread: 1
lgs_read_type:
parallel_jobs: 6
align_threads: 5
check_alive: None
task: [1, 2, 1, 2]
job_id_regex: None
sgs_max_depth: 100
lgs_max_depth: 100
multithread_jobs: 5
genome_size: 415305
lgs_max_read_len: 0
hifi_max_depth: 100
lgs_block_size: 500M
lgs_min_read_len: 1k
hifi_max_read_len: 0
polish_options: -p 5
hifi_block_size: 500M
hifi_min_read_len: 1k
job_prefix: nextPolish
sgs_block_size: 6921750.0
sgs_use_duplicate_reads: 0
lgs_minimap2_options: -x map-ont
hifi_minimap2_options: -x map-pb
workdir: /home/stxsk44/NextPolish
sgs_align_options: bwa mem -p -t 5
sgs_fofn: /home/stxsk44/NextPolish/sgs.fofn
snp_phase: /home/stxsk44/NextPolish/%02d.snp_phase
snp_valid: /home/stxsk44/NextPolish/%02d.snp_valid
genome: /home/stxsk44/NextPolish/L14W.contigs.fasta
lgs_polish: /home/stxsk44/NextPolish/%02d.lgs_polish
kmer_count: /home/stxsk44/NextPolish/%02d.kmer_count
hifi_polish: /home/stxsk44/NextPolish/%02d.hifi_polish
score_chain: /home/stxsk44/NextPolish/%02d.score_chain
[20784 WARNING] 2022-04-27 12:21:13 mv /home/stxsk44/NextPolish to /home/stxsk44/NextPolish.backup1
[20784 INFO] 2022-04-27 12:21:13 step 0 and task 1 start:
[20784 INFO] 2022-04-27 12:21:18 Total jobs: 3
[20784 INFO] 2022-04-27 12:21:18 Submitted jobID:[20793] jobCmd:[/home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh] in the local_cycle.
[20793 CRITICAL] 2022-04-27 12:21:18 �[35mCommand '/bin/sh /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh > /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh.o 2> /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh.e' returned non-zero exit status 127, error info: .�[0m
Traceback (most recent call last):
File "./nextPolish", line 515, in
main(args)
File "./nextPolish", line 369, in main
task.run.start()
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 344, in start
self._start()
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 368, in _start
self.submit(job)
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 252, in submit
_, stdout, _ = self.run(job.cmd)
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 288, in run
log.critical("Command '%s' returned non-zero exit status %d, error info: %s." % (cmd, p.returncode, stderr))
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1493, in critical
self._log(CRITICAL, msg, args, **kwargs)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1589, in _log
self.handle(record)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1599, in handle
self.callHandlers(record)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1661, in callHandlers
hdlr.handle(record)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 954, in handle
self.emit(record)
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/kit.py", line 42, in emit
raise Exception(record.msg)
Exception: Command '/bin/sh /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh > /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh.o 2> /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh.e' returned non-zero exit status 127, error info: .
[20784 INFO] 2022-04-27 12:21:19 Submitted jobID:[20803] jobCmd:[/home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh] in the local_cycle.
[20803 CRITICAL] 2022-04-27 12:21:19 �[35mCommand '/bin/sh /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh > /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh.o 2> /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh.e' returned non-zero exit status 127, error info: .�[0m
Traceback (most recent call last):
File "./nextPolish", line 515, in
main(args)
File "./nextPolish", line 369, in main
task.run.start()
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 344, in start
self._start()
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 368, in _start
self.submit(job)
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 252, in submit
_, stdout, _ = self.run(job.cmd)
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 288, in run
log.critical("Command '%s' returned non-zero exit status %d, error info: %s." % (cmd, p.returncode, stderr))
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1493, in critical
self._log(CRITICAL, msg, args, **kwargs)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1589, in _log
self.handle(record)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1599, in handle
self.callHandlers(record)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1661, in callHandlers
hdlr.handle(record)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 954, in handle
self.emit(record)
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/kit.py", line 42, in emit
raise Exception(record.msg)
Exception: Command '/bin/sh /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh > /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh.o 2> /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh.e' returned non-zero exit status 127, error info: .
[20784 INFO] 2022-04-27 12:21:19 Submitted jobID:[20809] jobCmd:[/home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh] in the local_cycle.
[20809 CRITICAL] 2022-04-27 12:21:19 �[35mCommand '/bin/sh /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh > /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh.o 2> /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh.e' returned non-zero exit status 127, error info: .�[0m
Traceback (most recent call last):
File "./nextPolish", line 515, in
main(args)
File "./nextPolish", line 369, in main
task.run.start()
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 344, in start
self._start()
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 368, in _start
self.submit(job)
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 252, in submit
_, stdout, _ = self.run(job.cmd)
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/task_control.py", line 288, in run
log.critical("Command '%s' returned non-zero exit status %d, error info: %s." % (cmd, p.returncode, stderr))
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1493, in critical
self._log(CRITICAL, msg, args, **kwargs)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1589, in _log
self.handle(record)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1599, in handle
self.callHandlers(record)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 1661, in callHandlers
hdlr.handle(record)
File "/home/stxsk44/miniconda3/lib/python3.8/logging/init.py", line 954, in handle
self.emit(record)
File "/home/stxsk44/miniconda3/lib/python3.8/site-packages/paralleltask/kit.py", line 42, in emit
raise Exception(record.msg)
Exception: Command '/bin/sh /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh > /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh.o 2> /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh.e' returned non-zero exit status 127, error info: .
[20784 ERROR] 2022-04-27 12:21:26 �[35mdb_split failed: please check the following logs:�[0m
[20784 ERROR] 2022-04-27 12:21:26 �[35m/home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split1/nextPolish.sh.e�[0m
[20784 ERROR] 2022-04-27 12:21:26 �[35m/home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split2/nextPolish.sh.e�[0m
[20784 ERROR] 2022-04-27 12:21:26 �[35m/home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3/nextPolish.sh.e�[0m
Error file:
hostname
- hostname
cd /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3 - cd /home/stxsk44/NextPolish/00.score_chain/01.db_split.sh.work/db_split3
time /home/stxsk44/NextPolish/bin/samtools faidx /home/stxsk44/NextPolish/00.score_chain/input.genome.fasta - time /home/stxsk44/NextPolish/bin/samtools faidx /home/stxsk44/NextPolish/00.score_chain/input.genome.fasta
time: cannot run /home/stxsk44/NextPolish/bin/samtools: No such file or directory
Command exited with non-zero status 127
0.00user 0.00system 0:00.00elapsed ?%CPU (0avgtext+0avgdata 1056maxresident)k
0inputs+0outputs (0major+27minor)pagefaults 0swaps
I've checked and samtools is listed in the bin folder of the backup folder that was created. Any thoughts? Thanks very much.
First, if you are sure the executable files such as samtools in the bin directory can be run normally, you can delete the working directory /home/stxsk44/NextPolish and try running NextPolish again. If you still get an error, the easiest way is to follow here to use your own alignment pipeline.
Just to update, it looks like my issue may have been the same as this one:
It was resolved by adding "rewrite = no" and a specific work directory to the run.cfg file. It seems to have run fine with those two lines added.
OK