Bowtie2 end to end executing process error
koushik20 opened this issue · comments
Hello,
I am running an nf-core/hic pipeline for breast samples (Total 6 samples including replicates) 500 Million reads each.
I am getting the following error
executor > local (26)
[1f/560b97] process > get_software_versions [100%] 1 of 1 ✔
[d3/5e1c5f] process > makeChromSize (genome.fa) [100%] 1 of 1 ✔
[93/6a6ae3] process > getRestrictionFragments (genome.fa [^... [100%] 1 of 1 ✔
[a9/b6cc6d] process > bowtie2_end_to_end (HiChIP_MCF10A-A_S... [ 9%] 22 of 234, failed...
[- ] process > trim_reads [ 0%] 0 of 5
[- ] process > bowtie2_on_trimmed_reads -
[- ] process > merge_mapping_steps -
[- ] process > combine_mapped_files -
[- ] process > get_valid_interaction -
[- ] process > remove_duplicates -
[- ] process > merge_sample -
[- ] process > build_contact_maps -
[- ] process > run_ice -
[- ] process > generate_cool -
[- ] process > multiqc -
[b7/f6bd1d] process > output_documentation (1) [100%] 1 of 1 ✔
Error executing process > 'bowtie2_end_to_end (HiChIP_MCF10A-A_S7_R2_001.15)'
Caused by:
Process exceeded running time limit (5h)
Command executed:
bowtie2 --rg-id BMG --rg SM:HiChIP_MCF10A-A_S7_R2_001.15 \
--very-sensitive --end-to-end --reorder \
-p 4 \
-x Bowtie2Index/genome \
--un HiChIP_MCF10A-A_S7_R2_001.15_unmap.fastq \
-U HiChIP_MCF10A-A_S7_R2_001.15.fastq | samtools view -F 4 -bS - > HiChIP_MCF10A-A_S7_R2_001.15.bam
Command exit status:
-
Command output:
(empty)
Work dir:
/home/koushik/hichip_fastq/work/58/23e74799c7ab12315e8a9e3a2dd28b
Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
The following is the script I ran
sudo nextflow run nf-core/hic -r 1.0.0 \
--reads '/home/koushik/hichip_fastq/HiChIP_MCF10A-{A,B}_S{7,8}_R{1,2}_001.fastq.gz' \
-profile docker \
-resume \
--splitFastq 10000000 \
--max_memory '100.GB' \
--max_cpus 60 \
--outdir "/media/HD/HiChIP_results_Sep_16_2021" \
--genome GRCh37 \
--bwt2_opts_end2end '--very-sensitive --end-to-end --reorder' \
--bwt2_opts_trimmed '--very-sensitive --end-to-end --reorder' \
--ligation_site 'GATCGATC' \
--restriction_site '^GATC' \
--min_cis_dist 1000 \
--min_mapq 20 \
--bin_size '1000,5000,20000,40000,150000,500000,1000000' \
--saveReference
What can I do to overcome this error?
Is it because of the time limit? Can I change the running time limit in base.config file?
Thank you!
Hi,
Could you please the run the latest version ? 1.3.0 ?
Thanks
Hi,
Could you please the run the latest version ? 1.3.0 ?
Thanks
Thanks for your reply
Ran with version 1.3.0 still got the same error.
executor > local (8)
[e9/ca4bf9] process > get_software_versions [100%] 1 of 1 ✔
[dc/5a7949] process > makeChromSize (genome.fa) [100%] 1 of 1 ✔
[c8/56dd3e] process > getRestrictionFragments (genome.fa ^GATC) [100%] 1 of 1 ✔
[e7/8d6769] process > bowtie2_end_to_end (HiChIP_MCF7_R2) [100%] 4 of 4, failed: 4 ✘
[- ] process > trim_reads -
[- ] process > bowtie2_on_trimmed_reads -
[- ] process > bowtie2_merge_mapping_steps -
[- ] process > combine_mates -
[- ] process > get_valid_interaction -
[- ] process > remove_duplicates -
[- ] process > merge_stats -
[- ] process > build_contact_maps -
[- ] process > run_ice -
[- ] process > convert_to_pairs -
[- ] process > cooler_raw -
[- ] process > cooler_balance -
[- ] process > cooler_zoomify -
[- ] process > dist_decay -
[- ] process > compartment_calling -
[- ] process > tads_hicexplorer -
[- ] process > tads_insulation -
[- ] process > multiqc -
[6c/7b05dd] process > output_documentation [100%] 1 of 1 ✔
Execution cancelled -- Finishing pending tasks before exit
- Ignore this warning: params.schema_ignore_params = "saveReference,splitFastq"
WARN: Found unexpected parameters:
* --saveReference: true
* --splitFastq: 10000000
Error executing process > 'bowtie2_end_to_end (HiChIP_MCF7_R1)'
Caused by:
Process exceeded running time limit (8h)
Command executed:
INDEX=`find -L ./ -name "*.rev.1.bt2" | sed 's/.rev.1.bt2//'`
bowtie2 --rg-id BMG --rg SM:HiChIP_MCF7-A_S1_R1_001 \
--very-sensitive --end-to-end --reorder \
-p 4 \
-x ${INDEX} \
--un HiChIP_MCF7-A_S1_R1_001_unmap.fastq \
-U HiChIP_MCF7-A_S1_R1_001.fastq.gz | samtools view -F 4 -bS - > HiChIP_MCF7-A_S1_R1_001.bam
Command exit status:
-
Command output:
(empty)
Work dir:
/home/koushik/hichip_fastq/work/8b/1382f2a81df213f543daf61859cb78
Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
With both the version the error occurs at "bowtie2_end_to_end" step and states caused by "Process exceeded running time limit". I tried to change the CPUs and memory provided but still the same error. Could you please suggest to me what I can do?
Hi I have encountered the same error Process exceeded running time limit, do you solve the problem finally? If so, can you share the solutions? Thanks!
This is a memory issue. You need to improve the memory for this task to be complete. Please follow the given link as it resolved my error.
https://github.com/nf-core/hic/issues/111#issuecomment-944159462