rule rseqc_gtf2bed failing with test dataset
ScientistSMC opened this issue · comments
Hi,
Thanks for all your work creating snakemake and putting together this pipeline!
I am running into an issue when trying to run the pipeline with the test data, particularly the rsec_gtf2bed rule. I am using snakemake version 5.7.0 (on a cluster). The error I get is pretty complicated and I am having trouble interpreting it. I think the first few error messages about the JSON Schema are unrelated. I have attached the output below of the command: snakemake --use-conda rseqc_gtf2bed
[user]$ snakemake --use-conda rseqc_gtf2bed
No validator found for JSON Schema version identifier 'http://json-schema.org/draft-06/schema#'
Defaulting to validator for JSON Schema version 'http://json-schema.org/draft-04/schema#'
Note that schema file may not be validated correctly.
No validator found for JSON Schema version identifier 'http://json-schema.org/draft-06/schema#'
Defaulting to validator for JSON Schema version 'http://json-schema.org/draft-04/schema#'
Note that schema file may not be validated correctly.
No validator found for JSON Schema version identifier 'http://json-schema.org/draft-06/schema#'
Defaulting to validator for JSON Schema version 'http://json-schema.org/draft-04/schema#'
Note that schema file may not be validated correctly.
Building DAG of jobs...
Creating conda environment envs/gffutils.yaml...
Downloading and installing remote packages.
Environment for envs/gffutils.yaml created (location: .snakemake/conda/a2417d57)
Using shell: /usr/bin/bash
Provided cores: 1
Rules claiming more threads will be scaled down.
Job counts:
count jobs
1 rseqc_gtf2bed
1
[Tue Oct 15 14:25:51 2019]
rule rseqc_gtf2bed:
input: .test/data/ref/annotation.chr21.gtf
output: qc/rseqc/annotation.bed, qc/rseqc/annotation.db
log: logs/rseqc_gtf2bed.log
jobid: 0
Activating conda environment: /data/project/rna-seq-star-deseq2-public-pipeline/.snakemake/conda/a2417d57
Traceback (most recent call last):
File "/data/project/rna-seq-star-deseq2-public-pipeline/.snakemake/scripts/tmp1qu6f74g.gtf2bed.py", line 3, in
import sys; sys.path.extend(["/share/apps/rc/software/snakemake/5.7.0-foss-2018b-Python-3.6.6/lib/python3.6/site-packages", "/data/project/rna-seq-star-deseq2-public-pipeline/scripts"]); import pickle; snakemake = pickle.loads(b'\x80\x03csnakemake.script\nSnakemake\nq\x00)\x81q\x01}q\x02(X\x05\x00\x00\x00inputq\x03csnakemake.io\nInputFiles\nq\x04)\x81q\x05X#\x00\x00\x00.test/data/ref/annotation.chr21.gtfq\x06a}q\x07X\x06\x00\x00\x00_namesq\x08}q\tsbX\x06\x00\x00\x00outputq\ncsnakemake.io\nOutputFiles\nq\x0b)\x81q\x0c(X\x17\x00\x00\x00qc/rseqc/annotation.bedq\rX\x16\x00\x00\x00qc/rseqc/annotation.dbq\x0ee}q\x0f(h\x08}q\x10(X\x03\x00\x00\x00bedq\x11K\x00N\x86q\x12X\x02\x00\x00\x00dbq\x13K\x01N\x86q\x14uh\x11h\rh\x13h\x0eubX\x06\x00\x00\x00paramsq\x15csnakemake.io\nParams\nq\x16)\x81q\x17}q\x18h\x08}q\x19sbX\t\x00\x00\x00wildcardsq\x1acsnakemake.io\nWildcards\nq\x1b)\x81q\x1c}q\x1dh\x08}q\x1esbX\x07\x00\x00\x00threadsq\x1fK\x01X\t\x00\x00\x00resourcesq csnakemake.io\nResources\nq!)\x81q"(K\x01K\x01e}q#(h\x08}q$(X\x06\x00\x00\x00_coresq%K\x00N\x86q&X\x06\x00\x00\x00_nodesq'K\x01N\x86q(uh%K\x01h'K\x01ubX\x03\x00\x00\x00logq)csnakemake.io\nLog\nq*)\x81q+X\x16\x00\x00\x00logs/rseqc_gtf2bed.logq,a}q-h\x08}q.sbX\x06\x00\x00\x00configq/}q0(X\x07\x00\x00\x00samplesq1X\x0b\x00\x00\x00samples.tsvq2X\x05\x00\x00\x00unitsq3X\t\x00\x00\x00units.tsvq4X\x08\x00\x00\x00trimmingq5}q6(X\x04\x00\x00\x00skipq7\x89X\x07\x00\x00\x00adapterq8X\x16\x00\x00\x00ACGGATCGATCGATCGATCGATq9uX\x03\x00\x00\x00refq:}q;(X\x05\x00\x00\x00indexq<X\x14\x00\x00\x00.test/data/ref/indexq=X\n\x00\x00\x00annotationq>X#\x00\x00\x00.test/data/ref/annotation.chr21.gtfq?uX\x03\x00\x00\x00pcaq@}qAX\x06\x00\x00\x00labelsqB]qCX\t\x00\x00\x00conditionqDasX\x07\x00\x00\x00diffexpqE}qFX\t\x00\x00\x00contrastsqG}qHX\x14\x00\x00\x00treated-vs-untreatedqI]qJ(X\x07\x00\x00\x00treatedqKX\t\x00\x00\x00untreatedqLessX\x06\x00\x00\x00paramsqM}qN(X\x04\x00\x00\x00starqOX\x00\x00\x00\x00qPX\x0b\x00\x00\x00cutadapt-seqQhPX\x0b\x00\x00\x00cutadapt-peqRhPuuX\x04\x00\x00\x00ruleqSX\r\x00\x00\x00rseqc_gtf2bedqTX\x0f\x00\x00\x00bench_iterationqUNX\t\x00\x00\x00scriptdirqVXL\x00\x00\x00/data/project/rna-seq-star-deseq2-public-pipeline/scriptsqWub.'); from snakemake.logging import logger; logger.printshellcmds = False; real_file = file; file = '/data/project/rna-seq-star-deseq2-public-pipeline/scripts/gtf2bed.py';
File "/share/apps/rc/software/snakemake/5.7.0-foss-2018b-Python-3.6.6/lib/python3.6/site-packages/snakemake/init.py", line 21, in
from snakemake.workflow import Workflow
File "/share/apps/rc/software/snakemake/5.7.0-foss-2018b-Python-3.6.6/lib/python3.6/site-packages/snakemake/workflow.py", line 30, in
from snakemake.dag import DAG
File "/share/apps/rc/software/snakemake/5.7.0-foss-2018b-Python-3.6.6/lib/python3.6/site-packages/snakemake/dag.py", line 1533
return f"#{hex_r:0>2X}{hex_g:0>2X}{hex_b:0>2X}"
^
SyntaxError: invalid syntax
[Tue Oct 15 14:25:53 2019]
Error in rule rseqc_gtf2bed:
jobid: 0
output: qc/rseqc/annotation.bed, qc/rseqc/annotation.db
log: logs/rseqc_gtf2bed.log (check log file(s) for error message)
conda-env: /data/project/rna-seq-star-deseq2-public-pipeline/.snakemake/conda/a2417d57
RuleException:
CalledProcessError in line 14 of /data/project/rna-seq-star-deseq2-public-pipeline/rules/qc.smk:
Command 'source /share/apps/rc/software/Anaconda3/5.3.1/bin/activate '/data/project/rna-seq-star-deseq2-public-pipeline/.snakemake/conda/a2417d57'; set -euo pipefail; python /data/project/rna-seq-star-deseq2-public-pipeline/.snakemake/scripts/tmp1qu6f74g.gtf2bed.py' returned non-zero exit status 1.
File "/data/project/rna-seq-star-deseq2-public-pipeline/rules/qc.smk", line 14, in __rule_rseqc_gtf2bed
File "/share/apps/rc/software/Python/3.6.6-foss-2018b/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /data/project/rna-seq-star-deseq2-public-pipeline/.snakemake/log/2019-10-15T142444.388725.snakemake.log
I apologize in advance if this is due to an error on my part but I welcome any insight you may have.
Thanks,
Sam
I tried running the rule with a different version of snakemake (version 5.2.4) and it successfully completed. It appears to be an issue with that specific version of snakemake (version 5.7.0).
I'm hoping this has not reappeared with newer snakemake versions. Please feel free to reopen if this happens again.