'mm10_no_alt_analysis_set_ENCODE.fasta.gz.lock' could not be acquired

Question

'mm10_no_alt_analysis_set_ENCODE.fasta.gz.lock' could not be acquired

hyjforesight opened this issue 2 years ago · comments

Describe the bug

Hello Encode,
Thanks for developing this amazing pipeline.
I met the errors 'mm10_no_alt_analysis_set_ENCODE.fasta.gz.lock' could not be acquired as below.
Could you please help me with this issue?
Thanks!
Best,
Yuanjian

caper run /home/hyjforesight/atac-seq-pipeline/atac.wdl -i /mnt/c/Users/yhuang14/Downloads/template_minimum_revised.json --conda
2022-09-12 15:32:42,281|caper.cli|INFO| Cromwell stdout: /home/hyjforesight/cromwell.out.4
2022-09-12 15:32:42,282|caper.caper_base|INFO| Creating a timestamped temporary directory. /home/hyjforesight/.caper_tmp/atac/20220912_153242_282178
2022-09-12 15:32:42,282|caper.caper_runner|INFO| Localizing files on work_dir. /home/hyjforesight/.caper_tmp/atac/20220912_153242_282178
2022-09-12 15:32:42,285|autouri.autouri|INFO| cp: (d627b290) started. src=https://www.encodeproject.org/files/mm10_no_alt_analysis_set_ENCODE/@@download/mm10_no_alt_analysis_set_ENCODE.fasta.gz, dest=/home/hyjforesight/.caper_tmp/72041e824adb499a4974d85ba31fe30d/mm10_no_alt_analysis_set_ENCODE.fasta.gz
Traceback (most recent call last):
  File "/home/hyjforesight/anaconda3/bin/caper", line 13, in <module>
    main()
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/caper/cli.py", line 713, in main
    return runner(parsed_args, nonblocking_server=nonblocking_server)
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/caper/cli.py", line 255, in runner
    subcmd_run(c, args)
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/caper/cli.py", line 386, in subcmd_run
    thread = caper_runner.run(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/caper/caper_runner.py", line 409, in run
    maybe_remote_file = self.localize_on_backend_if_modified(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/caper/caper_base.py", line 162, in localize_on_backend_if_modified
    f_loc = self.localize_on_backend(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/caper/caper_base.py", line 147, in localize_on_backend
    return AutoURI(f).localize_on(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/autouri.py", line 421, in localize_on
    return AutoURI.localize(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/autouri.py", line 643, in localize
    maybe_modified_contents, modified = fnc_recurse(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/loc_aux.py", line 56, in recurse_json
    modified = recurse_dict(d, fnc)
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/loc_aux.py", line 34, in recurse_dict
    modified |= recurse_dict(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/loc_aux.py", line 42, in recurse_dict
    new_val, modified_ = fnc(d)
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/autouri.py", line 628, in fnc_loc
    return cls.localize(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/autouri.py", line 643, in localize
    maybe_modified_contents, modified = fnc_recurse(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/loc_aux.py", line 83, in recurse_tsv
    new_val, modified_ = fnc(v)
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/autouri.py", line 628, in fnc_loc
    return cls.localize(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/autouri.py", line 660, in localize
    src_uri.cp(
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/autouri/autouri.py", line 298, in cp
    with d.get_lock(no_lock=no_lock):
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/filelock/_api.py", line 214, in __enter__
    self.acquire()
  File "/home/hyjforesight/anaconda3/lib/python3.9/site-packages/filelock/_api.py", line 177, in acquire
    raise Timeout(self._lock_file)
filelock._error.Timeout: The file lock '/home/hyjforesight/.caper_tmp/72041e824adb499a4974d85ba31fe30d/mm10_no_alt_analysis_set_ENCODE.fasta.gz.lock' could not be acquired.

BTW, I can downloaded this file manually by below coding.

wget https://www.encodeproject.org/files/mm10_no_alt_analysis_set_ENCODE/@@download/mm10_no_alt_analysis_set_ENCODE.fasta.gz
--2022-09-12 16:44:22--  https://www.encodeproject.org/files/mm10_no_alt_analysis_set_ENCODE/@@download/mm10_no_alt_analysis_set_ENCODE.fasta.gz
Resolving www.encodeproject.org (www.encodeproject.org)... 34.211.244.144
Connecting to www.encodeproject.org (www.encodeproject.org)|34.211.244.144|:443... connected.
HTTP request sent, awaiting response... 307 Temporary Redirect
Location: https://encode-public.s3.amazonaws.com/2016/01/23/b6a24661-be65-48a4-9e4e-44a007f35f5c/mm10_no_alt_analysis_set_ENCODE.fasta.gz?response-content-disposition=attachment%3B%20filename%3Dmm10_no_alt_analysis_set_ENCODE.fasta.gz&AWSAccessKeyId=ASIATGZNGCNX5UEV4ZEI&Signature=LWPPzULonFe2Uh3ztf6N5Nf3%2BRw%3D&x-amz-security-token=IQoJb3JpZ2luX2VjEP3%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLXdlc3QtMiJHMEUCIQCHqr6wSKtekV6ug0aQx1etwpF8C12Hpu4zqc2I7LwJXwIgXcJJaxgXJQA9oRuLYnqO8FH6uiaPDKEOGRCjvI1QhYUq1QQIlv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwyMjA3NDg3MTQ4NjMiDHLHjTQ6Vaf0Xoj%2BASqpBLcRw%2F3UiQ8sSFTCHEzL19f0rcfaMVrctWMo7CJjGfPN%2B9L7eE8jjyGk3QPhl%2Flq%2Bka2HjggOrRawjjhMgzBdePn4qACehed6dkbV5g1mxYQOS8QVK08pzEUJ9axOBdhlRkYib6sKF2iDBNwU2CmGHmDVoAZzOLV7PJ5MVD%2B1w6j1EvKookYOxlH2RnaismdtGXi5AyJtyW9Ygy86%2F%2B2s7z%2BEHWEcfF9BmU3Y9LiTmuJl7jhtZ0XuQKzj6Syy9zgZVQOsKLxY%2FKiG252z%2F1Qyx%2B%2FGXQfdE9HiBhR4fA%2Be19cPAUrsumeAISbtep2jgdxBcd%2FfNChkxpb3ikiisi0HzoShdR709gZlJ0OBhSZYmcjs0x6cws8tAAchmNT7v7pY1zhl3srP7CkwI6WAzUUkHGCLQSnvTS7bhwLrlazxuZb8o%2Ft2O52UrtySMRxMTmLvt65MS64WBgUKzRRDA86BSbs7XMt8bAV3Jm23Yi6a%2BqzVmPB1Cabn%2Blkszmg2KOFXywaAOqbWH6KTZ6c%2F7oGCB4oITpWA3nmTSqa1tY1NGHf9VR%2BuyUYcTgS%2FDiAk30nGGiVeLNuSTOY3zAz%2Fk%2BP2FFKWBsyJ5Kj9NjERGFVtZoXYD2wz1wM1eswCzqj7q66hUmLE8AK%2B6sXDkrRctXaXKgQIU0TuZvAwLtfRqC%2Bpx0fGm67D0dyFy6maZiTZYPuIe1M%2BeyFbSxtOUGUyEHdhdngma0suliQqJkwgbv%2BmAY6qQG5r42yAk5SW6OltNzfpsejmOD2h4Q2Z%2FggxQl8J3EvGX0NCbovaj5uI7ekn0dfH5DMhcYK%2BKQt3ojmvvnU5tybOkeOmjJ8JwGGl8Fze9QuXS1V1Dv4M2waxYLly7QeswuBT93N%2FAXADyeiq4q9vz98BRtoIDgXs9kKJGHiRpTWuUWWaX6vL%2FHAIKMdHFidEcI29ju5reXH3J24TzJX6Df0RrSkEDnVVH1S&Expires=1663148662 [following]
--2022-09-12 16:44:22--  https://encode-public.s3.amazonaws.com/2016/01/23/b6a24661-be65-48a4-9e4e-44a007f35f5c/mm10_no_alt_analysis_set_ENCODE.fasta.gz?response-content-disposition=attachment%3B%20filename%3Dmm10_no_alt_analysis_set_ENCODE.fasta.gz&AWSAccessKeyId=ASIATGZNGCNX5UEV4ZEI&Signature=LWPPzULonFe2Uh3ztf6N5Nf3%2BRw%3D&x-amz-security-token=IQoJb3JpZ2luX2VjEP3%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLXdlc3QtMiJHMEUCIQCHqr6wSKtekV6ug0aQx1etwpF8C12Hpu4zqc2I7LwJXwIgXcJJaxgXJQA9oRuLYnqO8FH6uiaPDKEOGRCjvI1QhYUq1QQIlv%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwyMjA3NDg3MTQ4NjMiDHLHjTQ6Vaf0Xoj%2BASqpBLcRw%2F3UiQ8sSFTCHEzL19f0rcfaMVrctWMo7CJjGfPN%2B9L7eE8jjyGk3QPhl%2Flq%2Bka2HjggOrRawjjhMgzBdePn4qACehed6dkbV5g1mxYQOS8QVK08pzEUJ9axOBdhlRkYib6sKF2iDBNwU2CmGHmDVoAZzOLV7PJ5MVD%2B1w6j1EvKookYOxlH2RnaismdtGXi5AyJtyW9Ygy86%2F%2B2s7z%2BEHWEcfF9BmU3Y9LiTmuJl7jhtZ0XuQKzj6Syy9zgZVQOsKLxY%2FKiG252z%2F1Qyx%2B%2FGXQfdE9HiBhR4fA%2Be19cPAUrsumeAISbtep2jgdxBcd%2FfNChkxpb3ikiisi0HzoShdR709gZlJ0OBhSZYmcjs0x6cws8tAAchmNT7v7pY1zhl3srP7CkwI6WAzUUkHGCLQSnvTS7bhwLrlazxuZb8o%2Ft2O52UrtySMRxMTmLvt65MS64WBgUKzRRDA86BSbs7XMt8bAV3Jm23Yi6a%2BqzVmPB1Cabn%2Blkszmg2KOFXywaAOqbWH6KTZ6c%2F7oGCB4oITpWA3nmTSqa1tY1NGHf9VR%2BuyUYcTgS%2FDiAk30nGGiVeLNuSTOY3zAz%2Fk%2BP2FFKWBsyJ5Kj9NjERGFVtZoXYD2wz1wM1eswCzqj7q66hUmLE8AK%2B6sXDkrRctXaXKgQIU0TuZvAwLtfRqC%2Bpx0fGm67D0dyFy6maZiTZYPuIe1M%2BeyFbSxtOUGUyEHdhdngma0suliQqJkwgbv%2BmAY6qQG5r42yAk5SW6OltNzfpsejmOD2h4Q2Z%2FggxQl8J3EvGX0NCbovaj5uI7ekn0dfH5DMhcYK%2BKQt3ojmvvnU5tybOkeOmjJ8JwGGl8Fze9QuXS1V1Dv4M2waxYLly7QeswuBT93N%2FAXADyeiq4q9vz98BRtoIDgXs9kKJGHiRpTWuUWWaX6vL%2FHAIKMdHFidEcI29ju5reXH3J24TzJX6Df0RrSkEDnVVH1S&Expires=1663148662
Resolving encode-public.s3.amazonaws.com (encode-public.s3.amazonaws.com)... 52.218.152.75
Connecting to encode-public.s3.amazonaws.com (encode-public.s3.amazonaws.com)|52.218.152.75|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 870379095 (830M) [binary/octet-stream]
Saving to: ‘mm10_no_alt_analysis_set_ENCODE.fasta.gz’

mm10_no_alt_analysis_set_ENCODE.fasta.gz                 0%[                                                                                                                            ]   1.90M   435KB/s    eta 42m 53s

OS/Platform

OS/Platform: Ubuntu 20.04
- Conda version: v4.14.0
Pipeline version: v2.2.0
Caper version: v2.2.2

Caper configuration file

Paste contents of ~/.caper/default.conf.

backend=local

# Local directory for localized files and Cromwell's intermediate files.
# If not defined then Caper will make .caper_tmp/ on CWD or `local-out-dir`.
# /tmp is not recommended since Caper store localized data files here.
local-loc-dir=

cromwell=/home/hyjforesight/.caper/cromwell_jar/cromwell-82.jar
womtool=/home/hyjforesight/.caper/womtool_jar/womtool-82.jar

Input JSON file

Paste contents of your input JSON file.

{
    "atac.title" : "Example (paired end)",
    "atac.description" : "This is a template input JSON for paired ended sample.",

    "atac.pipeline_type" : "atac",
    "atac.align_only" : false,
    "atac.true_rep_only" : false,

    "atac.genome_tsv" : "/home/hyjforesight/mm10.tsv",

    "atac.paired_end" : true,

    "atac.fastqs_rep1_R1" : [ "/home/hyjforesight/fastq/Cracd_KO1_IEC_1.fastq.gz" ],
    "atac.fastqs_rep1_R2" : [ "/home/hyjforesight/fastq/Cracd_KO1_IEC_2.fastq.gz" ],
    "atac.fastqs_rep2_R1" : [ "/home/hyjforesight/fastq/Cracd_KO2_IEC_1.fastq.gz" ],
    "atac.fastqs_rep2_R2" : [ "/home/hyjforesight/fastq/Cracd_KO2_IEC_2.fastq.gz" ],
    "atac.fastqs_rep3_R1" : [ "/home/hyjforesight/fastq/Cracd_WT2_IEC_1.fastq.gz" ],
    "atac.fastqs_rep3_R2" : [ "/home/hyjforesight/fastq/Cracd_WT2_IEC_2.fastq.gz" ],
    "atac.fastqs_rep4_R1" : [ "/home/hyjforesight/fastq/Cracd_WT3_IEC_1.fastq.gz" ],
    "atac.fastqs_rep4_R2" : [ "/home/hyjforesight/fastq/Cracd_WT3_IEC_2.fastq.gz" ],

    "atac.auto_detect_adapter" : false,
    "atac.adapter" : "CTGTCTCTTATACACATCT",
    "atac.adapters_rep1_R1" : [ "CTGTCTCTTATACACATCT" ],
    "atac.adapters_rep1_R2" : [ "CTGTCTCTTATACACATCT" ],
    "atac.adapters_rep2_R1" : [ "CTGTCTCTTATACACATCT" ],
    "atac.adapters_rep2_R2" : [ "CTGTCTCTTATACACATCT" ],
    "atac.adapters_rep3_R1" : [ "CTGTCTCTTATACACATCT" ],
    "atac.adapters_rep3_R2" : [ "CTGTCTCTTATACACATCT" ],
    "atac.adapters_rep4_R1" : [ "CTGTCTCTTATACACATCT" ],
    "atac.adapters_rep4_R2" : [ "CTGTCTCTTATACACATCT" ],

    "atac.multimapping" : 4
}

Troubleshooting result

If you ran caper run without Caper server then Caper automatically runs a troubleshooter for failed workflows. Find troubleshooting result in the bottom of Caper's screen log.

If you ran caper submit with a running Caper server then first find your workflow ID (1st column) with caper list and run caper debug [WORKFLOW_ID].

Paste troubleshooting result.

PASTE TROUBLESHOOTING RESULT HERE

Suffi Azizan · Answer 1 · Mon Sep 18 2023 20:27:27 GMT+0800 (China Standard Time)

Nobody knows the solution to this? I am facing the same error here.

Jin wook Lee · Answer 2 · Tue Nov 07 2023 05:45:35 GMT+0800 (China Standard Time)

To fix any .lock file issue, simply delete the lock file and try again.

$ rm /home/hyjforesight/.caper_tmp/72041e824adb499a4974d85ba31fe30d/mm10_no_alt_analysis_set_ENCODE.fasta.gz.lock