OSError: [Errno 13] Permission denied: '/local'

Question

OSError: [Errno 13] Permission denied: '/local'

ktroule opened this issue 6 years ago · comments

Hi

I know somebody down here had the same problem.

I'm trying to use optotype in docker:
docker run -v /local/pVACtools/:/data/ -t fred2/optitype --input SRR2672972_1.fastq SRR2672972_2.fastq --rna -o /local/pVACtools/Optitype/RNA_control

And I get this:

Traceback (most recent call last):
  File "/usr/local/bin/OptiType/OptiTypePipeline.py", line 235, in <module>
    os.makedirs(args.outdir)        
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/local'

I assume that python 2.7is also in the docker. I've tried to run this as sudo and ik keeps showing the same errors.

Thanks

Julius Muller · Answer 1 · Tue Mar 20 2018 20:26:38 GMT+0800 (China Standard Time)

Same here...

b-schubert · Answer 2 · Tue Mar 20 2018 22:13:06 GMT+0800 (China Standard Time)

the output path that you are specifying has to point to /data/

your path /local/pVACtools/ is the host system's path and is not known within Docker. What you are effectively trying to do is to write within the docker container to a virtual folder /local/pVACtools/. However, the only physical folder that has been mounted from your host system points to /data/ within the docker virtual environment.

Julius Muller · Answer 3 · Tue Mar 20 2018 23:19:08 GMT+0800 (China Standard Time)

Actually I did this, however the solution in my case was to give 777 permissions to the mounted, local folder.
I guess this is all related to the lack of experience with docker, but the next issue I ran into was then

sudo docker run -v /local/directory:/data/ -t fred2/optitype -i x_1.fastq.gz x_2.fastq.gz -d -o /data/

[E::hts_open_format] Failed to open file /data/2018_03_20_15_06_37/2018_03_20_15_06_37_1.bam Traceback (most recent call last): File "/usr/local/bin/OptiType/OptiTypePipeline.py", line 299, in <module> pos, read_details = ht.pysam_to_hdf(bam_paths[0]) File "/usr/local/bin/OptiType/hlatyper.py", line 186, in pysam_to_hdf sam = pysam.AlignmentFile(samfile, sam_or_bam) File "pysam/libcalignmentfile.pyx", line 444, in pysam.libcalignmentfile.AlignmentFile.__cinit__ File "pysam/libcalignmentfile.pyx", line 621, in pysam.libcalignmentfile.AlignmentFile._open IOError: [Errno 2] could not open alignment file /data/2018_03_20_15_06_37/2018_03_20_15_06_37_1.bam: No such file or directory

Any idea?

Weichi · Answer 4 · Fri May 04 2018 22:25:05 GMT+0800 (China Standard Time)

I have the same problem...

[weichi@ngsteam Optitype_test]$ docker run -v /home/weichi/working:/Optitype_test/  fred2/optitype -i TVB019ND_sorted.fastq -d -v -o ./Optitype_test/

mapping with 1 threads...

 0:00:00.36 Mapping TVB019ND_sorted.fastq to GEN reference...

 0:00:01.06 Generating binary hit matrix.
[E::hts_open_format] Failed to open file ./Optitype_test/2018_05_04_14_18_47/2018_05_04_14_18_47_1.bam
Traceback (most recent call last):
  File "/usr/local/bin/OptiType/OptiTypePipeline.py", line 358, in <module>
    pos, read_details = ht.pysam_to_hdf(bam_paths[0])
  File "/usr/local/bin/OptiType/hlatyper.py", line 186, in pysam_to_hdf
    sam = pysam.AlignmentFile(samfile, sam_or_bam)
  File "pysam/libcalignmentfile.pyx", line 728, in pysam.libcalignmentfile.AlignmentFile.__cinit__
  File "pysam/libcalignmentfile.pyx", line 918, in pysam.libcalignmentfile.AlignmentFile._open
IOError: [Errno 2] could not open alignment file `./Optitype_test/2018_05_04_14_18_47/2018_05_04_14_18_47_1.bam`: No such file or directory

b-schubert · Answer 5 · Fri May 04 2018 23:09:35 GMT+0800 (China Standard Time)

@ju-mu This might be related to gzipped fastq files? could you try to run dockerized optitype on an unzipped file?

@jo6fu3 you are using the wrong internal folder. /Optitype_test does not exist in the docker-VM and is not set as working directory. Please use the predefined /data folder

If you run optitype as follows you should not have any problems:

docker run -v /home/weichi/working:/data -d fred2/optitype -i TVB019ND_sorted.fastq -d -o /da

assuming that TVB019ND_sorted.fastq is un /home/weichi/working

Weichi · Answer 6 · Mon May 07 2018 11:15:56 GMT+0800 (China Standard Time)

@b-schubert Thanks for your help.
But I got the other problem.
I have no idea what happened.

docker run -v /home/weichi/working/Optitype_test:/data/ fred2/optitype -i TVB019ND_sorted.fastq -d -v -o /data/
/data/seqan-src/include/seqan/basic/basic_exception.h:363 FAILED!  (Uncaught exception of type std::bad_alloc: std::bad_alloc)

stack trace:
  0          [0x8917fd]  /usr/local/bin/razers3()
  1    [0x7fe23b98c6b6]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d6b6)
  2    [0x7fe23b98c701]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d701)
  3    [0x7fe23b98c919]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d919)
  4    [0x7fe23b98cebc]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8debc)
  5          [0x620458]  void seqan::AppendString_<seqan::Tag<seqan::TagGenerous_> >::append_<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> > const>(seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >&, seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> > const&) + 0x88
  6          [0x77bbed]  bool seqan::loadReads<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig>, seqan::RazerSOptions<seqan::RazerSSpec<false, false> > >(seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >&, seqan::FormattedFile<seqan::Tag<seqan::TagFastq_>, seqan::Tag<seqan::Input_>, void>&, seqan::RazerSOptions<seqan::RazerSSpec<false, false> >&) + 0x31d
  7          [0x88ddd2]  int mapReads<seqan::RazerSSpec<false, false> >(seqan::StringSet<seqan::String<char, seqan::Alloc<void> >, seqan::Owner<seqan::Tag<seqan::Default_> > >&, seqan::StringSet<seqan::String<char, seqan::Alloc<void> >, seqan::Owner<seqan::Tag<seqan::Default_> > >&, seqan::RazerSOptions<seqan::RazerSSpec<false, false> >&) + 0x532
  8          [0x572e65]  main + 0x845
  9    [0x7fe23abf7830]  __libc_start_main + 0xf0
 10          [0x573d69]  _start + 0x29

Aborted (core dumped)

mapping with 1 threads...

 0:00:03.14 Mapping TVB019ND_sorted.fastq to GEN reference...

 0:11:33.03 Generating binary hit matrix.
[E::hts_open_format] Failed to open file /data/2018_05_07_02_54_47/2018_05_07_02_54_47_1.bam
Traceback (most recent call last):
  File "/usr/local/bin/OptiType/OptiTypePipeline.py", line 358, in <module>
    pos, read_details = ht.pysam_to_hdf(bam_paths[0])
  File "/usr/local/bin/OptiType/hlatyper.py", line 186, in pysam_to_hdf
    sam = pysam.AlignmentFile(samfile, sam_or_bam)
  File "pysam/libcalignmentfile.pyx", line 728, in pysam.libcalignmentfile.AlignmentFile.__cinit__
  File "pysam/libcalignmentfile.pyx", line 918, in pysam.libcalignmentfile.AlignmentFile._open
IOError: [Errno 2] could not open alignment file `/data/2018_05_07_02_54_47/2018_05_07_02_54_47_1.bam`: No such file or directory

Weichi · Answer 7 · Mon May 07 2018 21:55:48 GMT+0800 (China Standard Time)

On the other hand, I used BWA to map my fastq to hla_reference genome data/hla_reference_dna.fasta then generate the filtered fastq. But I got the same error message when used this filtered fastq.

b-schubert · Answer 8 · Mon May 07 2018 21:57:33 GMT+0800 (China Standard Time)

Hmmm, how did you build the docker container? Did you use the pre-compiled docker container on docker-hub or did you compile it yourself?

Weichi · Answer 9 · Mon May 07 2018 22:28:13 GMT+0800 (China Standard Time)

I follow the " Installation via Docker " on this website (https://hub.docker.com/r/fred2/optitype/).
docker pull fred2/optitype
I think it's pre-compiled docker container on docker-hub.

b-schubert · Answer 10 · Mon May 07 2018 22:54:02 GMT+0800 (China Standard Time)

Ohm okay. Your base operating system is Linux a presume? Have you checked you read&write access to your data folder? It has to be write accessible to everybody. Or if that is an security concern than you have to add the user biodocker to the owner group of the folder.

Weichi · Answer 11 · Tue May 08 2018 00:07:02 GMT+0800 (China Standard Time)

Yes, my operating system is CentOS Linux release 7.4.1708.
I use chmod -R 777 data to change the read&write access, but got the same error message.
drwxrwxrwx. 3 weichi sftp_group 33 May 7 23:12 data
sudo docker run -v /home/weichi/working/Optitype_test:/data fred2/optitype -i TVB019ND_for_hla.fastq -d -v -o ./data

ps. I don't have the user named biodocker but have docker in the owner group of the folder.

/data/seqan-src/include/seqan/basic/basic_exception.h:363 FAILED!  (Uncaught exception of type std::bad_alloc: std::bad_alloc)

stack trace:
  0          [0x8917fd]  /usr/local/bin/razers3()
  1    [0x7f24705506b6]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d6b6)
  2    [0x7f2470550701]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d701)
  3    [0x7f2470550919]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d919)
  4    [0x7f2470550ebc]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8debc)
  5          [0x620458]  void seqan::AppendString_<seqan::Tag<seqan::TagGenerous_> >::append_<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> > const>(seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >&, seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> > const&) + 0x88
  6          [0x77bbed]  bool seqan::loadReads<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig>, seqan::RazerSOptions<seqan::RazerSSpec<false, false> > >(seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >&, seqan::FormattedFile<seqan::Tag<seqan::TagFastq_>, seqan::Tag<seqan::Input_>, void>&, seqan::RazerSOptions<seqan::RazerSSpec<false, false> >&) + 0x31d
  7          [0x88ddd2]  int mapReads<seqan::RazerSSpec<false, false> >(seqan::StringSet<seqan::String<char, seqan::Alloc<void> >, seqan::Owner<seqan::Tag<seqan::Default_> > >&, seqan::StringSet<seqan::String<char, seqan::Alloc<void> >, seqan::Owner<seqan::Tag<seqan::Default_> > >&, seqan::RazerSOptions<seqan::RazerSSpec<false, false> >&) + 0x532
  8          [0x572e65]  main + 0x845
  9    [0x7f246f7bb830]  __libc_start_main + 0xf0
 10          [0x573d69]  _start + 0x29

Aborted (core dumped)

mapping with 1 threads...

 0:00:02.67 Mapping TVB019ND_for_hla.fastq to GEN reference...

 0:17:01.84 Generating binary hit matrix.
[E::hts_open_format] Failed to open file ./data/2018_05_07_15_43_23/2018_05_07_15_43_23_1.bam
Traceback (most recent call last):
  File "/usr/local/bin/OptiType/OptiTypePipeline.py", line 358, in <module>
    pos, read_details = ht.pysam_to_hdf(bam_paths[0])
  File "/usr/local/bin/OptiType/hlatyper.py", line 186, in pysam_to_hdf
    sam = pysam.AlignmentFile(samfile, sam_or_bam)
  File "pysam/libcalignmentfile.pyx", line 728, in pysam.libcalignmentfile.AlignmentFile.__cinit__
  File "pysam/libcalignmentfile.pyx", line 918, in pysam.libcalignmentfile.AlignmentFile._open
IOError: [Errno 2] could not open alignment file `./data/2018_05_07_15_43_23/2018_05_07_15_43_23_1.bam`: No such file or directory

b-schubert · Answer 12 · Tue May 08 2018 01:42:43 GMT+0800 (China Standard Time)

-o ./data

has to be just /data

Weichi · Answer 13 · Tue May 08 2018 11:11:05 GMT+0800 (China Standard Time)

I tried this two command -o /data/ and -o /data, but got the same problem.

If I used -o ./data it would create a folder named data and put 2018_05_xx_xx_xx_xx folder under it.
If I used -o /data it would not create the data folder, only create the 2018_05_xx_xx_xx_xx folder.
But I always got the error message IOError: [Errno 2] could not open alignment file ./data/2018_05_07_15_43_23/2018_05_07_15_43_23_1.bam: No such file or directory in the end.

And is it correct to got this message?
/data/seqan-src/include/seqan/basic/basic_exception.h:363 FAILED! (Uncaught exception of type std::bad_alloc: std::bad_alloc)

Jules Lasne · Answer 14 · Wed Jun 06 2018 13:34:08 GMT+0800 (China Standard Time)

I am getting this error message on jenkins with blueocean:

https://imgur.com/JFGGkZh

https://pastebin.com/eumaEwVU

Any idea ?

racharyaUC · Answer 15 · Tue Jul 31 2018 10:43:13 GMT+0800 (China Standard Time)

Mounting an additional volume for output files still gives this error.
Example:
docker run -v /local/folder:/data/ -v /another/local/folder:/output/ -t fred2/optitype -i forward.fastq reverse.fastq -d -o /output/
results in the error:

Traceback (most recent call last):
  File "/usr/local/bin/OptiType/OptiTypePipeline.py", line 275, in <module>
    os.makedirs(out_dir)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/output/2018_07_31_02_28_19'

tufanchina · Answer 16 · Thu Jan 24 2019 07:15:03 GMT+0800 (China Standard Time)

I had the similar error message
IOError: [Errno 2] could not open alignment file ./data/2019_01_23_23_10_57/2019_01_23_23_10_57_1.bam: No such file or directory

I found that it was because the reads file is too large, probably out of the setting for docker,
I made a smaller subset fastq file out of the reads file and then it works great.

Hopefully it will help.

Deleted user · Answer 17 · Wed Dec 04 2019 04:53:50 GMT+0800 (China Standard Time)

I had the same error message, I increased the memory in docker (docker whale --> preferences --> advanced) and it worked.

Zhang Jingjing · Answer 18 · Wed Oct 21 2020 09:06:49 GMT+0800 (China Standard Time)

Yes, my operating system is CentOS Linux release 7.4.1708.
I use chmod -R 777 data to change the read&write access, but got the same error message.
drwxrwxrwx. 3 weichi sftp_group 33 May 7 23:12 data
sudo docker run -v /home/weichi/working/Optitype_test:/data fred2/optitype -i TVB019ND_for_hla.fastq -d -v -o ./data

ps. I don't have the user named biodocker but have docker in the owner group of the folder.

/data/seqan-src/include/seqan/basic/basic_exception.h:363 FAILED!  (Uncaught exception of type std::bad_alloc: std::bad_alloc)

stack trace:
  0          [0x8917fd]  /usr/local/bin/razers3()
  1    [0x7f24705506b6]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d6b6)
  2    [0x7f2470550701]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d701)
  3    [0x7f2470550919]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8d919)
  4    [0x7f2470550ebc]  /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8debc)
  5          [0x620458]  void seqan::AppendString_<seqan::Tag<seqan::TagGenerous_> >::append_<seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >, seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> > const>(seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> >&, seqan::String<seqan::SimpleType<unsigned char, seqan::Dna5Q_>, seqan::Alloc<void> > const&) + 0x88
  6          [0x77bbed]  bool seqan::loadReads<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig>, seqan::RazerSOptions<seqan::RazerSSpec<false, false> > >(seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig<MyFragStoreConfig> >&, seqan::FormattedFile<seqan::Tag<seqan::TagFastq_>, seqan::Tag<seqan::Input_>, void>&, seqan::RazerSOptions<seqan::RazerSSpec<false, false> >&) + 0x31d
  7          [0x88ddd2]  int mapReads<seqan::RazerSSpec<false, false> >(seqan::StringSet<seqan::String<char, seqan::Alloc<void> >, seqan::Owner<seqan::Tag<seqan::Default_> > >&, seqan::StringSet<seqan::String<char, seqan::Alloc<void> >, seqan::Owner<seqan::Tag<seqan::Default_> > >&, seqan::RazerSOptions<seqan::RazerSSpec<false, false> >&) + 0x532
  8          [0x572e65]  main + 0x845
  9    [0x7f246f7bb830]  __libc_start_main + 0xf0
 10          [0x573d69]  _start + 0x29

Aborted (core dumped)

mapping with 1 threads...

 0:00:02.67 Mapping TVB019ND_for_hla.fastq to GEN reference...

 0:17:01.84 Generating binary hit matrix.
[E::hts_open_format] Failed to open file ./data/2018_05_07_15_43_23/2018_05_07_15_43_23_1.bam
Traceback (most recent call last):
  File "/usr/local/bin/OptiType/OptiTypePipeline.py", line 358, in <module>
    pos, read_details = ht.pysam_to_hdf(bam_paths[0])
  File "/usr/local/bin/OptiType/hlatyper.py", line 186, in pysam_to_hdf
    sam = pysam.AlignmentFile(samfile, sam_or_bam)
  File "pysam/libcalignmentfile.pyx", line 728, in pysam.libcalignmentfile.AlignmentFile.__cinit__
  File "pysam/libcalignmentfile.pyx", line 918, in pysam.libcalignmentfile.AlignmentFile._open
IOError: [Errno 2] could not open alignment file `./data/2018_05_07_15_43_23/2018_05_07_15_43_23_1.bam`: No such file or directory

Hi, I had the same error message. Can I ask if you have solved this problem?
`/opt/conda/conda-bld/razers3_1592544302093/work/include/seqan/basic/basic_exception.h:363 FAILED! (Uncaught exception of type std::bad_alloc: std::bad_alloc)

stack trace:
0 [0x2ad856d790f4] razers3(+0x5cc0f4)
1 [0x2ad857605f69] /home/zhangjj/software/miniconda3/envs/optitype/bin/../lib/libstdc++.so.6(+0xacf69)
2 [0x2ad857605fab] /home/zhangjj/software/miniconda3/envs/optitype/bin/../lib/libstdc++.so.6(+0xacfab)
3 [0x2ad857606194] __cxa_rethrow + 0
4 [0x2ad85760656a] operator new(unsigned long) + 0x52
5 [0x2ad856a7351f] razers3(+0x2c651f)
6 [0x2ad856bd7876] bool seqan::loadReads<MyFragStoreConfig, seqan::FragmentStoreConfig, seqan::RazerSOptions<seqan::RazerSSpec<false, false> > >(seqan::FragmentStore<MyFragStoreConfig, seqan::FragmentStoreConfig >&, seqan::FormattedFile<seqan::Tagseqan::TagFastq_, seqan::Tagseqan::Input_, void>&, seqan::RazerSOptions<seqan::RazerSSpec<false, false> >&) + 0x2a6
7 [0x2ad856d78478] int mapReads<seqan::RazerSSpec<false, false> >(seqan::StringSet<seqan::String<char, seqan::Alloc >, seqan::Owner<seqan::Tagseqan::Default_ > >&, seqan::StringSet<seqan::String<char, seqan::Alloc >, seqan::Owner<seqan::Tagseqan::Default_ > >&, seqan::RazerSOptions<seqan::RazerSSpec<false, false> >&) + 0x6f8
8 [0x2ad856a212f9] main + 0x849
9 [0x2ad857bd0d5d] __libc_start_main + 0xfd
10 [0x2ad856a219d6] razers3(+0x2749d6)

Aborted (core dumped)`

Magisterh1 · Answer 19 · Thu May 05 2022 17:36:33 GMT+0800 (China Standard Time)

I had the same permission errors (Ubuntu server). I did the following and so far optitype is running:
sudo adduser biodocker
sudo chown -R :users /work/Projects
sudo chmod -R 777 /work/Projects
sudo docker run -v /work/Projects:/data -t fred2/optitype -i EA005_001_1.fq EA005_001_2.fq -d -o /data

Update:
Optitype finished without errors :)