IsADirectoryError: [Errno 21] Is a directory: '/home/mdimdadi/14777R/FastQ/Passed_Reads/14777x5/5_superfocus_output'

Question

IsADirectoryError: [Errno 21] Is a directory: '/home/mdimdadi/14777R/FastQ/Passed_Reads/14777x5/5_superfocus_output'

opened this issue 5 years ago · comments

When I run the following command I get an error:
$ superfocus -q 14777x5_All_Read_Merged.fastq -dir 5_superfocus_output/ -a diamond -t 25 -e 0.00001 -db DB_90 -l 5_superfocus_output/ &

[mdimdadi@rameshgoel 14777x5]$ Traceback (most recent call last):
File "/usr/bin/superfocus", line 10, in
sys.exit(main())
File "/usr/bin/miniconda3/lib/python3.6/site-packages/superfocus_app/superfocus.py", line 264, in main
logging.basicConfig(format=LOGGER_FORMAT, level=logging.INFO, filename=args.log)
File "/usr/bin/miniconda3/lib/python3.6/logging/init.py", line 1806, in basicConfig
h = FileHandler(filename, mode)
File "/usr/bin/miniconda3/lib/python3.6/logging/init.py", line 1030, in init
StreamHandler.init(self, self._open())
File "/usr/bin/miniconda3/lib/python3.6/logging/init.py", line 1059, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
IsADirectoryError: [Errno 21] Is a directory: '/home/mdimdadi/14777R/FastQ/Passed_Reads/14777x5/5_superfocus_output'

Geni Silva commented 5 years ago

yep

Geni Silva · Answer 1 · Fri Apr 05 2019 02:45:31 GMT+0800 (China Standard Time)

-l requires a file for the log rather than a directory.

Deleted user · Answer 2 · Fri Apr 05 2019 06:26:38 GMT+0800 (China Standard Time)

Hi @metageni Thanks, that helped.

Now I ran the super-focus, it launches successfully. But after some time, I get the following error:

Processing query chunk 0, reference chunk 0, shape 1, index chunk 1.
Building reference index... [5.16536s]
Building query index... [4.31462s]
Building seed filter... [0.37562s]
Searching alignments... Error: Error writing file /usr/bin/miniconda3/lib/python Error: terminate called after throwing an instance of 'File_write_exception'
Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superfocus_ap Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
what(): Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/su terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
Error: Error writing file /usr/bin/miniconda3/lib/python3.6/site-packages/superf terminate called recursively
diamond v0.8.22.84 | by Benjamin Buchfink p/db/tmp/diamond-5cb18e66-21.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-20.tmp. Disk full?
perfocus_app/db/tmp/diamond-5cb18e66-15.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-49.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-41.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-45.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-d.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-2c.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-9.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-19.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-3f.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-11.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-2d.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-35.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-39.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-5.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-1.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-25.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-31.tmp. Disk full?
ocus_app/db/tmp/diamond-5cb18e66-4d.tmp. Disk full?
rel="nofollow">buchfink@gmail.com
title="http://github.com/bbuchfink/diamond">http://github.com/bbuchfink/diamond for updates.

#CPU threads: 32
Error: Invalid DAA file. DIAMOND run probably has not completed successfully.
Traceback (most recent call last):
File "/usr/bin/miniconda3/bin/superfocus", line 10, in
sys.exit(main())
File "/usr/bin/miniconda3/lib/python3.6/site-packages/superfocus_app/superfocu s.py", line 342, in main
del_alignments)
ValueError: not enough values to unpack (expected 2, got 0)

Thanks again.

Geni Silva · Answer 3 · Fri Apr 05 2019 06:30:40 GMT+0800 (China Standard Time)

How much storage do you have available? The error message seems to point that the alignments temp file cannot be written Disk full?

Deleted user · Answer 4 · Fri Apr 05 2019 06:33:18 GMT+0800 (China Standard Time)

[mdimdadi@rameshgoel data]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 50G 44G 6.8G 87% /
devtmpfs 126G 0 126G 0% /dev
tmpfs 126G 0 126G 0% /dev/shm
tmpfs 126G 2.5G 124G 2% /run
tmpfs 126G 0 126G 0% /sys/fs/cgroup
/dev/sde1 497M 350M 147M 71% /boot
/dev/mapper/centos-lv_local 59G 918M 56G 2% /usr/local
/dev/mapper/vg_data-lv_shared 788G 275G 477G 37% /srv/data
/dev/mapper/vg_data-lv_user 1.4T 894G 445G 67% /home
tmpfs 26G 0 26G 0% /run/user/1001
tmpfs 26G 0 26G 0% /run/user/1012

can we divert to write the temp file to write in other place?

Geni Silva · Answer 5 · Fri Apr 05 2019 06:44:14 GMT+0800 (China Standard Time)

Try to modify https://github.com/metageni/SUPER-FOCUS/blob/master/superfocus_app/do_alignment.py#L77

I think it should be on your computer at /usr/bin/miniconda3/lib/python3.6/site-packages/superfocus_app/do_alignment.py

edit temp_folder to be the Path you want the temp files to go. I will add a fix on next release so people can pass it as an optional parameter.

Deleted user · Answer 6 · Fri Apr 05 2019 06:56:47 GMT+0800 (China Standard Time)

when I open the do_alignment.py, I see
temp_folder = Path("{}/db/tmp/".format(WORK_DIRECTORY))

I want to write the temp file say in /srv/data/ should I make it like the following:

temp_folder = Path("/srv/data/".format(WORK_DIRECTORY))

Thanks @metageni

Geni Silva · Answer 7 · Fri Apr 05 2019 06:57:41 GMT+0800 (China Standard Time)

Sure. Please keep me posted if it will solve your problem.

Geni Silva · Answer 8 · Fri Apr 05 2019 06:58:32 GMT+0800 (China Standard Time)

I think you will want to set temp_folder = PATH_WHERE_YOU_WANT_TEMP_FILES

temp_folder = Path("/srv/data/".format(WORK_DIRECTORY)) may not work.

Deleted user · Answer 9 · Fri Apr 05 2019 07:03:40 GMT+0800 (China Standard Time)

so like this:
temp_folder = Path("/srv/data/")

Deleted user · Answer 10 · Fri Apr 05 2019 07:29:05 GMT+0800 (China Standard Time)

The first way is working I think. It's not stopping anymore for DISK_FULL reason.

Thank you very much, You are awesome!!