FileNotFoundError for "Clean" data

Question

FileNotFoundError for "Clean" data

harsharaman opened this issue 4 years ago · comments

Hi Jaeho, my name is Harsha. Thank you for this repository. I am using it to preprocess data for my segmentation model. My issue is that as soon as your script encounters the first "clean" data files, it throws the following error:
Patient ID: LIDC-IDRI-0028 Dicom Shape: (512, 512, 141) Number of Annotated Nodules: 0
Clean Dataset LIDC-IDRI-0028
Traceback (most recent call last):
File "prepare_dataset.py", line 173, in
test.prepare_dataset()
File "prepare_dataset.py", line 157, in prepare_dataset
np.save(patient_clean_dir_mask / mask_name, lung_mask)
File "<array_function internals>", line 6, in save
File "/home/ramanha/env3/lib/python3.5/site-packages/numpy/lib/npyio.py", line 541, in save
fid = open(file, "wb")
FileNotFoundError: [Errno 2] No such file or directory: 'data/Clean/Mask/LIDC-IDRI-0028/LIDC-IDRI-0028/0028_CM001_slice000.npy'

jaeho3690 · Answer 1 · Tue Nov 17 2020 06:38:28 GMT+0800 (China Standard Time)

Hi, have you made the data/Clean directory?

Harsha Raman · Answer 2 · Tue Nov 17 2020 18:43:17 GMT+0800 (China Standard Time)

data/Clean was present from before. Do I have to explicitly make it?

jaeho3690 · Answer 3 · Tue Nov 17 2020 18:58:52 GMT+0800 (China Standard Time)

The error that is throwing is Filenotfounderror. 'data/Clean/Mask' this folder should exist in your directory as the clean .npy files are saved in that directory.

Harsha Raman · Answer 4 · Tue Nov 17 2020 19:29:14 GMT+0800 (China Standard Time)

data/Clean/Image and data/Clean/Mask folders exist and I have checked them. This is why I got confused about this error. The folders are present.

jaeho3690 · Answer 5 · Tue Nov 17 2020 20:53:22 GMT+0800 (China Standard Time)

It seems that the script is trying to make a same folder.
"LIDC-IDRI-0028/LIDC-IDRI-0028/"
-> there should only be one folder per patient. So it should actually be
'data/Clean/Mask/LIDC-IDRI-0028/0028_CM001_slice000.npy'

jaeho3690 · Answer 6 · Tue Nov 17 2020 20:55:15 GMT+0800 (China Standard Time)

Please check the config settings and codes that are making directories

Harsha Raman · Answer 7 · Fri Nov 20 2020 12:08:24 GMT+0800 (China Standard Time)

Hi, so I have found the error: In lines 152 and 153 of prepare_dataset.py, there's an extra {} which creates the folder.
nodule_name = "{}/{}_CN001_slice{}".format(pid,pid[-4:],prefix[slice])
mask_name = "{}/{}_CM001_slice{}".format(pid,pid[-4:],prefix[slice])

Please remove it and this issue is resolved :D

jaeho3690 · Answer 8 · Fri Nov 20 2020 12:12:13 GMT+0800 (China Standard Time)

Great to hear that! I am not sure why such a problem occurs because it worked for mine but I will have a look at it! Thanks a lot :) and good luck!