Giters

MIC-DKFZ / medicaldetectiontoolkit

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Batch generation error on my customized dataset

sunformoon opened this issue · comments

commented

Thanks for the very helpful repo!
First, I got the code running on the example LIDC data and the training looked fine. Then I switched to my customized dataset , which looks like the following snapshot:

Screenshot from 2021-05-16 21:39:42

Each patient has only one segmented ROI, so the class_target only has one value while the total number of foreground is 5 (larger than 2).

At the beginning, the training was stuck in the get_class_balanced_patients, so I switched to the random sample "batch_ixs = np.random.choice(len(class_targets_list), self.batch_size)".

However, I got the following errors from the batch generator. Would you have any comment on how to get rid of this error? Thank you a lot!!

data set loaded with: 118 train / 40 val / 40 test patients
starting training epoch 1
Exception in worker 4
Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/batchgenerators/dataloading/multi_threaded_augmenter.py", line 48, in producer
item = transform(**item)
File "/opt/conda/lib/python3.6/site-packages/batchgenerators/transforms/abstract_transforms.py", line 88, in call
data_dict = t(**data_dict)
File "/opt/conda/lib/python3.6/site-packages/batchgenerators/transforms/utility_transforms.py", line 225, in call
data_dict = convert_seg_to_bounding_box_coordinates(data_dict, self.dim, self.get_rois_from_seg_flag, class_specific_seg_flag=self.class_specific_seg_flag)
File "/opt/conda/lib/python3.6/site-packages/batchgenerators/augmentations/utils.py", line 522, in convert_seg_to_bounding_box_coordinates
p_roi_labels_list.append(data_dict['class_target'][b][rix] + 1)
IndexError: index 4 is out of bounds for axis 0 with size 1
Traceback (most recent call last):
File "exec.py", line 176, in
train(logger)
File "exec.py", line 68, in train
batch = next(batch_gen['train'])
File "/opt/conda/lib/python3.6/site-packages/batchgenerators/dataloading/multi_threaded_augmenter.py", line 190, in next
item = self.__get_next_item()
File "/opt/conda/lib/python3.6/site-packages/batchgenerators/dataloading/multi_threaded_augmenter.py", line 172, in __get_next_item
raise RuntimeError("MultiThreadedAugmenter.abort_event was set, something went wrong. Maybe one of "
RuntimeError: MultiThreadedAugmenter.abort_event was set, something went wrong. Maybe one of your workers crashed

commented

Hello! I am having the same problem. Did you manage to solve it? Thank you in advance.