some problems during traning and testing
thinkInJava33 opened this issue · comments
Epoch: [1/1000]
2019-10-26 08:58 Iteration: [0/22] Loss 8.0491 Loss_CE 8.0491 Loss_var 0.0000 Pixel_Accu 0.6642 IoU 0.0000
2019-10-26 08:59 => Train Avg: Loss 3.0851 Loss_CE 3.0851 Loss_var 0.0000 Pixel_Accu 0.8113 IoU 0.0000
here are some training logs,the dataset is GlaS ,i wonder why Loss_var and IoU is always 0 during traning(1000epochs) ,and during test,all metrics expect pixel accuracy is 0.i didn't change anything about the code
i found the reason is that in my_transform,code in label_encoding cause this problem
i change
new_label[label[:, :, 0] > 255*0.5] = 1 # inside
to
new_label[label[:, :, 0] > 0.5] = 1 # inside
my problem is solved.
@thinkInJava33 How about your results on GlaS dataset using this code， can you show me？I think I have made some mistakes so that I couldn't get a satisfactory result. Maybe you can help me.
@Megumi-cs I got the same result on testB but a little bit worse result on test A compared to the result in the paper.
I want to know some details. do you change some parameters or modify some codes,？do you make some operations to the glass dataset.？
@Megumi-cs the only change I made is showed in this issue. nothing else. what is your problem?
do you modify the self.dataset = 'GlaS',
here are the result on test A:
Metrics: pixel_acc recall precision F1 Dice IoU Hausdorff
Average: 0.9367 0.9300 0.8960 0.9056 0.9101 0.8592 39.3677
testB:
Metrics: pixel_acc recall precision F1 Dice IoU Hausdorff
Average: 0.9023 0.8780 0.7684 0.8089 0.8517 0.7919 89.5776
@thinkInJava33 I have modified the self.dataset to 'GlaS'，and i can got results. my problem is i cant got the same results as paper mentions, so i think i got something wrong.
here is my results:
on testA:
Metrics: pixel_acc recall precision F1 Dice IoU Hausdorff
Average: 0.9357 0.9229 0.9147 0.9099 0.9133 0.8634 37.9995
on testB:
Metrics: pixel_acc recall precision F1 Dice IoU Hausdorff
Average: 0.8903 0.8429 0.8138 0.8091 0.8400 0.7760 100.4424
@thinkInJava33 I have modified the self.dataset to 'GlaS'，and i can got results. my problem is i cant got the same results as paper mentions, so i think i got something wrong.
here is my results:
on testA:
Metrics: pixel_acc recall precision F1 Dice IoU Hausdorff
Average: 0.9357 0.9229 0.9147 0.9099 0.9133 0.8634 37.9995
on testB:
Metrics: pixel_acc recall precision F1 Dice IoU Hausdorff
Average: 0.8903 0.8429 0.8138 0.8091 0.8400 0.7760 100.4424
do you get the satisfied result now?
@thinkInJava33
Here is my best result now
on testA：
Metrics: F1 Dice Hausdorff
Average: 0.9099 0.9147 38.6177
on testeB：
Metrics: F1 Dice Hausdorff
Average: 0.8355 0.8521 92.1046
it is much better than before.
You need an eroding to your labels，because it actually should be the inside area of gland，however the original labels include both of inside area and the contour. You can read LabelEncoding function in my_transform.py for more about it.
Now, i‘m not sure the best radius of eroding operation. It needs some tries. Besides, min_areas in options.py can also improve the result, maybe 140 is better.
@thinkInJava33
Here is my best result now
on testA：
Metrics: F1 Dice Hausdorff
Average: 0.9099 0.9147 38.6177
on testeB：
Metrics: F1 Dice Hausdorff
Average: 0.8355 0.8521 92.1046
it is much better than before.
You need an eroding to your labels，because it actually should be the inside area of gland，however the original labels include both of inside area and the contour. You can read LabelEncoding function in my_transform.py for more about it.
Now, i‘m not sure the best radius of eroding operation. It needs some tries. Besides, min_areas in options.py can also improve the result, maybe 140 is better.
soryy, I'm not sure I understand what you meant,is there anything wrong about the following code?
0 for background, 1 for inside and 2 for boundary,and boundary = dilation - erosion
new_label = np.zeros((label.shape[0], label.shape[1]), dtype=np.uint8)
new_label[label[:, :, 0] > 0.5] = 1 # inside
boun = morphology.dilation(new_label) & (~morphology.erosion(new_label, morphology.disk(self.radius)))
new_label[boun > 0] = 2 # boundary
this code is right，i mean you should make an erosion to the label first.
this code is right，i mean you should make an erosion to the label first.
I did as what you said, but the result did not get better.
@thinkInJava33
mask[mask > 0] = 1
new_mask= morphology.erosion(mask, morphology.disk(4))
I use this code to _anno.bmp
@Megumi-cs I still can't get the better result, and I have no idea what is wrong.do you use the whole training data for traning or spilt the traning data into training set and Val set?
@thinkInJava33 I use the whole training data for traning，and testA data for val.
@Megumi-cs
still no better result, is this what you mean?
new_label = np.zeros((label.shape[0], label.shape[1]), dtype=np.uint8)
new_label[label[:, :, 0] > 0.5] = 1 # inside
new_label = morphology.erosion(new_label, morphology.disk(4))
boun = morphology.dilation(new_label) & (~morphology.erosion(new_label, morphology.disk(self.radius)))
new_label[boun > 0] = 2 # boundary
by the way,have you tested on the dataset 'multi-organ'
@thinkInJava33 erosion operation is used to your label. it means anno.img used for training is not the original imgs.
I didnt test on other dataset，my friends are doing this thing.
@Megumi-cs really need your help I thought I followed your suggestion but it didn't work. maybe I still get you wrong.would you mind telling me the change of your code in detail(the location and how to modify).
@thinkInJava33
Here is my best result now
on testA：
Metrics: F1 Dice Hausdorff
Average: 0.9099 0.9147 38.6177
on testeB：
Metrics: F1 Dice Hausdorff
Average: 0.8355 0.8521 92.1046
it is much better than before.
You need an eroding to your labels，because it actually should be the inside area of gland，however the original labels include both of inside area and the contour. You can read LabelEncoding function in my_transform.py for more about it.
Now, i‘m not sure the best radius of eroding operation. It needs some tries. Besides, min_areas in options.py can also improve the result, maybe 140 is better.
I think I understand what you mean. but in file test. py. code in maybe around line 150. the pred_label is postprocessed by dil ation(pred_labeled = morph.dilation(pred_labeled, selem=morph.selem.disk(opt.post['radius'])). so that the final pred_label include both inside area and the contour,therefore it is the same with the orginal label _anno.bmp.I don't think your problem exists.