Giters

kazuto1011 / deeplab-pytorch

PyTorch re-implementation of DeepLab v2 on COCO-Stuff / PASCAL VOC datasets

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

only 20% mIou

ChunmengLiu1 opened this issue · comments

commented

Thanks for a great job!
I am a student who is following your job. All of my operations are based on README.md, however , I got very strange mIou, about 20%.
I download the trained model from Github and the test result is normal, about 76.5%. The results mean that the testing process was fine. I also changed the pre-trained model to resnet101 which is trained on ImageNet. The result always about 20%. Sadly, I couldn't find any problem during training.
My PyTorch version is 1.7. I want to know if the PyTorch version will affect my results. when I training, I just changed the IMAGE.SIZE.TRAIN=289,257 ....
The following result is that IMAGE.SIZE.TRAIN=289, GPU 1 3060, pre-trained model = deeplabv2_resnet101_msc-vocaug-20000.pth.

{
"Class IoU": {
"0": 0.7948632887014013,
"1": 0.3304257442805647,
"2": 0.11077134876825119,
"3": 0.07046620852376487,
"4": 0.10895940525580272,
"5": 0.04745116392110705,
"6": 0.31005641398703143,
"7": 0.2915453936066487,
"8": 0.23258285017371,
"9": 0.005347773696150551,
"10": 0.10445717208326169,
"11": 0.10978767179703398,
"12": 0.18345830368166063,
"13": 0.12426651067058993,
"14": 0.2462254036792113,
"15": 0.415199601472948,
"16": 0.050153233366977974,
"17": 0.15261392666663348,
"18": 0.02888480390809123,
"19": 0.27664375976635347,
"20": 0.13747916669502674
},
"Frequency Weighted IoU": 0.6421553231368009,
"Mean Accuracy": 0.2748884341598109,
"Mean IoU": 0.1967447211762962,
"Pixel Accuracy": 0.7687984741459604
}

If you could give me some advice, I will appreciate you very much! thank you!
Best wishes to you!

commented

OK, I will check my codes with the latest PyTorch.
I'm a bit confused about the weights you used for evaluation. Could you clarify the following points?

  • Trained by yourself?
  • How you used deeplabv2_resnet101_msc-vocaug-20000.pth?
commented

OK, I will check my codes with the latest PyTorch.
I'm a bit confused about the weights you used for evaluation. Could you clarify the following points?

  • Trained by yourself?
  • How you used deeplabv2_resnet101_msc-vocaug-20000.pth?

Thanks for your quick reply!
I trained by myself when I used 'deeplabv2_resnet101_msc-vocaug-20000.pth' as a pre-trained model. I used the trained model as the pre-trained model because I wanted to find the wrong place. Luckily, When I changed 'model. base.load_state_dict(state_dict, strict=False)' to 'model.load_state_dict(state_dict, strict=False)' in main.py, I solved the problem, and I got the normal results, about 77%.
Although I solved the problem, I don't know the reason. If you know the reason, can you explain it to me? Thank you very much!
Best wishes to you!

commented

It's because of the inconsistency of parameter names.
The parameter names of deeplabv2_resnet101_msc-vocaug-20000.pth begin with the term "base" because of libs.models.msc.MSC class. Therefore, model.base.load_state_dict(state_dict, strict=False) loads nothing (see with strict=True). model has "base" while model.base does not.

import torch
state_dict = torch.load("deeplabv2_resnet101_msc-vocaug-20000.pth")
print(*state_dict.keys(), sep="\n")
# base.layer1.conv1.conv.weight
# base.layer1.conv1.bn.weight
# ...

On the other hand, the default pre-trained model deeplabv1_resnet101-coco.pth does not contain "base", which is converted from the official Caffe weights.

commented