Weakly-Supervised Image Semantic Segmentation Using Graph Convolutional Networks (ICME 2021)
An Official Pytorch Implementation of WSGCN-I. WSGCN-I is heavily based on [1] and [2].
Project Page: Link
Paper (arXiv): Link
Prerequisite
- Tested on Arch Linux, CUDA9.0, Python3.9, Pytorch 1.8.1, and NVIDIA GTX 1070; Tested on Ubuntu18.04, CUDA11.1, Python3.6, Pytorch 1.8.0, and NVIDIA Tesla V100
- Python dependencies (scipy, fire, torch, tensorboardX, pillow, torchvision, cython, tqdm, and pydensecrf...)
- PASCAL VOC 2012 datasets
- Pre-trained model for IRN
Download the VOC12 dataset
Download the VOC12 augmentation dataset
Download the pre-trained model for IRN
Download the pre-trained model for DeepLabV2
Setup data
Recommended directory structure
├── Data
│ ├── GCN4DeepLab
│ │ ├── Label
│ │ └── Logit
│ ├── IRN4GCN
│ │ ├── AFF_FEATURE
│ │ ├── AFF_MATRIX
│ │ ├── PARTIAL_PSEUDO_LABEL_DN
│ │ ├── PARTIAL_PSEUDO_LABEL_DN_UP
│ │ └── PARTIAL_PSEUDO_LABEL_UP
│ └── VOC12
│ ├── Split_List
│ └── VOC2012
│ ├── Annotations
│ ├── ImageSets
│ │ ├── Action
│ │ ├── Layout
│ │ ├── Main
│ │ └── Segmentation
│ ├── JPEGImages
│ ├── SegmentationClass
│ ├── SegmentationClassAug
│ └── SegmentationObject
├── GCN
│ └── runs
└── IRN
├── misc
├── net
├── result
│ ├── cam
│ ├── ins_seg
│ ├── ir_label
│ └── sem_seg
├── sess
├── step
└── voc12
└── Split_List
StageI (See train.sh for more details)
./train.sh
cd GCN/
python CRF.py
StageII (Please refer to another github)
Evaluation (See eval.py for more details)
python eval.py
Performance
Note that you may meet the performance fluctuation, which is about 0.5%, in these simplified codes for ordinary machines. This is because of the seed in train.py and -l in train.sh. Specifically, we set a seed for the train.py instead of resetting it for each GCN. For example, the performance of StageI is around 67.7% with -l 1464 in train.sh. In addition, the performance of StageII depends on StageI and the performance fluctuation is around 0.5%.
- Performance of StageI
| set | CRF | mIoU |
|---|---|---|
| train | X | 66.7% |
| train | O | 68.0% |
- Performance of StageII
| set | pre-train | mIoU |
|---|---|---|
| val | ImageNet | 66.7% |
| val | MSCOCO | 68.7% |
| test | ImageNet | 68.8% |
| test | MSCOCO | 69.3% |
Citation
If you find the code useful, please consider citing the paper.
@InProceedings{pan2021all,
author = {Shun-Yi Pan, Cheng-You Lu, Shih-Po Lee, and Wen-Hsiao Pen},
title = {Weakly-Supervised Image Semantic Segmentation Using Graph Convolutional Networks},
booktitle = {IEEE International Conference on Multimedia and Expo (ICME)},
year = {2021}
}