RogerQi / SSP

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Self-Support Few-Shot Semantic Segmentation

Qi Fan, Wenjie Pei, Yu-Wing Tai, Chi-Keung Tang

The codebase contains the official code of our paper Self-Support Few-Shot Semantic Segmentation, ECCV 2022.

中文解读: ECCV 2022 | SSP: 自支持匹配的小样本任务新**

PWC
PWC
PWC
PWC
PWC
PWC

Data preparation

Download

Pretrained model: ResNet-50 | ResNet-101

Dataset: Pascal images and ids | Semantic segmentation annotations

File Organization

You can follow HSNet to prepare the MS COCO and FSS-1000 datasets.

    ../                         # parent directory
    ├── ./SSP                   # current (project) directory
    |   ├── codes               # various codes
    |   └── ./pretrained        # pretrained model directory
    |            ├── resnet50.pth
    |            └── resnet101.pth
    └── Datasets_SSP/
        ├── VOC2012/            # PASCAL VOC2012 devkit
        │   ├── Annotations/
        │   ├── ImageSets/
        │   ├── ...
        │   └── SegmentationClassAug/
        ├── COCO2014/           
        │   ├── annotations/
        │   │   ├── train2014/  # (dir.) training masks (from Google Drive) 
        │   │   ├── val2014/    # (dir.) validation masks (from Google Drive)
        │   │   └── ..some json files..
        │   ├── train2014/
        │   └── val2014/
        └── FSS-1000/           # (dir.) contains 1000 object classes
            ├── abacus/   
            ├── ...
            └── zucchini/

Run the code

You can adapt the scripts of all.sh, test.sh and hsnet_test.sh (for the HSNet evaluation protocol) to train and evaluate your models.

CUDA_VISIBLE_DEVICES=0,1 python -W ignore main.py \
  --dataset pascal --data-root [Your Pascal Path] \
  --backbone resnet50 --fold 0 --shot 1

You may change the backbone from resnet50 to resnet101, change the fold from 0 to 1/2/3, or change the shot from 1 to 5 for other settings.

You can add/remove --refine to enable/disable the self-support refinement.

Performance and Trained Models

Pascal VOC

Method Setting Backbone SSP Refine Fold 0 Fold 1 Fold 2 Fold 3 Mean
Baseline 1-shot ResNet-50 No 54.9 66.5 61.7 48.3 57.9
Baseline 1-shot ResNet-101 No 57.2 68.5 61.3 53.3 60.1
Baseline 5-shot ResNet-50 No 61.6 70.3 70.5 56.4 64.7
Baseline 5-shot ResNet-101 No 64.2 74.0 71.5 61.3 67.8
SSP (Ours) 1-shot ResNet-50 Yes 61.4 67.8 66.5 50.9 61.7
SSP (Ours) 1-shot ResNet-101 Yes 63.2 70.4 68.5 56.3 64.6
SSP (Ours) 5-shot ResNet-50 Yes 67.5 72.3 75.2 62.1 69.3
SSP (Ours) 5-shot ResNet-101 Yes 70.9 77.1 78.9 66.1 73.3

MS COCO

Method Setting Backbone Eval Protocol Fold 0 Fold 1 Fold 2 Fold 3 Mean
SSP (Ours) 1-shot ResNet-50 Ours 46.4 35.2 27.3 25.4 33.6
SSP (Ours) 1-shot ResNet-101 Ours 50.4 39.9 30.6 30.0 37.7
SSP (Ours) 5-shot ResNet-50 Ours 53.9 42.0 36.0 33.7 41.4
SSP (Ours) 5-shot ResNet-101 Ours 57.8 47.0 40.2 39.9 46.2
SSP (Ours) 1-shot ResNet-50 HSNet 35.5 39.6 37.9 36.7 37.4
SSP (Ours) 1-shot ResNet-101 HSNet 39.1 45.1 42.7 41.2 42.0
SSP (Ours) 5-shot ResNet-50 HSNet 40.6 47.0 45.1 43.9 44.1
SSP (Ours) 5-shot ResNet-101 HSNet 47.4 54.5 50.4 49.6 50.2

Acknowledgement

This codebase is built based on MLC's baseline code and we borrow HSNet's evaluation protocol for the MS COCO dataset. We thank MLC and other FSS works for their great contributions.

Other related repos

Few-shot image/video object detection: FewX

Other related papers

@inproceedings{fan2021fsvod,
  title={Few-Shot Video Object Detection},
  author={Fan, Qi and Tang, Chi-Keung and Tai, Yu-Wing},
  booktitle={ECCV},
  year={2022}
}
@inproceedings{fan2020cpmask,
  title={Commonality-Parsing Network across Shape and Appearance for Partially Supervised Instance Segmentation},
  author={Fan, Qi and Ke, Lei and Pei, Wenjie and Tang, Chi-Keung and Tai, Yu-Wing},
  booktitle={ECCV},
  year={2020}
}
@inproceedings{fan2020fsod,
  title={Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector},
  author={Fan, Qi and Zhuo, Wei and Tang, Chi-Keung and Tai, Yu-Wing},
  booktitle={CVPR},
  year={2020}
}

Citation

@inproceedings{fan2022ssp,
  title={Self-Support Few-Shot Semantic Segmentation},
  author={Fan, Qi and Pei, Wenjie and Tai, Yu-Wing and Tang, Chi-Keung},
  journal={ECCV},
  year={2022}
}

About

License:MIT License


Languages

Language:Python 97.6%Language:Shell 2.4%