gaobb / DCFS

[NeurIPS 2022] Decoupling Classifier for Boosting Few-shot Object Detection and Instance Segmentation

Introduction

This repository is a official PyTorch implementation of Decoupling Classifier for Boosting Few-shot Object Detection and Instance Segmentation (NeurIPS 2022). This repo is created by Bin-Bin Gao.

Training and Evaluation

• To reproduce the FSOD/gFSOD results on COCO

  bash run_coco_fsod.sh r101 8 dcfs
  

• To reproduce the FSIS/gFSIS results on COCO

  bash run_coco_fsis.sh r101 8 dcfs 
  

• Model Weighs for Base Pre-Training

  datasets	Task	Model Weghts
  COCO-Base	Detection	model
  COCO-Base	Instance Segmentation	model
  VOC-Base1	Detection	model
  VOC-Base2	Detection	model
  VOC-Base3	Detection	model

Results on COCO Benchmark

• Few-shot Object Detection

  Method			mAPnovel
  Shot	1	2	3	5	10	30
  DeFRCN*	7.7	11.4	13.3	15.5	18.5	22.5
  DCFS	8.1	12.1	14.4	16.6	19.5	22.7

• Generalized Few-shot Object Detection

  Method			mAPnovel
  Shot	1	2	3	5	10	30
  DeFRCN	4.8	8.5	10.7	13.6	16.8	21.2
  DCFS	6.2	10.4	12.9	15.7	18.3	21.9

  Method			mAPBase
  Shot	1	2	3	5	10	30
  DeFRCN	30.4	31.4	32.1	32.6	34.0	34.8
  DCFS	34.4	34.7	34.9	35.0	35.7	35.8

• Few-shot Instance Segmentation

  Method			mAPnovel
  Shot	1	2	3	5	10	30
  Mask-DeFRCN	6.7	9.5	11.0	12.7	15.4	18.3
  DCFS	7.2	10.3	13.5	15.7	15.9	18.3

• Generalized Few-shot Instance Segmentation

  Method			mAPnovel
  Shot	1	2	3	5	10	30
  Mask-DeFRCN*	4.5	7.5	9.5	11.6	14.3	17.6
  DCFS	5.7	9.4	11.5	13.5	15.7	18.3

  Method			mAPbase
  Shot	1	2	3	5	10	30
  Mask-DeFRCN*	24.6	25.6	26.2	26.8	28.2	29.1
  DCFS	28.0	28.5	28.9	29.1	29.9	30.3

• Please refer to DeFRCN for data peparation and installation details.
• * indicates that the results are reproduced by us with the DeFRCN source code.
• The results of mAP^base and mAP^all for gFSOD and gFSIS can be seen at the Supplementary Material and ProjectPage.

Visualization Results

seed=1
shot=10
NET=r101

python3 tools/create_config.py \
        --dataset coco14      \
        --config_root configs/coco \
        --shot ${shot} \
        --seed ${seed} \
        --setting 'gfsod'
                
CONFIG_PATH=configs/coco/dcfs_gfsod_${NET}_novel_${shot}shot_seed${seed}.yaml
output_dir=./output/coco-${shot}shot-seed${seed}
python3 tools/visualize_data.py \
    --source annotation  \
    --config-file  $CONFIG_PATH  \
    --output-dir $output_dir


results_json='dcfs_gfsod_${NET}_novel/tfa-like-DC/${shot}shot_seed${seed}/only-inference/coco_instances_results.json'

python3 tools/visualize_results.py   \
        --input $results_json \
        --out ./output/coco14_${shot}shot_seed${seed}_vis_res  \
        --dataset  coco14_trainval_all_${shot}shot_seed${seed} 

The baseline DeFRCN may tend to incorrectly recognize positive object as background (middle two rows) due to the biased classification. This problem is greatly alleviated using our proposed method (DCFS).

Few-Shot Object Detection Leaderboard

• MSCOCO FSOD Leaderboard: [html]
• PASCAL-VOC FSOD Leaderboard: [html]

Citing

If you find this code useful in your research, please consider citing us:

@inproceedings{gao2022dcfs,
  title={Decoupling classifier for boosting few-shot object detection and instance segmentation},
  author={Gao, Bin-Bin and Chen, Xiaochen and Huang, Zhongyi and Nie, Congchong and Liu, Jun and Lai, Jinxiang and Jiang, Guannan and Wang, Xi and Wang, Chengjie},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS 2022)},
  pages={18640--18652},
  year={2022}
}

Acknowledgement

This repo is developed based on DeFRCN and Detectron2.

Discussion