A Tri-Layer Plugin to Improve Occluded Detection

This is the official implementation of BMVC 2022 paper "A Tri-Layer Plugin to Improve Occluded Detection" by Guanqi Zhan, Weidi Xie, and Andrew Zisserman, including the novel automatically generated real-image evaluation dataset Occluded COCO and Separated COCO to monitor the capability to detect occluded objects.

[Update] Occluded COCO and Separated COCO have been supported by MMDetection as a standard benchmark for object detection under occlusion. Now you can from mmdet.datasets import OccludedSeparatedCocoDataset.

Installation

conda create --name occ python=3.7 -y
conda activate occ
conda install pytorch=1.10.0 torchvision torchaudio cudatoolkit=11.3 -c pytorch
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html
git clone https://github.com/Championchess/Tri-Layer_Plugin_Occluded_Detection.git
cd Tri-Layer_Plugin_Occluded_Detection
pip install -v -e .
pip install ipdb
pip install scikit-image

Data Preparation

Download the COCO dataset, Occluded COCO dataset , Separated COCO dataset, Occluder Mask dataset, Occludee Mask dataset, and then.

In tools/test.py and tools/test_ap.py and tools/test_recall_occluded_separated.py, fill in the COCO dataset path, Occluded/Separated COCO dataset path, and result save path
In configs/_base_/datasets/coco_instance_occluder_tri.py, fill in the COCO dataset path
In mmdet/datasets/coco_occluder_tri.py, fill in the path to generated occluder(front)/occludee(back) masks dataset

Model

Detector	Backbone	Plugin	Recall Occluded	Recall Separated	BBox mAP (val)	Mask mAP (val)	BBox mAP (test-dev)	Mask mAP (test-dev)	#params	FLOPs	config	model
Mask R-CNN	Swin-T	ours	3441(62.00%)	1223(34.72%)	48.5	43.0	48.7	43.4	77.6M	583.33G	config	ckpt
Mask R-CNN	Swin-S	ours	3473(62.58%)	1261(35.80%)	50.3	44.2	50.6	44.9	98.9M	673.32G	config	ckpt
Cascade Mask R-CNN	Swin-B	ours*	3532(63.64%)	1299(36.88%)	52.1	45.4	52.7	45.9	164.3M	1353.68G	config	ckpt

*Only Tri-Layer Modelling is applied as Cascade Mask R-CNN has already used multiple iterations.

Inference

python tools/test.py <CONFIG_FILE> "" --eval bbox segm

to get the detection results of models, and then

Test on COCO (bbox/mask mAP)

python tools/test_ap.py <CONFIG_FILE> "" --eval bbox segm

Test on Occluded COCO & Separated COCO (recall)

python tools/test_recall_occluded_separated.py

Citation

Please cite our paper if you use the code/model/dataset of this github.

@article{zhan2022triocc,
  title={A Tri-Layer Plugin to Improve Occluded Detection},
  author={Zhan, Guanqi and Xie, Weidi and Zisserman, Andrew},
  journal={British Machine Vision Conference},
  year={2022}
}

wanglg007 / A-Tri-Layer-Plugin-to-Improve-Occluded-Detection