kristinyanah / Mask2Former-IBS

Intra-Batch Supervision applied to Mask2Former

Mask2Former + Intra-Batch Supervision

[Project page] [Paper]

Code for 'Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images', Daan de Geus and Gijs Dubbelman, WACV 2023.

This code applies Intra-Batch Supervision to Mask2Former, and is built upon the official Mask2Former code.

Installation

See installation instructions.

Getting Started

• See Preparing Datasets for Mask2Former.
• See Getting Started with Mask2Former.
• To prepare the datasets for our crop sampling, run these two commands:
  • python mask2former/data/datasets/prepare_cityscapes_sampling.py
  • python mask2former/data/datasets/prepare_mapillary_sampling.py

Results

Results and models on Cityscapes.

Method	Crop sampling	Backbone	Iters	PQ	PQ_th	PQ_st	Acc_th	Prec_th	config	model
Mask2Former	no	R50	90k	62.1	55.2	67.2	87.1	93.3	config	TBD
Mask2Former + IBS	yes	R50	90k	62.4	55.7	67.3	87.6	94.1	config	TBD

Results and models on Mapillary Vistas.

Method	Crop sampling	Backbone	Iters	PQ	PQ_th	PQ_st	Acc_th	Prec_th	config	model
Mask2Former	no	R50	300k	41.5	33.3	52.4	71.7	78.8	config	TBD
Mask2Former + IBS	yes	R50	300k	42.2	34.9	52.0	75.7	84.1	config	TBD

License

Shield:

This code builds upon the official Mask2Former code. The majority of Mask2Former is licensed under a MIT License.

However portions of the project are available under separate license terms: Swin-Transformer-Semantic-Segmentation is licensed under the MIT license, Deformable-DETR is licensed under the Apache-2.0 License.

Citing us

Please consider citing our work if it is useful for your research.

@inproceedings{degeus2023ibs,
  title={Intra-Batch Supervision for Panoptic Segmentation on High-Resolution Images},
  author={{de Geus}, Daan and Dubbelman, Gijs},
  booktitle={IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year={2023}
}

If you use Mask2Former in your research or wish to refer to the baseline results published in the Model Zoo, please also refer to the original Mask2Former paper.

@inproceedings{cheng2022mask2former,
  title={Masked-attention Mask Transformer for Universal Image Segmentation},
  author={Bowen Cheng and Ishan Misra and Alexander G. Schwing and Alexander Kirillov and Rohit Girdhar},
  journal={CVPR},
  year={2022}
}

Acknowledgement

Code is largely based on Mask2Former, which is largely based on MaskFormer (https://github.com/facebookresearch/MaskFormer).