SegmentationBLWX / sssegmentation

SSSegmentation: An Open Source Supervised Semantic Segmentation Toolbox Based on PyTorch.

Documents: https://sssegmentation.readthedocs.io/en/latest/

What's New

• 2023-12-20: Support EdgeSAM and SAMHQ, refer to inference-with-edgesam and inference-with-samhq for more details.
• 2023-10-25: Support ConvNeXtV2, refer to Results and Models for ConvNeXtV2 for more details.
• 2023-10-23: Support MobileViT and MobileViTV2, refer to Results and Models for MobileViT for more details.
• 2023-10-18: Support Mask2Former, refer to Results and Models for Mask2Former for more details.
• 2023-10-17: We release the source codes of IDRNet: Intervention-Driven Relation Network for Semantic Segmentation, which was accepted by NeurIPS 2023, refer to Results and Models for IDRNet for more details.
• 2023-10-15: Support MobileSAM, refer to inference-with-mobilesam for more details.
• 2023-09-27: Support SAM, refer to inference-with-sam for more details.

Introduction

SSSegmentation is an open source supervised semantic segmentation toolbox based on PyTorch. You can star this repository to keep track of the project if it's helpful for you, thank you for your support.

Major Features

• High Performance

  The performance of re-implemented segmentation algorithms is better than or comparable to other codebases.

• Modular Design and Unified Benchmark

  Various segmentation methods are unified into several specific modules. Benefiting from this design, SSSegmentation can integrate a great deal of popular and contemporary semantic segmentation frameworks and then, train and test them on unified benchmarks.

• Fewer Dependencies

  SSSegmenation tries its best to avoid introducing more dependencies when reproducing novel semantic segmentation approaches.

Benchmark and Model Zoo

Supported Backbones

Backbone	Model Zoo	Paper Link	Code Snippet
ConvNeXtV2	Click	CVPR 2023	Click
MobileViTV2	Click	ArXiv 2022	Click
ConvNeXt	Click	CVPR 2022	Click
MAE	Click	CVPR 2022	Click
MobileViT	Click	ICLR 2022	Click
BEiT	Click	ICLR 2022	Click
Twins	Click	NeurIPS 2021	Click
SwinTransformer	Click	ICCV 2021	Click
VisionTransformer	Click	IClR 2021	Click
BiSeNetV2	Click	IJCV 2021	Click
ResNeSt	Click	ArXiv 2020	Click
CGNet	Click	TIP 2020	Click
HRNet	Click	CVPR 2019	Click
MobileNetV3	Click	ICCV 2019	Click
FastSCNN	Click	ArXiv 2019	Click
BiSeNetV1	Click	ECCV 2018	Click
MobileNetV2	Click	CVPR 2018	Click
ERFNet	Click	T-ITS 2017	Click
ResNet	Click	CVPR 2016	Click
UNet	Click	MICCAI 2015	Click

Supported Segmentors

Segmentor	Model Zoo	Paper Link	Code Snippet
EdgeSAM	Click	ArXiv 2023	Click
IDRNet	Click	NeurIPS 2023	Click
MobileSAM	Click	ArXiv 2023	Click
SAMHQ	Click	NeurIPS 2023	Click
SAM	Click	ArXiv 2023	Click
MCIBI++	Click	TPAMI 2022	Click
Mask2Former	Click	CVPR 2022	Click
ISNet	Click	ICCV 2021	Click
MCIBI	Click	ICCV 2021	Click
MaskFormer	Click	NeurIPS 2021	Click
Segformer	Click	NeurIPS 2021	Click
SETR	Click	CVPR 2021	Click
ISANet	Click	IJCV 2021	Click
DNLNet	Click	ECCV 2020	Click
PointRend	Click	CVPR 2020	Click
OCRNet	Click	ECCV 2020	Click
GCNet	Click	TPAMI 2020	Click
APCNet	Click	CVPR 2019	Click
DMNet	Click	ICCV 2019	Click
ANNNet	Click	ICCV 2019	Click
EMANet	Click	ICCV 2019	Click
FastFCN	Click	ArXiv 2019	Click
SemanticFPN	Click	CVPR 2019	Click
CCNet	Click	ICCV 2019	Click
CE2P	Click	AAAI 2019	Click
DANet	Click	CVPR 2019	Click
PSANet	Click	ECCV 2018	Click
UPerNet	Click	ECCV 2018	Click
EncNet	Click	CVPR 2018	Click
Deeplabv3Plus	Click	ECCV 2018	Click
NonLocalNet	Click	CVPR 2018	Click
ICNet	Click	ECCV 2018	Click
Mixed Precision (FP16) Training	Click	ArXiv 2017	Click
Deeplabv3	Click	ArXiv 2017	Click
PSPNet	Click	CVPR 2017	Click
FCN	Click	TPAMI 2017	Click

Supported Datasets

Dataset	Project Link	Paper Link	Code Snippet	Download Script
VSPW	Click	CVPR 2021	Click	CMD bash scripts/prepare_datasets.sh vspw
Supervisely	Click	Website Release 2020	Click	CMD bash scripts/prepare_datasets.sh supervisely
Dark Zurich	Click	ICCV 2019	Click	CMD bash scripts/prepare_datasets.sh darkzurich
Nighttime Driving	Click	ITSC 2018	Click	CMD bash scripts/prepare_datasets.sh nighttimedriving
CIHP	Click	ECCV 2018	Click	CMD bash scripts/prepare_datasets.sh cihp
COCOStuff10k	Click	CVPR 2018	Click	CMD bash scripts/prepare_datasets.sh cocostuff10k
COCOStuff164k	Click	CVPR 2018	Click	CMD bash scripts/prepare_datasets.sh coco
MHPv1&v2	Click	ArXiv 2017	Click	CMD bash scripts/prepare_datasets.sh mhpv1 & bash scripts/prepare_datasets.sh mhpv2
LIP	Click	CVPR 2017	Click	CMD bash scripts/prepare_datasets.sh lip
ADE20k	Click	CVPR 2017	Click	CMD bash scripts/prepare_datasets.sh ade20k
SBUShadow	Click	ECCV 2016	Click	CMD bash scripts/prepare_datasets.sh sbushadow
CityScapes	Click	CVPR 2016	Click	CMD bash scripts/prepare_datasets.sh cityscapes
ATR	Click	ICCV 2015	Click	CMD bash scripts/prepare_datasets.sh atr
Pascal Context	Click	CVPR 2014	Click	CMD bash scripts/prepare_datasets.sh pascalcontext
MS COCO	Click	ECCV 2014	Click	CMD bash scripts/prepare_datasets.sh coco
HRF	Click	Int J Biomed Sci 2013	Click	CMD bash scripts/prepare_datasets.sh hrf
CHASE DB1	Click	TBME 2012	Click	CMD bash scripts/prepare_datasets.sh chase_db1
PASCAL VOC	Click	IJCV 2010	Click	CMD bash scripts/prepare_datasets.sh pascalvoc
DRIVE	Click	TMI 2004	Click	CMD bash scripts/prepare_datasets.sh drive
STARE	Click	TMI 2000	Click	CMD bash scripts/prepare_datasets.sh stare

Citation

If you use SSSegmentation in your research, please consider citing this project,

@article{jin2023sssegmenation,
    title={SSSegmenation: An Open Source Supervised Semantic Segmentation Toolbox Based on PyTorch},
    author={Jin, Zhenchao},
    journal={arXiv preprint arXiv:2305.17091},
    year={2023}
}

@inproceedings{jin2021isnet,
    title={ISNet: Integrate Image-Level and Semantic-Level Context for Semantic Segmentation},
    author={Jin, Zhenchao and Liu, Bin and Chu, Qi and Yu, Nenghai},
    booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
    pages={7189--7198},
    year={2021}
}

@inproceedings{jin2021mining,
    title={Mining Contextual Information Beyond Image for Semantic Segmentation},
    author={Jin, Zhenchao and Gong, Tao and Yu, Dongdong and Chu, Qi and Wang, Jian and Wang, Changhu and Shao, Jie},
    booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
    pages={7231--7241},
    year={2021}
}

@article{jin2022mcibi++,
    title={MCIBI++: Soft Mining Contextual Information Beyond Image for Semantic Segmentation},
    author={Jin, Zhenchao and Yu, Dongdong and Yuan, Zehuan and Yu, Lequan},
    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
    year={2022},
    publisher={IEEE}
}

@inproceedings{jin2023idrnet,
    title={IDRNet: Intervention-Driven Relation Network for Semantic Segmentation},
    author={Jin, Zhenchao and Hu, Xiaowei and Zhu, Lingting and Song, Luchuan and Yuan, Li and Yu, Lequan},
    booktitle={Thirty-Seventh Conference on Neural Information Processing Systems},
    year={2023}
}

References

We are very grateful to the following projects for their help in building SSSegmentation,

• MMSegmentation
• segment-anything
• MobileSAM
• Mask2Former
• Swin-Transformer
• HRNet-Semantic-Segmentation
• apex
• MMCV
• VSPW_code
• MMPreTrain
• PyTorch Image Models
• EdgeSAM
• sam-hq