shiwt03 / MUSTER

A Multi-scale Transformer-based Decoder for Semantic Segmentation

MUSTER

MUSTER: A Multi-scale Transformer-based Decoder for Semantic Segmentation

The core codes can be found at:

MUSTER/mmseg/models/decode_heads/MUSTER_head.py

We use MMSegmentation v0.29.0 as the codebase.

Installation

For install , please refer to the guidelines in MMSegmentation v0.29.0.

An example (works for me): CUDA 11.6 and pytorch 1.12.1

Step 0. Install MMCV using MIM.

pip install -U openmim
mim install mmcv-full

Step 1. Install MUSTER.

# The workdir of the terminal should be MUSTER/
pip install -v -e .
# "-v" means verbose, or more output
# "-e" means installing a project in editable mode,
# thus any local modifications made to the code will take effect without reinstallation.

Dataset Preparation

For dataset preparation, please refer to the guidelines in this link.

It is recommended to symlink the dataset root to MUSTER/data. If your folder structure is different, you may need to change the corresponding paths in config files.

The fold structure is recommended to be:

MUSTER
├── data
│   ├── cityscapes
│   │   ├── leftImg8bit
│   │   │   ├── train
│   │   │   ├── val
│   │   ├── gtFine
│   │   │   ├── train
│   │   │   ├── val
│   ├── ade
│   │   ├── ADEChallengeData2016
│   │   │   ├── annotations
│   │   │   │   ├── training
│   │   │   │   ├── validation
│   │   │   ├── images
│   │   │   │   ├── training
│   │   │   │   ├── validation

Cityscapes

The data could be found here after registration.

By convention, **labelTrainIds.png are used for cityscapes training. MMsegmentation provided a script based on cityscapesscripts to generate **labelTrainIds.png.

# --nproc means 8 process for conversion, which could be omitted as well.
python tools/convert_datasets/cityscapes.py data/cityscapes --nproc 8

ADE20K

The training and validation set of ADE20K could be download from this link. You may also download test set from here.

Evaluation

Because of anonymity, we cannot give trained weights at present.

ADE20K

Example: evaluate MUSTER with Swin Transformer on ADE20K:

# Single-gpu testing
python tools/test.py configs/MUSTER/MUSTER_swin-base_ade20k_160k.py /path/to/checkpoint_file --show

Cityscapes

Example: evaluate MUSTER with Swin Transformer on Cityscapes:

# Single-gpu testing
python tools/test.py configs/MUSTER/MUSTER_swin-base_ade20k_160k.py /path/to/checkpoint_file --show

Training

Example: train MUSTER with Swin Transformer on ADE20K:

# Single-gpu training
python tools/train.py configs/MUSTER/MUSTER_swin-base_ade20k_160k.py

License

Please check the LICENSE file.

Citation

Please check CITATION.cff.