English | 简体中文
The repository for this project is the code implementation of the paper RS-Mamba for Large Remote Sensing Image Dense Prediction, developed based on the environment of the VMamba.
If you find this project helpful, please give us a star ⭐️.
Main Contribution
- For the first time, the state space model (SSM) is introduced to dense prediction tasks in remote sensing, where our Recurrent State Space Model (RSM) maintains a global effective receptive field with linear complexity. ·
- Reflecting the spatial feature distribution across multiple directions in remote sensing images, RSM employs selective scanning over multiple directions. ·
- Experiments on semantic segmentation and change detection tasks has demonstrated that RSM can achieve state-of-the-art (SOTA) performance even with the simplest model architecture and training methodologies.
🌟 2024.03.29 Released the RS-Mamba project
🌟 2024.03.29 Released the code of models and training framwork
- Open source the model code
- Open source the training code
- Introduction
- Updates
- TODO
- Table of Contents
- Description of Folders and Files
- Installation
- Dataset Preparation
- Model Training and Inference
- FAQ
- Citation
- License
The folders semantci_segmentation_mamba and change_detection_mamba respectively contain the code for remote sensing semantic segmentation and change detection tasks, organized in a similar manner.
Taking the change_detection_mamba folder as an example, train.py and inference.py are the training and inference scripts respectively, rs_mamba_cd.py contains the model code, and the utils folder stores various other code files.
Within the utils folder, data_loading.py is the code file for data loading, dataset_process.py is the dataset processing file containing various functions for preprocessing change detection datasets, losses.py contains the code for loss functions, path_hyperparameter.py stores various model and training hyperparameters, dataset names, and model hyperparameters are also set therein, and utils.py includes code for training and validation.
Step 1: Follow the installation instructions of the VMamba project to install the "rs_mamba" environment.
Step 2: Run the following command to install dependencies.
If you only use the model code, this step is not needed.
pip install -r requirements.txt- Dataset Download: Massachusetts Roads Dataset。
- Dataset Download: WHU Dataset。
You need to organize the data set into the following format:
${DATASET_ROOT} # dataset root dir
├── train
├── image
└── 0001.tif
└── 0002.tif
└── ...
├── label
└── 0001.tif
└── 0002.tif
└── ...
├── val
├── image
└──0001.tif
└── 0002.tif
└── ...
├── label
├── 0001.tif
└── 0002.tif
└── ...
├── test
├── image
└── 0001.tif
└── 0002.tif
└── ...
├── label
└── 0001.tif
└── 0002.tif
└── ...
- Dataset Download: WHU-CD Dataset。
- Dataset Download: LEVIR-CD Dataset。
You need to organize the data set into the following format:
${DATASET_ROOT} # dataset root dir
├── train
├── t1
└── 0001.tif
└── 0002.tif
└── ...
├── t2
└── 0001.tif
└── 0002.tif
└── ...
├── label
└── 0001.tif
└── 0002.tif
└── ...
├── val
├── t1
└── 0001.tif
└── 0002.tif
└── ...
├── t2
└── 0001.tif
└── 0002.tif
└── ...
├── label
├── 0001.tif
└── 0002.tif
└── ...
├── test
├── t1
└── 0001.tif
└── 0002.tif
└── ...
├── t2
└── 0001.tif
└── 0002.tif
└── ...
├── label
└── 0001.tif
└── 0002.tif
└── ...
All hyperparameters for model training and inference are located in the utils/path_hyperparameter.py file, with corresponding comments for explanation.
Firstly, navigate to the semantic segmentation folder by running the following command in the command line:
cd semantic_segmentation_mamba
To start training, run the following code in the command line:
python train.py
If you want to debug during training, run the following command in the command line:
python -m ipdb train.py
To perform testing or inference in the command line, run the following code:
python inference.py
Firstly, navigate to the change detection folder by running the following command in the command line:
cd change_detection_mamba
To start training, run the following code in the command line:
python train.py
If you want to debug during training, run the following command in the command line:
python -m ipdb train.py
To perform testing or inference in the command line, run the following code:
python inference.py
Here we have listed some common problems in use and their corresponding solutions, and if there are frequently recurring problems in the issue, they will also be listed here.
You can refer to issue102 and issue95 of VMamba, My approach is to ask GPT4, and get a feasible solution, using conda to update GCC, relevant questions and answers are here.
You can refer to issue55, selective_scan_cuda is dispensable.
If you are using this project's code or performance benchmarks in your research, please refer to bibtex to reference RSMamba below.
@misc{zhao2024rsmamba,
title={RS-Mamba for Large Remote Sensing Image Dense Prediction},
author={Sijie Zhao and Hao Chen and Xueliang Zhang and Pengfeng Xiao and Lei Bai and Wanli Ouyang},
year={2024},
eprint={2404.02668},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
This project is licensed under the Apache 2.0 License。