Chen-Yang-Liu / RSCaMa

RSCaMa: Remote Sensing Image Change Captioning with State Space Model

RSCaMa: Remote Sensing Image Change Captioning with State Space Model

ArXiv

The code will be made public in the next few days.

Share us a ⭐ if you're interested in this repo

Welcome to our repository!

This repository contains the PyTorch implementation of "RSCaMa: Remote Sensing Image Change Captioning with State Space Model".

Installation and Dependencies

git clone https://github.com/Chen-Yang-Liu/RSCaMa.git
cd RSCaMa
conda create -n RSCaMa_env python=3.9
conda activate RSCaMa_env
pip install -r requirements.txt

Data Preparation

• Download the LEVIR_CC dataset: LEVIR-CC .
• The data structure of LEVIR-CC is organized as follows:

├─/root/Data/LEVIR_CC/
        ├─LevirCCcaptions.json
        ├─images
             ├─train
             │  ├─A
             │  ├─B
             ├─val
             │  ├─A
             │  ├─B
             ├─test
             │  ├─A
             │  ├─B

where folder A contains images of pre-phase, folder B contains images of post-phase.

• Extract text files for the change descriptions of each image pair in LEVIR-CC:

python preprocess_data.py --input_captions_json /DATA_PATH/Levir-CC-dataset/LevirCCcaptions.json

!NOTE: When preparing the text token files, we suggest setting the word count threshold of LEVIR-CC to 5 and Dubai_CC to 0 for fair comparisons.

Training

python train_CC.py --data_folder /DATA_PATH/Levir-CC-dataset/images

!NOTE: If the program encounters the error: "'Meteor' object has no attribute 'lock'," we recommend installing it with sudo apt install openjdk-11-jdk to resolve this issue.

Evaluate

python test.py --data_folder /DATA_PATH/Levir-CC-dataset/images --checkpoint xxxx.pth

Alternatively, you can download our pretrained model here: [Hugging face].

Experiment:

Citation:

@misc{liu2024rscama,
      title={RSCaMa: Remote Sensing Image Change Captioning with State Space Model}, 
      author={Chenyang Liu and Keyan Chen and Bowen Chen and Haotian Zhang and Zhengxia Zou and Zhenwei Shi},
      year={2024},
      eprint={2404.18895},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}