renjiawei-cr7 / ddpm-cd

Remote Sensing Change Detection using Denoising Diffusion Probabilistic Models

DDPM-CD: Remote Sensing Change Detection using Denoising Diffusion Probabilistic Models

Paper | Project

This is the offical Pytorch implementation of Remote Sensing Change Detection using Denoising Diffusion Probabilistic Models.

▶️Motivation

Images generated from the diffusion model trained on off-the-shelf remote sensing images. The generated images contain objects that we commonly see in real remote sensing images, such as buildings, trees, roads, vegetation, water surfaces, etc., demonstrating the powerful ability of the diffusion models to extract key semantics that can be further used in remote sensing change detection.

▶️Method

We fine-tune a light-weight change detection head which takes multi-level feature representations from the pre-trained diffusion model as inputs and outputs change prediction map.

▶️Environment

conda create -n ddpm-cd python=3.9

conda activate ddpm-cd

pip3 install -r requirement.txt

▶️Training diffusion model with remote sensing data

🔅Collect off-the-shelf remote sensing data to train diffusion model

Dump all the remote sensing data sampled from Google Earth Engine and any other publically available remote sensing images to dataset folder or create a simlink.

🔅Training/Resume unconditional diffusion model on remote sensing data

We use ddpm_train.json to setup the configurations. Update the dataset name and dataroot in the json file. Then run the following command to start training the diffusion model. The results and log files will be save to experiments folder. Also, we upload all the metrics to wandb.

python ddpm_train.py --config config/ddpm_train.json -enable_wandb -log_eval

In case, if you want to resume the training from previosely saved point, provide the path to saved model in path/resume_state, else keep it as null.

🔅Sampling from the diffusion model

If you want generate samples from the diffusion model, first update the path to trained diffusion model in [path][resume_state]. Then run the following command.

python ddpm_train.py --config config/ddpm_sampling.json --phase val

The generated images will be saved in experiments.

▶️Change Detection

🔅Download the datasets

Download the change detection datasets from the following links. Place them inside your datasets folder.

• LEVIR-CD
• WHU-CD
• DSIFN-CD
• CDD

Then, update the paths to those folders here [datasets][train][dataroot], [datasets][val][dataroot], [datasets][test][dataroot] in levir.json, whu.json, dsifn.json, and cdd.json.

Provide the path to pre-trained diffusion model

Udate the path to pre-trained diffusion model weights (*_gen.pth and *_opt.pth) here [path][resume_state] in levir.json, whu.json, dsifn.json, and cdd.json..

🔅Training the change detection network

Run the following code to start the training.

• Training on LEVIR-CD:

  python ddpm_cd.py --config config/levir.json -enable_wandb -log_eval

• Training on WHU-CD:

  python ddpm_cd.py --config config/whu.json -enable_wandb -log_eval

• Training on DSIFN-CD:

  python ddpm_cd.py --config config/dsifn.json -enable_wandb -log_eval

• Training on CDD:

  python ddpm_cd.py --config config/cdd.json -enable_wandb -log_eval

The results will be saved in experiments and also upload to wandb.

🔅Testing

To obtain the predictions and performance metrics (iou, f1, and OA), first provide the path to pre-trained diffusion model here [path][resume_state] and path to trained change detection model (the best model) here [path_cd][resume_state] in levir_test.json, whu_test.json, dsifn_test.json, and cdd_test.json.

Run the following code to start the training.

• Test on LEVIR-CD:

  python ddpm_cd.py --config config/levir_test.json --phase test -enable_wandb -log_eval

• Test on WHU-CD:

  python ddpm_cd.py --config config/whu_test.json --phase test -enable_wandb -log_eval

• Test on DSIFN-CD:

  python ddpm_cd.py --config config/dsifn_test.json --phase test -enable_wandb -log_eval

• Test on CDD:

  python ddpm_cd.py --config config/cdd_test.json --phase test -enable_wandb -log_eval

Predictions will be saved in experiments and performance metrics will be uploaded to wandb.

▶️Pre-trained models & Train/Val/Test logs

🔅Links to download pre-trained models

• Pre-trained diffusion model: Dropbox
• Pre-trained change detection networks:
  • "t": [50, 100]
    • LEVIR-CD Dropbox-cd-levir-50-100
    • WHU-CD Dropbox-cd-whu-50-100
    • DSIFN-CD Dropbox-cd-dsifn-50-100
    • CDD-CD Dropbox-cd-cdd-50-100
  • "t": [50, 100, 400] (Best Model)
    • LEVIR-CD Dropbox-cd-levir-50-100-400
    • WHU-CD Dropbox-cd-whu-50-100-400
    • DSIFN-CD Dropbox-cd-dsifn-50-100-400
    • CDD-CD Dropbox-cd-cdd-50-100-400
  • "t": [50, 100, 400, 650]
    • LEVIR-CD Dropbox-cd-levir-50-100-400-650
    • WHU-CD Dropbox-cd-whu-50-100-400-650
    • DSIFN-CD Dropbox-cd-dsifn-50-100-400-650
    • CDD-CD Dropbox-cd-cdd-50-100-400-650

💥 If you face a problem when downloading from the DropBox try one of the following options:

• [GoogleDrive] All pre-trained models in GooleDrive: GoogleDrive-pretrianed-models
• [GitHub] Pre-trained-models in GitHub

🔅Train/Val Reports on wandb

• LEVIR-CD-Train-Val-Reports-Wandb
• WHU-CD-Train-Val-reports-Wandb
• DSIFN-CD-Train-Val-Reports-Wandb
• CDD-CD-Train-Val-Reports-Wandb

🔅Test results on wandb

• LEVIR-WHU-DSIFN-CDD-Test-Results

▶️Results

🔅Quantitative

🔅Qualitative

• LEVIR-CD
• WHU-CD
• DSIFN-CD
• CDD

▶️Citation

@misc{https://doi.org/10.48550/arxiv.2206.11892,
  doi = {10.48550/ARXIV.2206.11892},
  
  url = {https://arxiv.org/abs/2206.11892},
  
  author = {Bandara, Wele Gedara Chaminda and Nair, Nithin Gopalakrishnan and Patel, Vishal M.},
  
  keywords = {Computer Vision and Pattern Recognition (cs.CV), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
  
  title = {Remote Sensing Change Detection (Segmentation) using Denoising Diffusion Probabilistic Models},
  
  publisher = {arXiv},
  
  year = {2022},
  
  copyright = {Creative Commons Attribution 4.0 International}
}

▶️References

• The code of diffusion model is from here.