Giters

Xinzhe99 / SwinMFF

Code for "SwinMFF: Pure Transformer for End-to-End Multi-focus Image Fusion"

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

image# SwinMFF Official code for "SwinMFF: Revitalizing and Setting a Benchmark for End-to-End Multi-Focus Image Fusion" Lytro: image MFFW: image MFI-WHU: image

Method EI Q^{ab/f} STD SF AVG MI EN VIF
DWT 70.7942 0.6850 57.2776 19.3342 6.8336 15.0872 7.5436 1.1114
DTCWT 70.5666 0.6929 57.2315 19.3204 6.8134 15.0791 7.5396 1.1079
NSCT 70.4289 0.6901 57.3601 19.2662 6.8027 15.0816 7.5408 1.1249
GFF 70.5179 0.6998 57.4451 19.2947 6.8058 15.0716 7.5358 1.1277
SR 70.2498 0.6944 57.3795 19.2819 6.7818 15.0650 7.5325 1.1208
ASR 70.3342 0.6951 57.3616 19.2818 6.7897 15.0654 7.5327 1.1201
MWGF 69.8052 0.7037 57.4136 19.1900 6.7273 15.0669 7.5334 1.1343
ICA 68.3180 0.6766 56.9383 18.5968 6.6125 15.0655 7.5327 1.0708
NSCT-SR 70.6705 0.6995 57.3924 19.3355 6.8213 15.0676 7.5338 1.1251
Proposed 72.4041 0.7321 57.9737 19.7954 6.9734 15.0826 7.5413 1.1810
Method EI Q^{ab/f} STD SF AVG MI EN VIF
SSSDI 70.7102 0.6966 57.4770 19.3567 6.8234 15.0668 7.5334 1.1309
QUADTREE 70.8957 0.7027 57.5334 19.4163 6.8412 15.0684 7.5342 1.1368
DSIFT 70.9808 0.7046 57.5319 19.4194 6.8493 15.0688 7.5344 1.1381
SRCF 71.0810 0.7036 57.5394 19.4460 6.8607 15.0690 7.5345 1.1374
GFDF 70.6258 0.7049 57.4973 19.3312 6.8145 15.0674 7.5337 1.1336
BRW 70.6777 0.7040 57.5020 19.3433 6.8200 15.0675 7.5337 1.1336
MISF 70.4148 0.6984 57.4437 19.2203 6.7945 15.0671 7.5335 1.1222
Proposed 72.4041 0.7321 57.9737 19.7954 6.9734 15.0826 7.5413 1.1810
Method Year Journal/Conference Network EI Q^{ab/f} STD SF AVG MI EN VIF
CNN 2017 Information Fusion CNN 70.3238 0.7019 57.4354 19.2295 6.7860 15.0663 7.5331 1.1255
ECNN 2019 Information fusion CNN 70.7432 0.7030 57.5089 19.3837 6.8261 15.0675 7.5338 1.1337
SESF 2020 Neural. Comput. Appl. CNN 70.9403 0.7031 57.5495 19.4158 6.8448 15.0696 7.5348 1.1395
MFIF-GAN 2021 SPIC GAN 71.0395 0.7029 57.5430 19.4370 6.8560 15.0690 7.5345 1.1393
MSFIN 2021 IEEE TIM CNN 71.0914 0.7045 57.5642 19.4438 6.8602 15.0695 7.5348 1.1420
ZMFF 2023 Information Fusion DIP 70.8298 0.6635 57.0347 18.9707 6.8045 15.0735 7.5368 1.1331
Proposed 2024 TBD Transformer 72.4041 0.7321 57.9737 19.7954 6.9734 15.0826 7.5413 1.1810
Method Year Journal/Conference Network EI Q^{ab/f} STD SF AVG MI EN VIF
IFCNN-MAX 2020 Information Fusion CNN 70.9193 0.6784 57.4896 19.3793 6.8463 15.0722 7.5361 1.1322
U2Fusion 2020 IEEE TPAMI CNN 59.8957 0.6190 51.9356 14.9334 5.6515 14.6153 7.3077 0.9882
SDNet 2021 IJCV CNN 60.3437 0.6441 55.2655 16.9252 5.8725 14.9332 7.4666 0.9281
MFF-GAN 2021 Information Fusion GAN 66.0601 0.6222 55.1920 18.4022 6.4089 14.8153 7.4076 1.0084
SwinFusion 2022 IEEE/CAA JAS CNN & Transformer 62.8130 0.6597 56.8142 16.6430 5.9862 15.0476 7.5238 1.0685
FusionDiff 2024 ESWA Diffusion Model 67.4911 0.6744 56.1372 18.8483 6.5325 14.9817 7.4909 1.0448
Proposed 2024 TBD Transformer 72.4041 0.7321 57.9737 19.7954 6.9734 15.0826 7.5413 1.1810

Train

Make dataset for training SwinMFF

  1. Download DUTS
  2. Extract it to the project path
  3. Run the following code to get the data set needed for training

python .\make_dataset.py --mode='TR'

python .\make_dataset.py --mode='TE'

Start to train

python .\train.py

Test

Download Weights in Baidu and put in the project path

Lytro

python .\predict.py --dataset_path='./assets/Lytro' --model_path='./checkpoint.ckpt' --is_gray=False

MFFW

python .\predict.py --dataset_path='./assets/MFFW' --model_path='./checkpoint.ckpt' --is_gray=False

MFI-WHU

python .\predict.py --dataset_path='./assets/MFI-WHU' --model_path='./checkpoint.ckpt' --is_gray=False

Others

python .\predict.py --dataset_path='your path' --model_path='your path' --is_gray=False/True

Results of various methods

Method Download link
CNN https://github.com/yuliu316316/CNN-Fusion
ECNN https://github.com/mostafaaminnaji/ECNN
SESF https://github.com/Keep-Passion/SESF-Fuse
MFIF-GAN https://github.com/ycwang-libra/MFIF-GAN
MSFIN https://github.com/yuliu316316/MSFIN-Fusion
ZMFF https://github.com/junjun-jiang/ZMFF
IFCNN-MAX https://github.com/uzeful/IFCNN
U2Fusion https://github.com/hanna-xu/U2Fusion
SDNet https://github.com/HaoZhang1018/SDNet
MFF-GAN https://github.com/HaoZhang1018/MFF-GAN
SwinFusion https://github.com/Linfeng-Tang/SwinFusion
FusionDiff https://github.com/lmn-ning/ImageFusion

Result of various learning-based methods compared can be download in Baidu

Includes traditional methods download in https://github.com/yuliu316316/MFIF

Acknowledgements

The research was supported by the Hainan Provincial Joint Project of Sanya Yazhou Bay Science and Technology City (No: 2021JJLH0079), Innovational Fund for Scientific and Technological Personnel of Hainan Province (NO. KJRC2023D19), and the Hainan Provincial Joint Project of Sanya Yazhou Bay Science and Technology City (No. 2021CXLH0020). Thanks for help by Hainan Provincial Observatory of Ecological Environment and Fishery Resource in Yazhou Bay. Also, we want to thank Chloe Alex Schaff for her contribution in polishing the article.

If our work is helpful to you, please help cite our work

About

Code for "SwinMFF: Pure Transformer for End-to-End Multi-focus Image Fusion"

Languages

Language:Python 100.0%