xingchenzhang / MFIF

Multi-focus image fusion: papers, datasets and results

This is the repository corresponding to the paper "Deep Learning-based Multi-focus Image Fusion: A Survey and A Comparative Study" [Paper] published on Transactions on Pattern Analysis and Machine Intelligence. The author is from Imperial College London, UK.

In the paper, 35 MFIF methods have been evaluated using 19 metrics on 63 image pairs.

Abstract

Multi-focus image fusion (MFIF) is an important area in image processing. Since 2017, deep learning has been introduced to the field of MFIF and various methods have been proposed. However, there is a lack of survey papers that discuss deep learning-based MFIF methods in detail. In this study, we fill this gap by giving a detailed survey on deep learning-based MFIF algorithms, including categories, methods, datasets and evaluation metrics. To the best of our knowledge, this is the first survey paper which focuses on deep learning based approaches in the field of MFIF. Besides, extensive experiments have been conducted to compare the performances of deep learning-based MFIF algorithms with conventional MFIF approaches. By analyzing qualitative and quantitative results, we give some observations on the current status of MFIF and discuss some future prospects of this field.

Source images

In total, 63 image pairs from three datasets, i.e. Lytro [1], MFFW [2], MFI-WHU [3] are utilized.

Fused images

2205 images generated using 35 MFIF algorithms are provided.

Download links of the 35 MFIF methods

• ASR
• BFMF
• BGSC
• CBF
• CNN
• CSR
• DCT_Corr
• DCT_EOL
• DSIFT
• DRPL
• DWTDE
• ECNN
• FusionDN
• GCF
• GD
• GFDF
• GFF
• IFCNN
• IFM
• MADCNN
• MFF-GAN
• MFGM
• MFM
• MGFF
• LP_SR
• MWGF
• NSCT_SR
• PCANet
• PMGI
• RP_SR
• QB
• SESF
• SFMD
• SVDDCT
• TF
• U2Fusion

Citation

If you find this work useful, please cite this paper:

@article{zhang2021deep,
  title={Deep Learning-based Multi-focus Image Fusion: A Survey and A Comparative Study},
  author={Zhang, Xingchen},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2021},
  publisher={IEEE}
}

References

[1] M. Nejati, S. Samavi and S. Shirani. Multi-focus image fusion using dictionary-based sparse representation. Information Fusion, vol. 25, pp. 72-84, 2015.

[2] S. Xu, X. Wei, C. Zhang, J. Liu and J. Zhang. MFFW: A new dataset for multi-focus imgae fusion. arXiv, 2020.

[3] H. Zhang, Z. Le, Z. Shao, H. Xu and J. Ma. MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion. Information Fusion, vol. 66, pp. 40-53, 2020.