Giters

xingchenzhang / Visible-infrared-image-fusion-based-on-deep-learning

Deep learning-based methods for visible and infrared image fusion

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Deep learning-based visible and infrared image fusion: papers & codes

This is the repository corresponding to our TPAMI2023 paper "Visible and Infrared Image Fusion Using Deep Learning".

A list of papers, datasets (benchmarks) and codes in deep learning-based visible and infrared image fusion methods.

  • If you think this is useful, please consider citing our paper and giving a star, thanks!
  • If you think some papers are missing and you want to add, please feel free to raise an issue or contact me.
  • Contact detail: xingchen.zhang@imperial.ac.uk

Table of Contents

  1. Timeline

  2. Visible-infrared image fusion (VIF)

  3. General image fusion

Timeline

Visible-infrared image fusion

Review

  1. Changqi Sun, Cong Zhang, Naixue Xiong.
    Infrared and Visible Image Fusion Techniques Based on Deep Learning: A Review, Electronics, 2020.

  2. Jiayi Ma, Yong Ma, Chang Li.
    Infrared and visible image fusion methods and applications: A survey, Information Fusion, vol.45, pp.153-178,2019.

  3. Ami Patel, Jayesh Chaudhary.
    A Review on Infrared and Visible Image Fusion Techniques, ICICV, 2019.

Benchmark

  1. Xingchen Zhang, Ping Ye, Gang Xiao.
    VIFB: a visible and infrared image fusion benchmark, CVPRW, 2020. [Code]

2023

2022

Journal

  1. Chang Liu, Bin Yang, Xiaozhi Zhang, Lihui Pang.
    IBPNet: a multi-resolution and multi-modal image fusion networkvia iterative back-projection, Applied Intelligence, 2022.

  2. Wanxin Xiao , Yafei Zhang , Hongbin Wang ,Fan Li ,and Hua Jin.
    Heterogeneous Knowledge Distillation for Simultaneous Infrared-Visible Image Fusion and Super-Resolution, IEEE Transactions on Instrumentation and Measurement, 2022.

  3. Hongmei Wang, Wenbo An, Lin Li, Chenkai Li, Daming Zhou. Infrared and visible image fusion based on multi-channel convolutional neural network, IET Image Processing, 2022.

  4. Jingjing Wang, Jinwen Ren, Hongzhen Li, Zengzhao Sun, Zhenye Luan, Zishu Yu, Chunhao Liang, Yashar E. Monfared, Huaqiang Xu and Qing Hua.
    DDGANSE: Dual-Discriminator GAN with a Squeeze-and-Excitation Module for Infrared and Visible Image Fusion, photonics, 2022.

  5. Zhiguang Yang and Shan Zeng.
    TPFusion: Texture Preserving Fusion of Infrared and Visible Images via Dense Networks, Entropy, 2022.

  6. Zhisheng Gao, Qiaolu Wang, Chenglin Zuo.
    A total variation global optimization framework and its application on infrared and visible image fusion, Signal, Image and Video Processing, 2022. [Code]

  7. Zhishe Wang, Yuanyuan Wu, Junyao Wang, Jiawei Xu, Wenyu Shao.
    Res2Fusion: Infrared and visible image fusion based on dense Res2net and double non-local attention models, IEEE Transactions on Instrumentation and Measurement, 2022.

  8. Xin Yang, Hongtao Huo, Jing Li, Chang Li, Zhao Liu, Xun Chen.
    DSG-Fusion: Infrared and visible image fusion via generative adversarial networks and guided filter, Expert Systems With Applications, 2022.

  9. Liming Zhang, Heng Li, Rui Zhu, Ping Du.
    An infrared and visible image fusion algorithm based on ResNet-152, Multimedia Tools and Applications, 2022.

  10. Xianglong Chen, Haipeng Wang, Yaohui Liang, Ying Meng, Shifeng Wang.
    A Novel Infrared and Visible Image Fusion Approach Based on Adversarial Neural Network, Sensors, 2022.

Conference

arXiv

  1. Dongyu Rao, Xiao-Jun Wu, Tianyang Xu
    TGFuse: An Infrared and Visible Image Fusion Approach Based on Transformer and Generative Adversarial Network, 2022.

  2. Zhancheng Zhang, Yuanhao Gao, Mengyu Xiong, Xiaoqing Luo and Xiao-Jun Wu.
    A Joint Convolution Auto-encoder Network for Infrared and Visible Image Fusion, 2022.

  3. Zhishe Wang, Wenyu Shao, Yanlin Chen, Jiawei Xu, Xiaoqin Zhang.
    Infrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning, 2022.

2021

Journal

  1. Kan Ren, Dawei Zhang, Minjie Wan, Xin Miao, Guohua Gu, Qian Chen.
    An infrared and visible image fusion method based on improved DenseNet and mRMR-ZCA, Infrared Physics and Technology, Vol. 115, 2021. [Paper]

  2. Hui Li, Xiao-Jun Wu, Josef Kittler.
    RFN-Nest: An end-to-end residual fusion network for infrared and visible images, Information Fusion, 2021. [Pytorch]

  3. Yu Fu, Xiao-Jun We, Tariq Durrani.
    Image fusion based on generative adversarial network consistent with perception, Information Fusion, 2021.

  4. Yongzhi Long, Haitao Jia, Yida Zhong, Yadong Jiang, Yuming Jia.
    RXDNFuse: A aggregated residual dense network for infrared and visible image fusion, Information Fusion, Vol. 69, 2021.

  5. Lihua Jian, Xiaomin Yang, Zheng Liu, Gwanggil Jeon, Mingliang Gao, David Chisholm.
    SEDRFuse: A Symmetric Encoder-Decoder With Residual Block Network for Infrared and Visible Image Fusion, IEEE Transactions on Instrumentation and Measurement, Vol. 70, 2021.

  6. Han Xu, Xinya Wang, Jiayi Ma.
    DRF: Disentangled Representation for Visible and Infrared Image Fusion, IEEE Transactions on Instrumentation and Measurement, Vol. 70, 2021.

  7. Jiayi Ma, Hao Zhang, Zhenfeng Shao, Pengwei Liang, Han Xu.
    GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion, IEEE Transactions on Instrumentation and Measurement, Vol. 70, 2021. [Tensorflow]

  8. Yong Yang, Jiaxiang Liu, Shuying Huang, Weiguo Wan, Wenying Wen, Juwei Guan.
    Infared and Visible Image Fusion via Texture Conditional Generative Adversarial Network, IEEE Transactions on Circuits and Systems for Video Technology, 2021.

  9. Jinyuan Liu, Xin Fan, Ji Jiang, Risheng Liu, Zhongxuan Luo.
    Learning a Deep Multi-scale Feature Ensemble and an Edge-attention Guidance for Image Fusion, IEEE Transactions on Circuits and Systems for Video Technology, 2021. [No code]

  10. Asif Raza, Jingdong Liu, Yifan Liu, Zeng Li, Jian Liu, Xi Chen, Hong Huo, Tao Fang.
    IR-MSDNet: Infrared and Visible Image Fusion based on Infrared Features & Multiscale Dense Network, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sesing, 2021.

  11. Zhao Xu, Gang Liu, Lili Tang, Yanhui Li.
    Blur Regional Features Based Infrared and Visible Image Fusion Using An Improved C3Net Model, Journal of Physics: Conference Series, 2021.

  12. Yue Pan, Dechang Pi, Lzhar Ahmed Khan, Zaheer Ullah Khan, Junfu Chen, Han Meng.
    DenseNetFuse: a study of deep unsupervised DenseNet to infrared and visual image fusion, Journal of Ambient Intelligent and Humanized Computing2021.

  13. Yansong Gu, Xinya Wang, Can Zhang, Baiyang Li.
    Advanced Driving Assistance Based on the Fusion of Infrared and Visible Images, Entropy, 2021.

  14. Jilei Hou, Dazhi Zhang, Wei Wu, Jiayi Ma, Huabing Zhou.
    A Generative Adversarial Network for Infrared and Visible Image Fusion Based on Semantic Segmentation, Entropy, 2021.

  15. Heena Patel, Kishor P. Upla.
    DepthFuseNet: an approach for fusion of thermal and visible images usings a convolution neural network, Optical Engineering, 2021.

  16. Yong Yang, Jia-Xiang Liu, Shu-Ying Huang, Hang-Yang Lu, Wen-Ying Wen.
    VMDM-fusion: a saliency feature representation method for infrared and visible image fusion, Signal, Image and Video Processing, 2021.

  17. Qilei Li, Lu Lu, Zhen Li, Wei Wu, Zheng Liu, Gwanggil Jeon, Xiaomin Yang.
    Coupled GAN with relativistic discriminators for infrared and visible images fusion, IEEE Sensors Journal, Vol. 21, No. 6, 2021.

  18. Huabing Hua, Jilei Hou, Wei Wu, Yanduo Zhang, Yuntao Wu, Jiayi Wu.
    Infrared and Visible Image Fusion Based on Semantic Segmentation, Journal of Computer Research and Development, Vol. 58, No. 2, 2021.

  19. Huafeng Li, Yueliang Cen, Yu Liu, Xun Chen, Zhengtao Yu.
    Different Input Resolutions and Arbitrary Output Resolution: A Meta Learning-based Deep Framework for Infrared and Visible Image Fusion, IEEE TIP, Vol. 30, 2021.[Paper]Pytorch]

  20. Zhao Xu, Gang Liu, Gang Xiao, Lili Tang, Yanhui Li.
    JCa2Co: A joint cascade convolution coding network based on fuzzy regional characteristics for infrared and visible image fusion, IET Computer Vision, 2021.

  21. Jiayi Ma, Linfeng Tang, Meilong Xu, Hao Zhang, Guobao Xiao.
    STDFusionNet: An Infrared and Visible Image Fusion Network Based on Salient Target Detection, IEEE Transactions on Instrumentation and Measurement, 2021. [Tensorflow]

  22. Zixiang Zhao, Shuang Xu, Jiangshe Zhang, Chengyang Liang, Chunxia Zhang, Junmin Liu.
    Efficient and Model-Based Infrared and Visible Image Fusion via Algorithm Unrolling, IEEE TCSVT, 2021.

  23. Juan Wang, Cong Ke, Minghu Wu, Min Liu, Chunyan Zeng.
    Infrared and visible image fusion based on Laplacian pyramid and generative adversarial network, KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS, vol. 15, no. 5, 2021.

  24. Luolin Liu, Mulin Chen, Mingliang Xu, Xuelong Li.
    Two-Stream Network for Infrared and Visible Images Fusion, Neurocomputing, 2021.

  25. Yu Fu, Xiaojun Wu, Josef Kittler.
    Effective method for fusing infrared and visible images, Journal of Electronic Imaging, 2021.

Conference

  1. Yan Zou, Linfei Zhang, Chengqian Liu, Bowen Wang, Yan Hu, Qian Chen.
    Infrared visible color night image fusion based on deep learning, Proc. of SPIE, 2021.

  2. Jixiao Wang, Yang Li, Zhuang Miao.
    A New Infrared and Visible Image Fusion Method Based on Generative Adversarial Networks and Attention Mechanism, ICIGP, 2021.

  3. Yu Fu, Xiao-Jun Wu.
    A Dual-branch Network for Infrared and Visible Image Fusion, ICPR, 2021.

arXiv

  1. Yu Fu, Xiao-Jun Wu, Josef Kittler.
    A Deep Decomposition Network for Image Processing: A Case Study for Visible and Infrared Image Fusion, arXiv, 2021.

  2. Yu Fu, TianYang Xu, XiaoJun Wu, Josef Kittler.
    PPT Fusion: Pyramid Patch Transformerfor a Case Study in Image Fusion, 2021.

2020

Journal

  1. Jiayi Ma, Han Xu, Junjun Jiang, Xiaoguang Mei, Xiao-Ping Zhang.
    DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion, IEEE Transactions on Image Processing, Vol. 29, 2020. [Tensorflow]

  2. Ruichao Hou, Dongming Zhou, Rencan Nie, Dong Liu, Lei Xiong, Yanbu Guo, Chuanbo Yu. VIF-Net: An Unsupervised Framework for Infrared and Visible Image Fusion, IEEE Transactions on Computational Imaging, Vol. 6, 2020.

  3. Jing Li, Hongtao Huo, Chang Li, Renhua Wang, Qi Feng.
    AttentionFGAN: Infrared and Visible Image Fusion using Attention-based Generative Adversarial Networks, IEEE Transactions on Multimedia, 2020.

  4. Jiayi Ma, Pengwei Liang, Wei Yu, Chen Chen, Xiaojie Guo, Jia Wu, Junjun Jiang.
    Infrared and Visible image fusion via detail preserving adversarial learning, Information Fusion, vol. 54, 2020.

  5. C. Yuan, C. Q. Sun, X. Y. Tang, R. F. Liu.
    FLGC-Fusion GAN: An Enhanced Fusion GAN Model by Importing Fully Learnable Group Convolution, Mathematical Problems in Engineering, 2020.

  6. Yongxin Zhang, Deguang Li, Wenpeng Zhu.
    Infrared and Visible Image Fusion with Hybrid Image Filtering, Mathematical Problems in Engineering, 2020.

  7. Yuqing Zhao, Guangyuan Fu, Hongqiao Wang, Shaolei Zhang.
    The Fusion of Unmatched Infrared and Visible Images Based on Generative Adversarial Networks, Mathematical Problems in Engineering, 2020.

  8. Yufang Feng, Houqing Lu, Jingbo Bai, Lin Cao, Hong Yin.
    Fully convolutional network-based infrared and visible image fusion, Multimedia Tools and Applications, 2020.

  9. Yang Li, Jixiao Wang, Zhuang Miao, Jiabao Wang.
    Unsupervised densely attentional network for infrared and visible image fusion, Multimedia Tools and Applications, 2020.

  10. Bin Liao, You Du, Xiangyun Yin.
    Fusion of Infrared-Visible Images in UE-IOT for Fault Point Detection Based on GAN, IEEE Access, 2020.

  11. Lei Yan, Jie Cao, Saad Rizvi, Kaiyu Zhang, Qun Hao, Xuemin Cheng.
    Improving the performance of image fusion based on visual saliency weight map combined with CNN, IEEE Access, 2020.

  12. Hafiz Tayyab Mustafa, Jie Yang, Hamza Mustafa, Masoumeh Zareapoor.
    Infrared and visible image fusion based on dilated residual attention network, Optik, 2020.

  13. Jing Li, Hongtao Huo, Kejian Liu, Chang Li.
    Infrared and visible image fusion using dual discriminators generative adversirial networks with Wasserstein distance, Information Sciences, Vol. 59, 2020.

  14. Dongdong Xu, Yongcheng Wang, Shuyan Xu, Kaiguang Zhu, Ning Zhang, Xin Zhang.
    Infrared and Visible Image Fusion with a Generative Adversarial Network and a Residual Network, Applied Science, 2020.

  15. Wen-Bo An, Hong-Mei Wang.
    Infrared and visible image fusion with supervised convolutional neural network, Optik, 2020.

  16. Jiangtao Xu, Xingping Shi, Shuzhen Qin, Kaige Lu, Han Wang, Jianguo Ma.
    LBP-BEGAN: A generative adversarial network archiecture for infrared and visible image fusion, Infrared Physics & Technology, Vol. 104, 2020.

  17. Jiahui Zhu, Qingyu Dou, Lihua Jian, Kai Liu, Farhan Hussain, Xiaomin Yang.
    Multiscale channel attention network for infrared and visible image fusion, Concurrency Computational, 2020.

Conference

  1. Zixiang Zhao, Shuang Xu, Chunxia Zhang, Junmin Liu, Pengfei Li, Jiangshe Zhang.
    DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion, IJCAI, 2020.

  2. Heena Patel, Kalpesh Prajapati, Vishal Chudasama, Kishor P. Upla.
    An Approach for Fusion of Thermal and Visible Images, International Conference on Emerging Technology Trends in Electronics Communication and Networking, 2020.

arXiv

  1. Aiqing Fang, Xinbo Zhao, Jiaqi Yang, Shihao Cao, Yanning Zhang.
    AE-Net: Autonomous Evolution Image Fusion Method Inspired by Human Cognitive Mechanism, arXiv, 2020.

  2. Aiqing Fang, Xinbo Zhao, Jiaqi Yang, Beibei Qin, Yanning Zhang.
    AE-Net2: Optimization of Image Fusion Efficiency and Network Architecture, arXiv, 2020.

  3. Zixiang Zhao, Shuang Xu, Chunxia Zhang, Junmin Liu, Jiangshe Zhang.
    Efficient and Interpretable Infrared and Visible Image Fusion Via Algorithm Unrolling, arXiv, 2020.

  4. Snigdha Bhagat, S. D. Joshi, Brejesh Lall.
    Image fusion using symmetric skip autoencoder via an Adversarial Regulariser, arXiv, 2020.

  5. Shaolei Liu, Manning Wang, Zhijian Song.
    WaveFuse: A Unified Deep Framework for Image Fusion with Discrete Wavelet Transform, arXiv, 2020.

  6. Zixiang Zhao, Shuang Xu, Rui Feng, Chunxia Zhang, Junmin Liu, Jiangshe Zhang.
    When Image Decomposition Meets Deep Learning: A Novel Infrared and Visible Image Fusion Method, arXiv, 2020.

2019

Journal

  1. Hui Li, Xiao-Jun Wu.
    DenseFuse: A Fusion Approach to Infrared and Visible Images, IEEE TIP, Vol. 28, No. 5, 2019.

  2. Jiayi Ma, Wei Yu, Pengwei Liang, Chang Li, Junjun Jiang.
    FusionGAN: A generative adversarial network for infrared and visible image fusion, Information Fusion, Vol. 48, 2019.

  3. Jingchun Piao, Yunfan Chen, Hyunchul Shin.
    A New Deep Learning Based Multi-Spectral Image Fusion Method, Entropy, 2019.

  4. Ivana Shopovska, Ljubomir Jovanov, Wilfried Philips.
    Deep Visible and Thermal Image Fusion for Enhanced Pedestrian Visibility, Sensors, 2019.

  5. Yaochen Liu, Lili Dong, Yuanyuan Ji, Wenhai Xu.
    Infrared and Visible Image Fusion through Details Preservation, Sensors, 2019.

  6. Yinchan Cui, Huiqian Du, Wenbo Mei.
    Infrared and Visible Image Fusion Using Detail Enhanced Channel Attention Network, IEEE Access, 2019.

  7. Hui Li, Xiao-Jun Wu, Tariq S. Durrani.
    Infrared and visible image fusion with ResNet and zero-phase component analysis, Infrared Physics & Technology, Vol. 102, 2019. [Matlab]

Conference

  1. Han Xu, Pengwei Liang, Wei Yu, Junjun Jiang, Jiayi Ma.
    Learning a Generative Model for Fusing Infrared and Visible Images via Conditional Generative Adversarial Network with Dual Discriminators, IJCAI, 2019.

  2. Yun Ge, Guodong Jing.
    Infrared and Visible Image Fusion Using Multi-resolution Convolution Neural Network, International Conference on Artificial Intelligence, Information Processing and Cloud Computing, 2019.

  3. Jing Li, Hongtao Huo, Kejian Liu, Chang Li, Shuo Li, Xin Yang.
    Infrared and Visible Image Fusion via Multi-Discriminators Wasserstein Generative Adversarial Network, International Conference on Machine Learning and Applications, 2019.

  4. Lebedev M. A., Komarov D. V., Vygolov O. V., Vizilter Yu. V.
    Multisensor Image Fusion based on Generative Adversarial Networks, Image and Signal Processing for Remote Sensing, 2019.

2018

Journal

  1. Wei Wu, Zongming Qiu, Min Zhao, Qiuhong Huang, Yang Lei.
    Visible and infrared image fusion using NSST and deep Boltzmann machine, Optik, Vol. 157, 2018.

  2. Yu Liu, Xun Chen, Juan Cheng, Hu Peng.
    Infrared and visible image fusion with convolutional neural networks, International Journal of Wavelets, Multiresolution and Information Processing, Vol. 16, No. 2, 2018.

Conference

  1. Hui Li, Xiao-Jun Wu, Josef Kittler.
    Infrared and Visible Image Fusion using a Deep Learning Framework, ICPR, 2018. [Matlab]

  2. Xianyi Ren, Fanyang Meng, Tao Hu, Zhijun Liu, Changwei Wang.
    Infrared-visible Image Fusion Based on Convolutional Neural Networks, International Conference on Intelligent Science and Big Data Engineering, 2018.

General-image-fusion

GIF-Review

  1. Harpreet Kaur, Deepika Koundal, Virender Kadyan.
    Image Fusion Techniques: A Survey, Archives of Computational Methods in Engineering, 2021.

  2. Simrandeep Singh, Nitin Mittal, Harbinder Singh.
    Review of Various Image Fusion Algorithms and Image Fusion Peformance Metric, Archives of Computational Methods in Engineering, 2021.

GIF-2023

Journal

GIF-2022

Journal

GIF-2021

Journal

  1. Risheng Liu, Jinyuan Liu, Zhiying Jiang, Xin Fan, Zhongxuan Luo.
    A Bilevel Integrated Model With Data-Driven Layer Ensemble for Multi-Modality Image Fusion, IEEE Transactions on Image Processing, Vol, 30, 2021. [MedIF, VIF]

arXiv

  1. Zhengwen Shen, Jun Wang, Zaiyu Pan, Yulian Li, Jiangyu Wang,
    Cross Attention-guided Dense Network for Images Fusion, arXiv.2109.11393v1.

GIF-2020

Journal

  1. Han Xu, Jiayi Ma, Junjun Jiang, Xiaojie Guo, Haibin Ling.
    U2Fusion: A Unified Unsupervised Image Fusion Network, IEEE TPAMI,2020. [Tensorflow]

  2. Asif Raza, Hong Huo, Tao Fang.
    PFAF-Net: Pyramid Feature Network for Multimodal Fusion, IEEE Sensors Journal, Vol. 4, No. 12, 2020

  3. Xin Deng, Pier Luigi Dragotti.
    Deep Convolutional Neural Network for Multi-modal Image Restoration and Fusion, IEEE TPAMI, 2020. (MFIF, MEF, MedIF)

  4. Fan Zhao, Wenda Zhao.
    Learning Specific and General Realm Feature Representations for Image Fusion, IEEE Transactions on Multimedia, 2020.

  5. Yu Zhang, Yu Liu, Peng Sun, Han Yan, Xiaolin Zhao, Li Zhang.
    IFCNN: A general image fusion framework based on convolutional neural network, Information Fusion, Vol. 54, 2020.

  6. Hyungjoo Jung, Youngjun Kim, Hyunsung Jang, Namkoo Ha, Kwanghoon Sohn.
    Unsupervised Deep Image Fusion With Structure Tensor Representations, IEEE Transactions on Image Processing, Vol. 29, 2020.

Conference

  1. Han Xu, Jiayi Ma, Zhuliang Le, Junjun Jiang, Xiaojie Guo.
    FusionDN: A Unified Densely Connected Network for Image Fusion, AAAI, 2020.

  2. Hao Zhang, Han Xu, Yang Xiao, Xiaojie Guo, Jiayi Ma. Rethinking the Image Fusion: A Fast Unified Image Fusion Network based on Proportional Maintenance of Gradient and Intensity, AAAI, 2020.

GIF-2019

Journal

  1. Meng Wang, Xingwang Liu, Huaiping Jin.
    A generative image fusion approach based on supervised deep convolutional network driven by weighted gradient flow, Image and Vision Computing, Vol. 86, 2020.

arXiv

  1. Fayez Lahoud, Sabine Susstruck.
    Fast and Efficient Zero-Learning Image Fusion, arXiv, 2019.

Citation

If you find this work is useful, please consider citing our paper:

@article{zhang2023visible,
  title={Visible and Infrared Image Fusion Using Deep Learning},
  author={Zhang, Xingchen and Demiris, Yiannis},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2023},
  publisher={IEEE}
}

About

Deep learning-based methods for visible and infrared image fusion