π SMFANet: A Lightweight Self-Modulation Feature Aggregation Network for Efficient Image Super-Resolution
Mingjun Zheng, Long Sun, Jiangxin Dong, and Jinshan Pan
IMAG Lab, Nanjing University of Science and Technology
Network architecture of the proposed SMFANet. The proposed s SMFANet consists of a shallow feature extraction module, feature modulation blocks, and a lightweight image reconstruction module. Feature modulation block contains one self-modulation feature aggregation (SMFA) module and one partial convolution-based feed-forward network (PCFN).
- [2024-07-16] The paper is available Here.
- [2024-07-16] We add π€Hugging Face Demo.
- [2024-07-01] Our SMFANet is accepted by ECCV 2024.
- [2024-06-25] Our SMFANet places 2nd and 3rd in the Parameters and FLOPs sub-track of the NTIRE2024 ESR.
- Python 3.8, PyTorch >= 1.8
- BasicSR 1.4.2
- Platforms: Ubuntu 18.04, cuda-11
# Clone the repo
git clone https://github.com/Zheng-MJ/SMFANet.git
# Install dependent packages
cd SMFANet
conda create --name smfan python=3.8
conda activate smfan
pip install -r requirements.txt
# Install BasicSR
python setup.py develop
You can also refer to this INSTALL.md for installation
Please refer to datasets/REDAME.md for data preparation.
Run the following commands for training:
# train SMFANet for x4 effieicnt SR
python basicsr/train.py -opt options/train/SMFANet/SMFANet_DIV2K_100w_x4SR.yml
# train SMFANet+ for x4 effieicnt SR
python basicsr/train.py -opt options/train/SMFANet/SMFANet_plus_DIV2K_100w_x4SR.yml
- Download the testing dataset.
- The pretrained models are in './pretrain'.
- Run the following commands:
# test SMFANet for x4 efficient SR
python basicsr/test.py -opt options/test/SMFANet_DF2K_x2SR.yml
- The test results will be in './results'.
- The script for exporting TensorRT model is available at to_tensorrt/READEME.md
- The Hugging Face Demo is available here.
- Comparison with CNN-based lightweight SR methods
- Comparison with ViT-based lightweight SR methods
- Memory and running time comparisons on x4 SR
- Visual comparisons for x4 SR on the Urban100 dataset
- Comparison of local attribution maps (LAMs) and diffusion indices (DIs)
- The power spectral density (PSD) visualizations of feature
If this work is helpful for your research, please consider citing the following BibTeX entry.
@inproceedings{smfanet,
title={SMFANet: A Lightweight Self-Modulation Feature Aggregation Network for Efficient Image Super-Resolution},
author={Zheng, Mingjun and Sun, Long and Dong, Jiangxin and Pan, Jinshan},
booktitle={ECCV},
year={2024}
}
This code is based on BasicSR toolbox. Thanks for the awesome work.
If you have any questions, please feel free to reach me out at mingjunzheng@njust.edu.cn