Yong Liu,
Hang Dong,
Jinshan Pan,
Qingji Dong,
Kai Chen,
Rongxiang Zhang,
Lean Fu,
Fei Wang
💖 If our PatchScaler is helpful to your researches or projects, please help star this repository. Thanks! 🤗
Diffusion models significantly improve the quality of super-resolved images with their impressive content generation capabilities. However, the huge computational costs limit the applications of these methods. Recent efforts have explored reasonable inference acceleration to reduce the number of sampling steps, but the computational cost remains high as each step is performed on the entire image. This paper introduces PatchScaler, a patch-independent diffusion-based single image super-resolution (SR) method, designed to enhance the efficiency of the inference process. The proposed method is motivated by the observation that not all the image patches within an image need the same sampling steps for reconstructing high-resolution images. Based on this observation, we thus develop a Patch-adaptive Group Sampling (PGS) to divide feature patches into different groups according to the patch-level reconstruction difficulty and dynamically assign an appropriate sampling configuration for each group so that the inference speed can be better accelerated. In addition, to improve the denoising ability at each step of the sampling, we develop a texture prompt to guide the estimations of the diffusion model by retrieving high-quality texture priors from a patch-independent reference texture memory. Experiments show that our PatchScaler achieves favorable performance in both quantitative and qualitative evaluations with fast inference speed.
- 2024.05.27: Create this repository.
- New project website
- The training scripts
- Releasing pretrained models
- The inference scripts
This project is released under the Apache 2.0 license. Redistribution and use should follow this license.
If you find this project useful for your research, please use the following BibTeX entry.
@inproceedings{liu2024patchscaler,
title={PatchScaler: An Efficient Patch-independent Diffusion Model for Super-Resolution},
author={Yong Liu, Hang Dong, Jinshan Pan, Qingji Dong, Kai Chen, Rongxiang Zhang, Xing Mei, Lean Fu, and Fei Wang},
year={2024}
}