Jiezhang Cao, Yue Shi, Kai Zhang, Yulun Zhang, Radu Timofte, Luc Van Gool
Computer Vision Lab, ETH Zurich
Abstract: Diffusion-based image restoration (IR) methods aim to use diffusion models to recover high-quality (HQ) images from degraded images and achieve promising performance. Due to the inherent property of diffusion models, most of these methods need long serial sampling chains to restore HQ images step-by-step. As a result, it leads to expensive sampling time and high computation costs. Moreover, such long sampling chains hinder understanding the relationship between the restoration results and the inputs since it is hard to compute the gradients in the whole chains. In this work, we aim to rethink the diffusion-based IR models through a different perspective, i.e., a deep equilibrium (DEQ) fixed point system. Specifically, we derive an analytical solution by modeling the entire sampling chain in diffusion-based IR models as a joint multivariate fixed point system. With the help of the analytical solution, we are able to conduct single-image sampling in a parallel way and restore HQ images without training. Furthermore, we compute fast gradients in DEQ and found that initialization optimization can boost performance and control the generation direction. Extensive experiments on benchmarks demonstrate the effectiveness of our proposed method on typical IR tasks and real-world settings.
- Complete this repository
We achieved state-of-the-art performance on many image restoration tasks. More results can be found in the paper.
Quantitative Comparison (click to expan)
Visual Comparison (click to expan)
- Classical image restoration
- Real-world image restoration
- Generation diversity
- Initialization optimization via Inversion
If you find the code helpful in your resarch or work, please cite the following paper(s).
@article{cao2023deqir,
title = {Deep Equilibrium Diffusion Restoration with Parallel Sampling},
author = {Jiezhang Cao and Yue Shi and Kai Zhang and Yulun Zhang and Radu Timofte and Luc Van Gool},
journal = {arxiv},
year = {2023},
}