Re-boosting Self-Collaboration Parallel Prompt GAN for Unsupervised Image Restoration

Xin Lin, Yuyan Zhou, Jingtong Yue, Chao Ren, Kelvin C.K. Chan, Lu Qi, Ming-Hsuan Yang

This is the journal version of SCPGabNet

This is the official code of RSCP^2^GAN for denoising tasks.

Abstract

Deep learning methods have shown remarkable performance in image restoration, particularly when trained on large-scale paired datasets. However, acquiring such data in real-world scenarios poses a significant challenge. Although unsupervised restoration approaches based on generative adversarial networks (GANs) offer a promising solution without paired datasets, they are difficult to surpass the performance limitations of conventional unsupervised GAN-based frameworks without significantly modifying existing structures or increasing the computational complexity of the restorer. To address this problem, we propose a self-collaboration (SC) strategy for existing restoration models. It utilizes the information from the previous stage as feedback to guide the subsequent stages and achieves significant performance improvement without increasing the framework's inference complexity. It comprises a prompt learning (PL) module and a restorer ($Res$). It iteratively replaces the previous less powerful fixed restorer $\overline{Res}$ in the PL module with the current powerful $Res$. The enhanced PL module will generate better pseudo-degraded/clean image pairs, leading to a more powerful $Res$ for the next iteration without any extra parameters or inference computational complexity. The proposed iteratively updated process generates better synthetic clean/degraded image pairs, leading to a more powerful $Res$ for the next iteration. Additionally, the traditional self-ensemble (SE) and our self-collaboration (SC) strategies can enhance the performance of the pre-trained restorers from different perspectives. However, the SE increases the computational complexity during the inference phase. Therefore, we propose a re-boosting module to the SC (Reb-SC) to improve the SC strategy further by incorporating SE into SC without increasing the inference time. In this way, the performance of the restorer can be improved further (approximately 0.3dB). In addition, we propose a baseline framework that includes parallel generative adversarial branches with complementary self-synthesis'' and unpaired-synthesis'' constraints. This baseline ensures the effectiveness of the training framework. Extensive experimental results on various restoration tasks demonstrate that the proposed method performs favorably against existing state-of-the-art unsupervised restoration methods.