去噪作为适应性：噪声空间域自适应图像恢复

Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration

June 26, 2024

作者: Kang Liao, Zongsheng Yue, Zhouxia Wang, Chen Change Loy

cs.AI

摘要

尽管基于学习的图像恢复方法取得了显著进展，但由于在合成数据上训练导致的实际场景的领域差距较大，它们仍然难以实现有限的泛化能力。现有方法通过改进数据合成流程、估计退化核、采用深度内部学习，并进行领域自适应和正则化来解决这一问题。先前的领域自适应方法试图通过在特征空间或像素空间中学习领域不变的知识来弥合领域差距。然而，这些技术通常难以在稳定而紧凑的框架内扩展到低级视觉任务。本文展示了可以通过噪声空间使用扩散模型进行领域自适应的可能性。特别是，通过利用辅助条件输入如何影响多步去噪过程的独特属性，我们推导出一个有意义的扩散损失，该损失指导恢复模型逐步使恢复的合成和真实输出与目标干净分布对齐。我们将这种方法称为去噪自适应。为了防止在联合训练过程中出现捷径，我们提出了关键策略，如通道混洗层和残差交换对比学习在扩散模型中。它们隐式地模糊了条件合成和真实数据之间的边界，并防止模型依赖于容易区分的特征。对三个经典图像恢复任务，即去噪、去模糊和去雨，进行的实验结果展示了所提方法的有效性。

English

Although learning-based image restoration methods have made significant progress, they still struggle with limited generalization to real-world scenarios due to the substantial domain gap caused by training on synthetic data. Existing methods address this issue by improving data synthesis pipelines, estimating degradation kernels, employing deep internal learning, and performing domain adaptation and regularization. Previous domain adaptation methods have sought to bridge the domain gap by learning domain-invariant knowledge in either feature or pixel space. However, these techniques often struggle to extend to low-level vision tasks within a stable and compact framework. In this paper, we show that it is possible to perform domain adaptation via the noise space using diffusion models. In particular, by leveraging the unique property of how auxiliary conditional inputs influence the multi-step denoising process, we derive a meaningful diffusion loss that guides the restoration model in progressively aligning both restored synthetic and real-world outputs with a target clean distribution. We refer to this method as denoising as adaptation. To prevent shortcuts during joint training, we present crucial strategies such as channel-shuffling layer and residual-swapping contrastive learning in the diffusion model. They implicitly blur the boundaries between conditioned synthetic and real data and prevent the reliance of the model on easily distinguishable features. Experimental results on three classical image restoration tasks, namely denoising, deblurring, and deraining, demonstrate the effectiveness of the proposed method.