自适应盲全能图像恢复

Adaptive Blind All-in-One Image Restoration

November 27, 2024

作者: David Serrano-Lozano, Luis Herranz, Shaolin Su, Javier Vazquez-Corral

cs.AI

摘要

盲目的全能图像恢复模型旨在从输入中受损的图像中恢复高质量图像，而这些模型需要在训练阶段定义所有可能的退化类型，同时对未知的退化类型的泛化能力有限，这限制了它们在复杂情况下的实际应用。在本文中，我们提出了一种简单但有效的自适应盲目全能恢复（ABAIR）模型，可以处理多种退化，对未知的退化类型具有良好的泛化能力，并通过训练少量参数有效地整合新的退化类型。首先，我们在大量自然图像数据集上训练基准模型，这些图像带有多种合成退化，并增加了一个分割头来估计每个像素的退化类型，从而产生一个强大的骨干网络，能够泛化到各种退化。其次，我们通过独立的低秩适配器将基准模型调整到不同的图像恢复任务。第三，我们学习通过灵活轻量的退化估计器来自适应地组合适配器以适用于多样的图像。我们的模型在处理特定失真方面非常强大，同时在适应复杂任务方面非常灵活，不仅在五项和三项IR设置上远远优于现有技术，而且在对未知退化和复合失真的泛化方面也有所改进。

English

Blind all-in-one image restoration models aim to recover a high-quality image from an input degraded with unknown distortions. However, these models require all the possible degradation types to be defined during the training stage while showing limited generalization to unseen degradations, which limits their practical application in complex cases. In this paper, we propose a simple but effective adaptive blind all-in-one restoration (ABAIR) model, which can address multiple degradations, generalizes well to unseen degradations, and efficiently incorporate new degradations by training a small fraction of parameters. First, we train our baseline model on a large dataset of natural images with multiple synthetic degradations, augmented with a segmentation head to estimate per-pixel degradation types, resulting in a powerful backbone able to generalize to a wide range of degradations. Second, we adapt our baseline model to varying image restoration tasks using independent low-rank adapters. Third, we learn to adaptively combine adapters to versatile images via a flexible and lightweight degradation estimator. Our model is both powerful in handling specific distortions and flexible in adapting to complex tasks, it not only outperforms the state-of-the-art by a large margin on five- and three-task IR setups, but also shows improved generalization to unseen degradations and also composite distortions.