適應性盲目全能圖像修復

摘要

盲目的全能影像修復模型旨在從受到未知失真的輸入中恢復高質量影像。然而，這些模型在訓練階段需要定義所有可能的失真類型，同時對未知失真的泛化能力有限，這限制了它們在複雜情況下的實際應用。本文提出了一種簡單但有效的自適應盲目全能修復（ABAIR）模型，能處理多種失真，對未知失真有良好泛化能力，並通過訓練少量參數有效地整合新的失真。首先，我們在大量自然影像數據集上訓練基準模型，其中包含多種合成失真，並增加了一個分割頭部來估計每像素的失真類型，從而產生一個強大的骨幹，能夠泛化到各種失真。其次，我們使用獨立的低秩適配器將基準模型適應到不同的影像修復任務。第三，我們通過靈活輕量的失真估計器學習如何自適應地組合適配器以適應多樣的影像。我們的模型在處理特定失真方面強大且靈活適應複雜任務，不僅在五項和三項任務的影像修復設置中遠遠優於最先進技術，而且在對未知失真和複合失真的泛化方面也有所提升。

English

Blind all-in-one image restoration models aim to recover a high-quality image from an input degraded with unknown distortions. However, these models require all the possible degradation types to be defined during the training stage while showing limited generalization to unseen degradations, which limits their practical application in complex cases. In this paper, we propose a simple but effective adaptive blind all-in-one restoration (ABAIR) model, which can address multiple degradations, generalizes well to unseen degradations, and efficiently incorporate new degradations by training a small fraction of parameters. First, we train our baseline model on a large dataset of natural images with multiple synthetic degradations, augmented with a segmentation head to estimate per-pixel degradation types, resulting in a powerful backbone able to generalize to a wide range of degradations. Second, we adapt our baseline model to varying image restoration tasks using independent low-rank adapters. Third, we learn to adaptively combine adapters to versatile images via a flexible and lightweight degradation estimator. Our model is both powerful in handling specific distortions and flexible in adapting to complex tasks, it not only outperforms the state-of-the-art by a large margin on five- and three-task IR setups, but also shows improved generalization to unseen degradations and also composite distortions.

適應性盲目全能圖像修復

Adaptive Blind All-in-One Image Restoration

摘要

Summary

Support