AdaIR：自适应全能图像修复：通过频率挖掘和调制

AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation

March 21, 2024

作者: Yuning Cui, Syed Waqas Zamir, Salman Khan, Alois Knoll, Mubarak Shah, Fahad Shahbaz Khan

cs.AI

摘要

在图像获取过程中，通常会引入各种形式的退化，包括噪声、雾霾和雨水。这些退化通常源自相机固有的限制或不利的环境条件。为了从退化版本中恢复清晰图像，已经开发了许多专门的恢复方法，每种方法针对特定类型的退化。最近，全能算法通过在单个模型中处理不同类型的退化而无需先验输入退化类型信息，引起了广泛关注。然而，这些方法纯粹在空间域中运行，没有深入研究不同退化类型固有的不同频率变化。为了填补这一空白，我们提出了一种基于频率挖掘和调制的自适应全能图像恢复网络。我们的方法受到这样一观察的启发，即不同的退化类型会影响图像内容在不同频率子带上的不同方式，因此需要针对每个恢复任务进行不同处理。具体来说，我们首先从输入特征中挖掘低频和高频信息，由退化图像的自适应解耦谱引导。然后，提取的特征通过双向操作符进行调制，以促进不同频率分量之间的交互。最后，调制后的特征与原始输入合并，进行逐步引导的恢复。通过这种方法，模型通过强调根据不同输入退化强调信息频率子带，实现自适应重建。大量实验证明，所提出的方法在不同图像恢复任务上取得了最先进的性能，包括去噪、去雾、去雨、运动去模糊和低光图像增强。我们的代码可在https://github.com/c-yn/AdaIR找到。

English

In the image acquisition process, various forms of degradation, including noise, haze, and rain, are frequently introduced. These degradations typically arise from the inherent limitations of cameras or unfavorable ambient conditions. To recover clean images from degraded versions, numerous specialized restoration methods have been developed, each targeting a specific type of degradation. Recently, all-in-one algorithms have garnered significant attention by addressing different types of degradations within a single model without requiring prior information of the input degradation type. However, these methods purely operate in the spatial domain and do not delve into the distinct frequency variations inherent to different degradation types. To address this gap, we propose an adaptive all-in-one image restoration network based on frequency mining and modulation. Our approach is motivated by the observation that different degradation types impact the image content on different frequency subbands, thereby requiring different treatments for each restoration task. Specifically, we first mine low- and high-frequency information from the input features, guided by the adaptively decoupled spectra of the degraded image. The extracted features are then modulated by a bidirectional operator to facilitate interactions between different frequency components. Finally, the modulated features are merged into the original input for a progressively guided restoration. With this approach, the model achieves adaptive reconstruction by accentuating the informative frequency subbands according to different input degradations. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance on different image restoration tasks, including denoising, dehazing, deraining, motion deblurring, and low-light image enhancement. Our code is available at https://github.com/c-yn/AdaIR.