具有擴散先驗的降級引導一步圖像超分辨率

摘要

基於擴散的影像超分辨率（SR）方法通過利用大型預訓練的文本到影像擴散模型作為先驗取得了顯著成功。然而，這些方法仍然面臨兩個挑戰：需要數十個採樣步驟才能達到令人滿意的結果，這限制了在實際情況下的效率，以及忽略了降解模型，這是解決SR問題中至關重要的輔助信息。在這項工作中，我們引入了一種新型的一步SR模型，顯著解決了基於擴散的SR方法的效率問題。與現有的微調策略不同，我們專門為SR設計了一個基於降解引導的低秩適應（LoRA）模塊，根據從低分辨率影像中預估的降解信息來校正模型參數。該模塊不僅促進了強大的數據依賴或降解依賴的SR模型，還盡可能保留了預訓練擴散模型的生成先驗。此外，我們通過引入在線負樣本生成策略，量身定制了一種新型的訓練流程。結合推斷過程中的無分類器引導策略，大大提高了超分辨率結果的感知質量。大量實驗證明了所提出模型相對於最近的最先進方法具有卓越的效率和有效性。

English

Diffusion-based image super-resolution (SR) methods have achieved remarkable success by leveraging large pre-trained text-to-image diffusion models as priors. However, these methods still face two challenges: the requirement for dozens of sampling steps to achieve satisfactory results, which limits efficiency in real scenarios, and the neglect of degradation models, which are critical auxiliary information in solving the SR problem. In this work, we introduced a novel one-step SR model, which significantly addresses the efficiency issue of diffusion-based SR methods. Unlike existing fine-tuning strategies, we designed a degradation-guided Low-Rank Adaptation (LoRA) module specifically for SR, which corrects the model parameters based on the pre-estimated degradation information from low-resolution images. This module not only facilitates a powerful data-dependent or degradation-dependent SR model but also preserves the generative prior of the pre-trained diffusion model as much as possible. Furthermore, we tailor a novel training pipeline by introducing an online negative sample generation strategy. Combined with the classifier-free guidance strategy during inference, it largely improves the perceptual quality of the super-resolution results. Extensive experiments have demonstrated the superior efficiency and effectiveness of the proposed model compared to recent state-of-the-art methods.

具有擴散先驗的降級引導一步圖像超分辨率

Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors

摘要

Summary

Support

Support