DiffuMural：運用多尺度擴散技術修復敦煌壁畫

DiffuMural: Restoring Dunhuang Murals with Multi-scale Diffusion

April 13, 2025

作者: Puyu Han, Jiaju Kang, Yuhang Pan, Erting Pan, Zeyu Zhang, Qunchao Jin, Juntao Jiang, Zhichen Liu, Luqi Gong

cs.AI

摘要

大規模預訓練擴散模型在條件圖像生成領域已取得卓越成果。然而，作為該領域重要下游任務的古代壁畫修復，因其大面積缺損和訓練樣本稀缺，對基於擴散模型的修復方法提出了重大挑戰。條件修復任務更關注修復部分在整體風格和接縫細節上是否符合壁畫修復的美學標準，而當前研究中缺乏評估啟發式圖像補全的此類指標。因此，我們提出了DiffuMural，它結合了多尺度收斂與協同擴散機制，並利用ControlNet和循環一致性損失來優化生成圖像與條件控制之間的匹配。DiffuMural在壁畫修復中展現了卓越能力，其訓練數據來自23幅具有一致視覺美學的大規模敦煌壁畫。該模型在恢復精細細節、實現整體外觀連貫性以及應對缺乏事實依據的不完整壁畫所帶來的獨特挑戰方面表現出色。我們的評估框架整合了四個關鍵指標，以定量評估不完整壁畫：事實準確性、紋理細節、上下文語義和整體視覺連貫性。此外，我們還融入了人文價值評估，確保修復後的壁畫保留其文化和藝術意義。大量實驗驗證，我們的方法在質量和數量指標上均優於當前最先進（SOTA）的方法。

English

Large-scale pre-trained diffusion models have produced excellent results in the field of conditional image generation. However, restoration of ancient murals, as an important downstream task in this field, poses significant challenges to diffusion model-based restoration methods due to its large defective area and scarce training samples. Conditional restoration tasks are more concerned with whether the restored part meets the aesthetic standards of mural restoration in terms of overall style and seam detail, and such metrics for evaluating heuristic image complements are lacking in current research. We therefore propose DiffuMural, a combined Multi-scale convergence and Collaborative Diffusion mechanism with ControlNet and cyclic consistency loss to optimise the matching between the generated images and the conditional control. DiffuMural demonstrates outstanding capabilities in mural restoration, leveraging training data from 23 large-scale Dunhuang murals that exhibit consistent visual aesthetics. The model excels in restoring intricate details, achieving a coherent overall appearance, and addressing the unique challenges posed by incomplete murals lacking factual grounding. Our evaluation framework incorporates four key metrics to quantitatively assess incomplete murals: factual accuracy, textural detail, contextual semantics, and holistic visual coherence. Furthermore, we integrate humanistic value assessments to ensure the restored murals retain their cultural and artistic significance. Extensive experiments validate that our method outperforms state-of-the-art (SOTA) approaches in both qualitative and quantitative metrics.