DiffuMural：基于多尺度扩散的敦煌壁画修复

DiffuMural: Restoring Dunhuang Murals with Multi-scale Diffusion

April 13, 2025

作者: Puyu Han, Jiaju Kang, Yuhang Pan, Erting Pan, Zeyu Zhang, Qunchao Jin, Juntao Jiang, Zhichen Liu, Luqi Gong

cs.AI

摘要

大规模预训练扩散模型在条件图像生成领域已取得卓越成果。然而，作为该领域重要下游任务的古代壁画修复，因其大面积缺损区域和稀缺的训练样本，对基于扩散模型的修复方法提出了重大挑战。条件修复任务更关注修复部分在整体风格和接缝细节上是否符合壁画修复的美学标准，而当前研究中缺乏评估启发式图像补全的此类指标。为此，我们提出了DiffuMural，结合多尺度收敛与协作扩散机制，利用ControlNet和循环一致性损失优化生成图像与条件控制之间的匹配。DiffuMural在壁画修复中展现出卓越能力，得益于23幅具有一致视觉美学的大型敦煌壁画训练数据。该模型在恢复精细细节、实现整体外观一致性以及应对缺乏事实依据的不完整壁画独特挑战方面表现优异。我们的评估框架包含四项关键指标，用于定量评估不完整壁画：事实准确性、纹理细节、上下文语义和整体视觉连贯性。此外，我们整合了人文价值评估，确保修复后的壁画保留其文化与艺术意义。大量实验验证，我们的方法在定性和定量指标上均优于现有最先进（SOTA）方法。

English

Large-scale pre-trained diffusion models have produced excellent results in the field of conditional image generation. However, restoration of ancient murals, as an important downstream task in this field, poses significant challenges to diffusion model-based restoration methods due to its large defective area and scarce training samples. Conditional restoration tasks are more concerned with whether the restored part meets the aesthetic standards of mural restoration in terms of overall style and seam detail, and such metrics for evaluating heuristic image complements are lacking in current research. We therefore propose DiffuMural, a combined Multi-scale convergence and Collaborative Diffusion mechanism with ControlNet and cyclic consistency loss to optimise the matching between the generated images and the conditional control. DiffuMural demonstrates outstanding capabilities in mural restoration, leveraging training data from 23 large-scale Dunhuang murals that exhibit consistent visual aesthetics. The model excels in restoring intricate details, achieving a coherent overall appearance, and addressing the unique challenges posed by incomplete murals lacking factual grounding. Our evaluation framework incorporates four key metrics to quantitatively assess incomplete murals: factual accuracy, textural detail, contextual semantics, and holistic visual coherence. Furthermore, we integrate humanistic value assessments to ensure the restored murals retain their cultural and artistic significance. Extensive experiments validate that our method outperforms state-of-the-art (SOTA) approaches in both qualitative and quantitative metrics.