Light-A-Video：通过渐进光融合实现无需训练的视频照明调整

Light-A-Video: Training-free Video Relighting via Progressive Light Fusion

February 12, 2025

作者: Yujie Zhou, Jiazi Bu, Pengyang Ling, Pan Zhang, Tong Wu, Qidong Huang, Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Anyi Rao, Jiaqi Wang, Li Niu

cs.AI

摘要

最近，受大规模数据集和预训练扩散模型推动，图像照明模型取得了显著进展，实现了一致的照明效果。然而，视频照明仍然滞后，主要是由于训练成本过高以及多样性和高质量视频照明数据集的稀缺。将图像照明模型简单应用于逐帧基础会导致几个问题：照明源不一致和照明效果不一致，从而在生成的视频中出现闪烁。在这项工作中，我们提出了Light-A-Video，这是一种无需训练的方法，可实现视频照明的时间平滑处理。Light-A-Video从图像照明模型中演变出来，引入了两个关键技术来增强照明的一致性。首先，我们设计了一个一致照明注意（CLA）模块，它增强了自注意力层内的跨帧交互，以稳定生成背景照明源。其次，利用光传输独立的物理原理，我们在源视频外观和照明后的外观之间应用线性混合，使用渐进式光融合（PLF）策略，以确保照明的平滑时间过渡。实验证明，Light-A-Video提高了照明视频的时间一致性，同时保持图像质量，确保帧间照明过渡连贯。项目页面：https://bujiazi.github.io/light-a-video.github.io/。

English

Recent advancements in image relighting models, driven by large-scale datasets and pre-trained diffusion models, have enabled the imposition of consistent lighting. However, video relighting still lags, primarily due to the excessive training costs and the scarcity of diverse, high-quality video relighting datasets. A simple application of image relighting models on a frame-by-frame basis leads to several issues: lighting source inconsistency and relighted appearance inconsistency, resulting in flickers in the generated videos. In this work, we propose Light-A-Video, a training-free approach to achieve temporally smooth video relighting. Adapted from image relighting models, Light-A-Video introduces two key techniques to enhance lighting consistency. First, we design a Consistent Light Attention (CLA) module, which enhances cross-frame interactions within the self-attention layers to stabilize the generation of the background lighting source. Second, leveraging the physical principle of light transport independence, we apply linear blending between the source video's appearance and the relighted appearance, using a Progressive Light Fusion (PLF) strategy to ensure smooth temporal transitions in illumination. Experiments show that Light-A-Video improves the temporal consistency of relighted video while maintaining the image quality, ensuring coherent lighting transitions across frames. Project page: https://bujiazi.github.io/light-a-video.github.io/.