DynamicScaler：全景場景的無縫且可擴展視頻生成

DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes

December 15, 2024

作者: Jinxiu Liu, Shaoheng Lin, Yinxiao Li, Ming-Hsuan Yang

cs.AI

摘要

隨著對沉浸式擴增實境（AR）/虛擬實境（VR）應用和空間智能的需求不斷增加，生成高質量的場景級和360度全景視頻的需求日益迫切。然而，大多數視頻擴散模型受到有限的解析度和寬高比的限制，這限制了它們對場景級動態內容合成的適用性。在這項工作中，我們提出了DynamicScaler，通過實現空間可伸縮和全景動態場景合成，以解決這些挑戰，保持跨任意大小全景場景的一致性。具體來說，我們引入了一種Offset Shifting Denoiser，通過一個無縫旋轉的窗口，便利地、同步地和一致地對全景動態場景進行去噪，通過具有固定解析度的擴散模型，確保無縫的邊界過渡和整個全景空間的一致性，以滿足不同解析度和寬高比的需求。此外，我們採用全局運動引導機制，以確保局部細節的保真度和全局運動的連續性。大量實驗證明，我們的方法在全景場景級視頻生成中實現了卓越的內容和運動質量，為沉浸式動態場景創建提供了一種無需訓練、高效且可擴展的解決方案，無論輸出視頻的解析度如何，都能保持恆定的VRAM消耗。我們的項目頁面位於https://dynamic-scaler.pages.dev/。

English

The increasing demand for immersive AR/VR applications and spatial intelligence has heightened the need to generate high-quality scene-level and 360{\deg} panoramic video. However, most video diffusion models are constrained by limited resolution and aspect ratio, which restricts their applicability to scene-level dynamic content synthesis. In this work, we propose the DynamicScaler, addressing these challenges by enabling spatially scalable and panoramic dynamic scene synthesis that preserves coherence across panoramic scenes of arbitrary size. Specifically, we introduce a Offset Shifting Denoiser, facilitating efficient, synchronous, and coherent denoising panoramic dynamic scenes via a diffusion model with fixed resolution through a seamless rotating Window, which ensures seamless boundary transitions and consistency across the entire panoramic space, accommodating varying resolutions and aspect ratios. Additionally, we employ a Global Motion Guidance mechanism to ensure both local detail fidelity and global motion continuity. Extensive experiments demonstrate our method achieves superior content and motion quality in panoramic scene-level video generation, offering a training-free, efficient, and scalable solution for immersive dynamic scene creation with constant VRAM consumption regardless of the output video resolution. Our project page is available at https://dynamic-scaler.pages.dev/.