DynamicScaler：全景场景视频生成的无缝可扩展性。

DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes

December 15, 2024

作者: Jinxiu Liu, Shaoheng Lin, Yinxiao Li, Ming-Hsuan Yang

cs.AI

摘要

随着对沉浸式增强现实（AR）/虚拟现实（VR）应用和空间智能的需求不断增加，生成高质量的场景级和360°全景视频的需求日益迫切。然而，大多数视频扩散模型受限于有限的分辨率和宽高比，这限制了它们对场景级动态内容合成的适用性。在本文中，我们提出了DynamicScaler，通过实现空间可伸缩和全景动态场景合成，解决了这些挑战，保持了任意大小全景场景之间的连贯性。具体地，我们引入了一种Offset Shifting去噪器，通过一个无缝旋转的窗口，利用固定分辨率的扩散模型，促进了高效、同步和连贯的去噪全景动态场景，确保了全景空间的无缝边界过渡和一致性，适应了不同的分辨率和宽高比。此外，我们采用了全局运动引导机制，以确保局部细节的保真度和全局运动的连续性。大量实验证明，我们的方法在全景场景级视频生成中实现了卓越的内容和运动质量，为沉浸式动态场景创作提供了一个无需训练、高效且可伸缩的解决方案，无论输出视频分辨率如何，都能保持恒定的VRAM消耗。我们的项目页面位于https://dynamic-scaler.pages.dev/。

English

The increasing demand for immersive AR/VR applications and spatial intelligence has heightened the need to generate high-quality scene-level and 360{\deg} panoramic video. However, most video diffusion models are constrained by limited resolution and aspect ratio, which restricts their applicability to scene-level dynamic content synthesis. In this work, we propose the DynamicScaler, addressing these challenges by enabling spatially scalable and panoramic dynamic scene synthesis that preserves coherence across panoramic scenes of arbitrary size. Specifically, we introduce a Offset Shifting Denoiser, facilitating efficient, synchronous, and coherent denoising panoramic dynamic scenes via a diffusion model with fixed resolution through a seamless rotating Window, which ensures seamless boundary transitions and consistency across the entire panoramic space, accommodating varying resolutions and aspect ratios. Additionally, we employ a Global Motion Guidance mechanism to ensure both local detail fidelity and global motion continuity. Extensive experiments demonstrate our method achieves superior content and motion quality in panoramic scene-level video generation, offering a training-free, efficient, and scalable solution for immersive dynamic scene creation with constant VRAM consumption regardless of the output video resolution. Our project page is available at https://dynamic-scaler.pages.dev/.