MoDec-GS：全局到局部运动分解和时间间隔调整，用于紧凑的动态3D高斯飞溅。

MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting

January 7, 2025

作者: Sangwoon Kwak, Joonsoo Kim, Jun Young Jeong, Won-Sik Cheong, Jihyong Oh, Munchurl Kim

cs.AI

摘要

3D 高斯飘逸（3DGS）在场景表示和神经渲染方面取得了显著进展，人们致力于将其应用于动态场景。尽管现有方法在提供出色的渲染质量和速度方面表现出色，但在存储需求和复杂现实世界运动表示方面存在困难。为了解决这些问题，我们提出了MoDecGS，这是一个内存高效的高斯飘逸框架，旨在重建具有复杂运动挑战性场景中的新视图。我们引入了全局到局部运动分解（GLMD），以有效地以粗到细的方式捕捉动态运动。该方法利用全局规范支架（Global CS）和局部规范支架（Local CS），将静态支架表示扩展到动态视频重建。对于全局 CS，我们提出了全局锚点变形（GAD），通过直接变形隐式支架属性（锚点位置、偏移和局部上下文特征）来高效表示沿复杂运动的全局动态。接下来，我们通过局部高斯变形（LGD）明确地对局部 CS 进行微调。此外，我们引入了时间间隔调整（TIA），在训练过程中自动控制每个局部 CS 的时间覆盖范围，使 MoDecGS 能够基于指定数量的时间段找到最佳的间隔分配。广泛的评估表明，MoDecGS 在来自真实世界动态视频的动态 3D 高斯模型方面，相较于最先进方法，实现了平均模型尺寸减少 70%，同时保持甚至提高了渲染质量。

English

3D Gaussian Splatting (3DGS) has made significant strides in scene representation and neural rendering, with intense efforts focused on adapting it for dynamic scenes. Despite delivering remarkable rendering quality and speed, existing methods struggle with storage demands and representing complex real-world motions. To tackle these issues, we propose MoDecGS, a memory-efficient Gaussian splatting framework designed for reconstructing novel views in challenging scenarios with complex motions. We introduce GlobaltoLocal Motion Decomposition (GLMD) to effectively capture dynamic motions in a coarsetofine manner. This approach leverages Global Canonical Scaffolds (Global CS) and Local Canonical Scaffolds (Local CS), extending static Scaffold representation to dynamic video reconstruction. For Global CS, we propose Global Anchor Deformation (GAD) to efficiently represent global dynamics along complex motions, by directly deforming the implicit Scaffold attributes which are anchor position, offset, and local context features. Next, we finely adjust local motions via the Local Gaussian Deformation (LGD) of Local CS explicitly. Additionally, we introduce Temporal Interval Adjustment (TIA) to automatically control the temporal coverage of each Local CS during training, allowing MoDecGS to find optimal interval assignments based on the specified number of temporal segments. Extensive evaluations demonstrate that MoDecGS achieves an average 70% reduction in model size over stateoftheart methods for dynamic 3D Gaussians from realworld dynamic videos while maintaining or even improving rendering quality.