MoDec-GS：全域到局部運動分解與時間間隔調整，用於緊湊動態3D高斯濺射。

MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting

January 7, 2025

作者: Sangwoon Kwak, Joonsoo Kim, Jun Young Jeong, Won-Sik Cheong, Jihyong Oh, Munchurl Kim

cs.AI

摘要

3D 高斯點陣化（3DGS）在場景表示和神經渲染方面取得了顯著進展，並且積極致力於將其應用於動態場景。儘管現有方法在提供卓越的渲染質量和速度方面取得了成功，但在存儲需求和表示複雜現實世界運動方面仍然存在困難。為了應對這些問題，我們提出了MoDecGS，這是一個記憶效率高的高斯點陣化框架，旨在重建具有複雜運動的具有挑戰性場景中的新視圖。我們引入了全局到局部運動分解（GLMD），以有效地以粗到細的方式捕捉動態運動。這種方法利用全局標準支架（Global CS）和局部標準支架（Local CS），將靜態支架表示擴展到動態視頻重建。對於全局 CS，我們提出全局錨點變形（GAD）來有效地表示沿複雜運動的全局動態，通過直接變形隱式支架屬性，這些屬性是錨點位置、偏移和局部上下文特徵。接下來，我們通過局部高斯變形（LGD）明確調整局部運動。此外，我們引入了時間間隔調整（TIA）來在訓練期間自動控制每個局部 CS 的時間覆蓋範圍，使 MoDecGS 能夠基於指定的時間段數找到最佳的時間間隔分配。廣泛的評估表明，MoDecGS 在來自現實世界動態視頻的動態 3D 高斯模型方面，相對於最先進的方法，實現了平均模型大小減少 70％，同時保持甚至提高了渲染質量。

English

3D Gaussian Splatting (3DGS) has made significant strides in scene representation and neural rendering, with intense efforts focused on adapting it for dynamic scenes. Despite delivering remarkable rendering quality and speed, existing methods struggle with storage demands and representing complex real-world motions. To tackle these issues, we propose MoDecGS, a memory-efficient Gaussian splatting framework designed for reconstructing novel views in challenging scenarios with complex motions. We introduce GlobaltoLocal Motion Decomposition (GLMD) to effectively capture dynamic motions in a coarsetofine manner. This approach leverages Global Canonical Scaffolds (Global CS) and Local Canonical Scaffolds (Local CS), extending static Scaffold representation to dynamic video reconstruction. For Global CS, we propose Global Anchor Deformation (GAD) to efficiently represent global dynamics along complex motions, by directly deforming the implicit Scaffold attributes which are anchor position, offset, and local context features. Next, we finely adjust local motions via the Local Gaussian Deformation (LGD) of Local CS explicitly. Additionally, we introduce Temporal Interval Adjustment (TIA) to automatically control the temporal coverage of each Local CS during training, allowing MoDecGS to find optimal interval assignments based on the specified number of temporal segments. Extensive evaluations demonstrate that MoDecGS achieves an average 70% reduction in model size over stateoftheart methods for dynamic 3D Gaussians from realworld dynamic videos while maintaining or even improving rendering quality.