動量-高斯自我蒸餾：用於高品質大場景重建

Momentum-GS: Momentum Gaussian Self-Distillation for High-Quality Large Scene Reconstruction

December 6, 2024

作者: Jixuan Fan, Wanhua Li, Yifei Han, Yansong Tang

cs.AI

摘要

3D高斯點陣在大規模場景重建中取得顯著成功，但由於高訓練記憶體消耗和存儲開銷，仍存在挑戰。整合隱式和顯式特徵的混合表示提供了一種減輕這些限制的方法。然而，在並行塊訓練中應用時，會出現兩個關鍵問題，因為當獨立訓練每個塊時，由於數據多樣性減少，重建準確性會下降，並且並行訓練會限制分割塊的數量與可用GPU數量相等。為了應對這些問題，我們提出了Momentum-GS，一種利用基於動量的自蒸餾來促進塊之間一致性和準確性的新方法，同時將塊的數量與物理GPU數量解耦。我們的方法維護一個使用動量更新的教師高斯解碼器，在訓練過程中確保穩定的參考。這個教師以自蒸餾方式為每個塊提供全局引導，促進重建中的空間一致性。為了進一步確保塊之間的一致性，我們引入塊加權，根據其重建準確性動態調整每個塊的權重。在大規模場景上進行的大量實驗表明，我們的方法始終優於現有技術，相對於CityGaussian，LPIPS提高了12.8%，並且使用更少的分割塊，建立了新的技術水平。項目頁面：https://jixuan-fan.github.io/Momentum-GS_Page/

English

3D Gaussian Splatting has demonstrated notable success in large-scale scene reconstruction, but challenges persist due to high training memory consumption and storage overhead. Hybrid representations that integrate implicit and explicit features offer a way to mitigate these limitations. However, when applied in parallelized block-wise training, two critical issues arise since reconstruction accuracy deteriorates due to reduced data diversity when training each block independently, and parallel training restricts the number of divided blocks to the available number of GPUs. To address these issues, we propose Momentum-GS, a novel approach that leverages momentum-based self-distillation to promote consistency and accuracy across the blocks while decoupling the number of blocks from the physical GPU count. Our method maintains a teacher Gaussian decoder updated with momentum, ensuring a stable reference during training. This teacher provides each block with global guidance in a self-distillation manner, promoting spatial consistency in reconstruction. To further ensure consistency across the blocks, we incorporate block weighting, dynamically adjusting each block's weight according to its reconstruction accuracy. Extensive experiments on large-scale scenes show that our method consistently outperforms existing techniques, achieving a 12.8% improvement in LPIPS over CityGaussian with much fewer divided blocks and establishing a new state of the art. Project page: https://jixuan-fan.github.io/Momentum-GS_Page/