动量高斯自蒸馏：用于高质量大场景重建的动量高斯自蒸馏

Momentum-GS: Momentum Gaussian Self-Distillation for High-Quality Large Scene Reconstruction

December 6, 2024

作者: Jixuan Fan, Wanhua Li, Yifei Han, Yansong Tang

cs.AI

摘要

3D高斯点云投影在大规模场景重建中取得了显著成功，但由于高训练内存消耗和存储开销，仍存在挑战。融合隐式和显式特征的混合表示提供了缓解这些限制的途径。然而，在并行化分块训练中应用时，会出现两个关键问题，因为在独立训练每个块时，由于数据多样性降低，重建精度会下降，并且并行训练会限制分割块的数量与可用GPU数量相匹配。为了解决这些问题，我们提出了Momentum-GS，这是一种新颖方法，利用基于动量的自蒸馏来促进各块之间的一致性和准确性，同时将块的数量与物理GPU数量解耦。我们的方法维护一个使用动量更新的教师高斯解码器，确保在训练过程中有一个稳定的参考。这个教师以自蒸馏的方式为每个块提供全局指导，促进重建中的空间一致性。为了进一步确保各块之间的一致性，我们引入了块加权，根据其重建准确性动态调整每个块的权重。在大规模场景上进行的大量实验表明，我们的方法始终优于现有技术，相比CityGaussian，LPIPS提高了12.8%，且分割块数量更少，创立了一个新的技术水平。项目页面：https://jixuan-fan.github.io/Momentum-GS_Page/

English

3D Gaussian Splatting has demonstrated notable success in large-scale scene reconstruction, but challenges persist due to high training memory consumption and storage overhead. Hybrid representations that integrate implicit and explicit features offer a way to mitigate these limitations. However, when applied in parallelized block-wise training, two critical issues arise since reconstruction accuracy deteriorates due to reduced data diversity when training each block independently, and parallel training restricts the number of divided blocks to the available number of GPUs. To address these issues, we propose Momentum-GS, a novel approach that leverages momentum-based self-distillation to promote consistency and accuracy across the blocks while decoupling the number of blocks from the physical GPU count. Our method maintains a teacher Gaussian decoder updated with momentum, ensuring a stable reference during training. This teacher provides each block with global guidance in a self-distillation manner, promoting spatial consistency in reconstruction. To further ensure consistency across the blocks, we incorporate block weighting, dynamically adjusting each block's weight according to its reconstruction accuracy. Extensive experiments on large-scale scenes show that our method consistently outperforms existing techniques, achieving a 12.8% improvement in LPIPS over CityGaussian with much fewer divided blocks and establishing a new state of the art. Project page: https://jixuan-fan.github.io/Momentum-GS_Page/