GSTAR：高斯表面跟踪和重建

摘要

3D 高斯飘带技术已经实现了对静态场景的高效逼真渲染。最近的研究将这些方法扩展到支持表面重建和跟踪。然而，使用 3D 高斯方法跟踪动态表面仍然具有挑战性，因为存在复杂的拓扑变化，比如表面的出现、消失或分裂。为了解决这些挑战，我们提出了 GSTAR，这是一种新颖的方法，实现了对具有变化拓扑的一般动态场景的逼真渲染、准确表面重建和可靠的 3D 跟踪。给定多视图捕获作为输入，GSTAR 将高斯绑定到网格面以表示动态对象。对于拓扑一致的表面，GSTAR 保持网格拓扑并使用高斯跟踪网格。在拓扑变化的区域，GSTAR 自适应地将高斯从网格解绑，实现准确的配准并基于这些优化的高斯生成新表面。此外，我们引入了一种基于表面的场景流方法，为帧间跟踪提供了稳健的初始化。实验证明我们的方法有效地跟踪和重建动态表面，实现了一系列应用。我们的项目页面及代码发布可在 https://eth-ait.github.io/GSTAR/ 上找到。

English

3D Gaussian Splatting techniques have enabled efficient photo-realistic rendering of static scenes. Recent works have extended these approaches to support surface reconstruction and tracking. However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians. In regions where topology changes, GSTAR adaptively unbinds Gaussians from the mesh, enabling accurate registration and the generation of new surfaces based on these optimized Gaussians. Additionally, we introduce a surface-based scene flow method that provides robust initialization for tracking between frames. Experiments demonstrate that our method effectively tracks and reconstructs dynamic surfaces, enabling a range of applications. Our project page with the code release is available at https://eth-ait.github.io/GSTAR/.

GSTAR：高斯表面跟踪和重建

GSTAR: Gaussian Surface Tracking and Reconstruction

摘要

Summary

Support