SelfSplat：无需姿势和无需三维先验的通用三维高斯飞溅

摘要

我们提出了SelfSplat，这是一种新颖的3D高斯点喷模型，旨在从未定位的多视图图像中进行无姿态和无3D先验的通用3D重建。由于缺乏地面真实数据、学习的几何信息以及需要在没有微调的情况下实现准确的3D重建，这些设置本质上是不适定的，因此传统方法很难实现高质量的结果。我们的模型通过有效地将显式3D表示与自监督深度和姿态估计技术相结合，从而在姿态准确性和3D重建质量之间实现相互改进来解决这些挑战。此外，我们还结合了一个具有匹配感知的姿态估计网络和一个深度细化模块，以增强视图间的几何一致性，确保更准确和稳定的3D重建。为了展示我们方法的性能，我们在包括RealEstate10K、ACID和DL3DV在内的大规模真实世界数据集上进行了评估。SelfSplat在外观和几何质量方面均优于先前的最新方法，还展示了强大的跨数据集泛化能力。广泛的消融研究和分析也验证了我们提出方法的有效性。代码和预训练模型可在https://gynjn.github.io/selfsplat/获取。

English

We propose SelfSplat, a novel 3D Gaussian Splatting model designed to perform pose-free and 3D prior-free generalizable 3D reconstruction from unposed multi-view images. These settings are inherently ill-posed due to the lack of ground-truth data, learned geometric information, and the need to achieve accurate 3D reconstruction without finetuning, making it difficult for conventional methods to achieve high-quality results. Our model addresses these challenges by effectively integrating explicit 3D representations with self-supervised depth and pose estimation techniques, resulting in reciprocal improvements in both pose accuracy and 3D reconstruction quality. Furthermore, we incorporate a matching-aware pose estimation network and a depth refinement module to enhance geometry consistency across views, ensuring more accurate and stable 3D reconstructions. To present the performance of our method, we evaluated it on large-scale real-world datasets, including RealEstate10K, ACID, and DL3DV. SelfSplat achieves superior results over previous state-of-the-art methods in both appearance and geometry quality, also demonstrates strong cross-dataset generalization capabilities. Extensive ablation studies and analysis also validate the effectiveness of our proposed methods. Code and pretrained models are available at https://gynjn.github.io/selfsplat/

SelfSplat：无需姿势和无需三维先验的通用三维高斯飞溅

SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting

摘要

Summary

Support

Support