SelfSplat：無需姿勢和三維先驗的通用化三維高斯點陣化

摘要

我們提出了SelfSplat，一種新穎的3D高斯Splatting模型，旨在從未擺姿勢的多視圖圖像中執行無姿勢和無3D先驗的通用3D重建。這些設置由於缺乏地面真實數據、學習的幾何信息以及需要實現精確的3D重建而無需微調，因此本質上是不透明的，這使得傳統方法難以實現高質量的結果。我們的模型通過有效地將明確的3D表示與自監督深度和姿勢估計技術相結合，從而實現姿勢準確性和3D重建質量的相互改進。此外，我們還結合了一個匹配感知的姿勢估計網絡和深度細化模塊，以增強視圖之間的幾何一致性，確保更準確和穩定的3D重建。為了展示我們方法的性能，我們在大規模真實世界數據集上進行了評估，包括RealEstate10K、ACID和DL3DV。SelfSplat在外觀和幾何質量方面均優於先前的最先進方法，同時展現了強大的跨數據集泛化能力。廣泛的消融研究和分析也驗證了我們提出方法的有效性。代碼和預訓練模型可在https://gynjn.github.io/selfsplat/上找到。

English

We propose SelfSplat, a novel 3D Gaussian Splatting model designed to perform pose-free and 3D prior-free generalizable 3D reconstruction from unposed multi-view images. These settings are inherently ill-posed due to the lack of ground-truth data, learned geometric information, and the need to achieve accurate 3D reconstruction without finetuning, making it difficult for conventional methods to achieve high-quality results. Our model addresses these challenges by effectively integrating explicit 3D representations with self-supervised depth and pose estimation techniques, resulting in reciprocal improvements in both pose accuracy and 3D reconstruction quality. Furthermore, we incorporate a matching-aware pose estimation network and a depth refinement module to enhance geometry consistency across views, ensuring more accurate and stable 3D reconstructions. To present the performance of our method, we evaluated it on large-scale real-world datasets, including RealEstate10K, ACID, and DL3DV. SelfSplat achieves superior results over previous state-of-the-art methods in both appearance and geometry quality, also demonstrates strong cross-dataset generalization capabilities. Extensive ablation studies and analysis also validate the effectiveness of our proposed methods. Code and pretrained models are available at https://gynjn.github.io/selfsplat/

SelfSplat：無需姿勢和三維先驗的通用化三維高斯點陣化

SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting

摘要

Summary

Support