SPAR3D：從單張圖像穩定地感知重建3D物體

SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images

January 8, 2025

作者: Zixuan Huang, Mark Boss, Aaryaman Vasishta, James M. Rehg, Varun Jampani

cs.AI

摘要

我們研究單張圖像的三維物體重建問題。最近的研究分為兩個方向：基於回歸的建模和生成式建模。回歸方法能有效推斷可見表面，但在被遮擋區域方面表現不佳。生成方法通過建模分佈更好地處理不確定區域，但計算成本高且生成的結果常與可見表面不一致。在本文中，我們提出了SPAR3D，一種新的兩階段方法，旨在兼顧這兩個方向的優勢。SPAR3D的第一階段使用輕量級點擴散模型生成稀疏的三維點雲，具有快速採樣速度。第二階段利用採樣的點雲和輸入圖像創建高度詳細的網格。我們的兩階段設計實現了對單張圖像三維任務的概率建模，同時保持高計算效率和出色的輸出保真度。使用點雲作為中間表示進一步允許互動式用戶編輯。在多樣數據集上評估，SPAR3D展示了優於先前最先進方法的性能，在推理速度為0.7秒。項目頁面連結包含代碼和模型：https://spar3d.github.io

English

We study the problem of single-image 3D object reconstruction. Recent works have diverged into two directions: regression-based modeling and generative modeling. Regression methods efficiently infer visible surfaces, but struggle with occluded regions. Generative methods handle uncertain regions better by modeling distributions, but are computationally expensive and the generation is often misaligned with visible surfaces. In this paper, we present SPAR3D, a novel two-stage approach aiming to take the best of both directions. The first stage of SPAR3D generates sparse 3D point clouds using a lightweight point diffusion model, which has a fast sampling speed. The second stage uses both the sampled point cloud and the input image to create highly detailed meshes. Our two-stage design enables probabilistic modeling of the ill-posed single-image 3D task while maintaining high computational efficiency and great output fidelity. Using point clouds as an intermediate representation further allows for interactive user edits. Evaluated on diverse datasets, SPAR3D demonstrates superior performance over previous state-of-the-art methods, at an inference speed of 0.7 seconds. Project page with code and model: https://spar3d.github.io