Vista3D：揭開單張影像的3D黑暗面

Vista3D: Unravel the 3D Darkside of a Single Image

September 18, 2024

作者: Qiuhong Shen, Xingyi Yang, Michael Bi Mi, Xinchao Wang

cs.AI

摘要

我們踏上古老的探索之旅：從僅僅一瞥可見部分揭示物體的隱藏維度。為了應對這一挑戰，我們提出了Vista3D，一個能夠在短短5分鐘內實現快速且一致的3D生成的框架。Vista3D的核心是一種兩階段方法：粗略階段和精細階段。在粗略階段，我們從單張圖像中使用高斯Splatting快速生成初始幾何形狀。在精細階段，我們直接從學習到的高斯Splatting中提取一個符號距離函數（SDF），並通過可微的等值面表示進行優化。此外，它通過使用兩個獨立的隱式函數來捕捉物體的可見和隱藏部分，提高了生成的質量。此外，通過角度擴散先驗合成將2D擴散先驗的梯度與3D感知擴散先驗進行協調。通過廣泛的評估，我們展示了Vista3D有效地在生成的3D物體之間維持了一個一致性和多樣性的平衡。演示和代碼將在https://github.com/florinshen/Vista3D 上提供。

English

We embark on the age-old quest: unveiling the hidden dimensions of objects from mere glimpses of their visible parts. To address this, we present Vista3D, a framework that realizes swift and consistent 3D generation within a mere 5 minutes. At the heart of Vista3D lies a two-phase approach: the coarse phase and the fine phase. In the coarse phase, we rapidly generate initial geometry with Gaussian Splatting from a single image. In the fine phase, we extract a Signed Distance Function (SDF) directly from learned Gaussian Splatting, optimizing it with a differentiable isosurface representation. Furthermore, it elevates the quality of generation by using a disentangled representation with two independent implicit functions to capture both visible and obscured aspects of objects. Additionally, it harmonizes gradients from 2D diffusion prior with 3D-aware diffusion priors by angular diffusion prior composition. Through extensive evaluation, we demonstrate that Vista3D effectively sustains a balance between the consistency and diversity of the generated 3D objects. Demos and code will be available at https://github.com/florinshen/Vista3D.