MAtCha高斯：從稀疏視圖中獲得高質量幾何和照片逼真度的圖表集

摘要

我們提出了一種新穎的外觀模型，同時實現了明確的高質量3D表面網格恢復和逼真的從稀疏視角樣本合成新視圖。我們的關鍵思想是將底層場景幾何形狀模型化為一個圖表的集合，我們使用2D高斯surfel（MAtCha高斯）來渲染它。MAtCha從現成的單眼深度估算器中提取高頻場景表面細節，並通過高斯surfel渲染進行精煉。高斯surfel會動態附加到圖表上，滿足神經體積渲染的逼真性和網格模型的清晰幾何形狀，即在單一模型中實現了兩個看似矛盾的目標。MAtCha的核心是一種新穎的神經變形模型和一種結構損失，保留了從學習的單眼深度中提取的細微表面細節，同時解決了它們的基本尺度模糊問題。廣泛的實驗驗證結果表明，MAtCha在表面重建和逼真方面具有與頂尖競爭對手相當的最新水準，但輸入視圖數量和計算時間大幅減少。我們相信MAtCha將成為視覺、圖形和機器人領域中任何需要明確幾何形狀和逼真外觀的應用的基礎工具。我們的項目頁面如下：https://anttwo.github.io/matcha/

English

We present a novel appearance model that simultaneously realizes explicit high-quality 3D surface mesh recovery and photorealistic novel view synthesis from sparse view samples. Our key idea is to model the underlying scene geometry Mesh as an Atlas of Charts which we render with 2D Gaussian surfels (MAtCha Gaussians). MAtCha distills high-frequency scene surface details from an off-the-shelf monocular depth estimator and refines it through Gaussian surfel rendering. The Gaussian surfels are attached to the charts on the fly, satisfying photorealism of neural volumetric rendering and crisp geometry of a mesh model, i.e., two seemingly contradicting goals in a single model. At the core of MAtCha lies a novel neural deformation model and a structure loss that preserve the fine surface details distilled from learned monocular depths while addressing their fundamental scale ambiguities. Results of extensive experimental validation demonstrate MAtCha's state-of-the-art quality of surface reconstruction and photorealism on-par with top contenders but with dramatic reduction in the number of input views and computational time. We believe MAtCha will serve as a foundational tool for any visual application in vision, graphics, and robotics that require explicit geometry in addition to photorealism. Our project page is the following: https://anttwo.github.io/matcha/

MAtCha高斯：從稀疏視圖中獲得高質量幾何和照片逼真度的圖表集

MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views

摘要

Summary

Support