MAtCha高斯：从稀疏视图获得高质量几何和照片逼真度的图表集

MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views

December 9, 2024

作者: Antoine Guédon, Tomoki Ichikawa, Kohei Yamashita, Ko Nishino

cs.AI

摘要

我们提出了一种新颖的外观模型，可以同时实现明确的高质量3D表面网格恢复和逼真的稀疏视角样本的新视图合成。我们的关键思想是将场景的基础几何网格建模为图表的图集，我们使用二维高斯Surfel（MAtCha高斯）进行渲染。MAtCha从现成的单目深度估计器中提取高频场景表面细节，并通过高斯Surfel渲染进行细化。高斯Surfel会动态附加到图表上，同时满足神经体积渲染的逼真性和网格模型的清晰几何，即在单一模型中实现两个看似矛盾的目标。MAtCha的核心是一种新颖的神经变形模型和一个结构损失，可以保留从学习的单目深度中提取的精细表面细节，同时解决它们的基本尺度模糊问题。广泛的实验验证结果表明，MAtCha在表面重建和逼真度方面达到了与顶尖竞争者相媲美的最新水平，但输入视图数量和计算时间却大幅减少。我们相信MAtCha将成为视觉、图形和机器人领域需要明确几何和逼真度的任何视觉应用的基础工具。我们的项目页面如下：https://anttwo.github.io/matcha/

English

We present a novel appearance model that simultaneously realizes explicit high-quality 3D surface mesh recovery and photorealistic novel view synthesis from sparse view samples. Our key idea is to model the underlying scene geometry Mesh as an Atlas of Charts which we render with 2D Gaussian surfels (MAtCha Gaussians). MAtCha distills high-frequency scene surface details from an off-the-shelf monocular depth estimator and refines it through Gaussian surfel rendering. The Gaussian surfels are attached to the charts on the fly, satisfying photorealism of neural volumetric rendering and crisp geometry of a mesh model, i.e., two seemingly contradicting goals in a single model. At the core of MAtCha lies a novel neural deformation model and a structure loss that preserve the fine surface details distilled from learned monocular depths while addressing their fundamental scale ambiguities. Results of extensive experimental validation demonstrate MAtCha's state-of-the-art quality of surface reconstruction and photorealism on-par with top contenders but with dramatic reduction in the number of input views and computational time. We believe MAtCha will serve as a foundational tool for any visual application in vision, graphics, and robotics that require explicit geometry in addition to photorealism. Our project page is the following: https://anttwo.github.io/matcha/