3D凸壳点渲染：使用3D平滑凸壳的辐射场渲染

3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes

November 22, 2024

作者: Jan Held, Renaud Vandeghen, Abdullah Hamdi, Adrien Deliege, Anthony Cioppa, Silvio Giancola, Andrea Vedaldi, Bernard Ghanem, Marc Van Droogenbroeck

cs.AI

摘要

最近在辐射场重建方面取得的进展，如3D高斯飞溅（3DGS），通过用高斯基元的组合表示场景，实现了高质量的新视角合成和快速渲染。然而，3D高斯存在一些限制用于场景重建。在不显著增加高斯数量的情况下准确捕捉硬边缘是具有挑战性的，这会导致较大的内存占用。此外，它们难以表示平坦表面，因为它们在空间中扩散。没有手工制作的正则化器，它们往往会在实际表面周围不规则地分散。为了规避这些问题，我们引入了一种名为3D凸飞溅（3DCS）的新方法，它利用3D光滑凸体作为基元，从多视图图像中建模几何意义的辐射场。光滑凸形状比高斯更灵活，可以更好地表示具有硬边缘和密集体积的3D场景，使用更少的基元。借助我们高效的基于CUDA的光栅化器，3DCS在Mip-NeRF360、坦克与庙宇和深度混合等基准测试中比3DGS表现出更优异的性能。具体而言，我们的方法在PSNR上提高了高达0.81，在LPIPS上提高了0.026，同时保持高渲染速度并减少所需基元的数量。我们的结果突显了3D凸飞溅成为高质量场景重建和新视角合成的新标准的潜力。项目页面：convexsplatting.github.io。

English

Recent advances in radiance field reconstruction, such as 3D Gaussian Splatting (3DGS), have achieved high-quality novel view synthesis and fast rendering by representing scenes with compositions of Gaussian primitives. However, 3D Gaussians present several limitations for scene reconstruction. Accurately capturing hard edges is challenging without significantly increasing the number of Gaussians, creating a large memory footprint. Moreover, they struggle to represent flat surfaces, as they are diffused in space. Without hand-crafted regularizers, they tend to disperse irregularly around the actual surface. To circumvent these issues, we introduce a novel method, named 3D Convex Splatting (3DCS), which leverages 3D smooth convexes as primitives for modeling geometrically-meaningful radiance fields from multi-view images. Smooth convex shapes offer greater flexibility than Gaussians, allowing for a better representation of 3D scenes with hard edges and dense volumes using fewer primitives. Powered by our efficient CUDA-based rasterizer, 3DCS achieves superior performance over 3DGS on benchmarks such as Mip-NeRF360, Tanks and Temples, and Deep Blending. Specifically, our method attains an improvement of up to 0.81 in PSNR and 0.026 in LPIPS compared to 3DGS while maintaining high rendering speeds and reducing the number of required primitives. Our results highlight the potential of 3D Convex Splatting to become the new standard for high-quality scene reconstruction and novel view synthesis. Project page: convexsplatting.github.io.