3D凸面片：使用3D光滑凸面渲染辐射场

3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes

November 22, 2024

作者: Jan Held, Renaud Vandeghen, Abdullah Hamdi, Adrien Deliege, Anthony Cioppa, Silvio Giancola, Andrea Vedaldi, Bernard Ghanem, Marc Van Droogenbroeck

cs.AI

摘要

最近在輝度場重建方面取得的進展，如三維高斯飛濺（3DGS），通過以高斯基元素的組合來表示場景，實現了高質量的新視角合成和快速渲染。然而，三維高斯存在幾個限制，對於準確捕捉硬邊緣而不顯著增加高斯數量以減少記憶體佔用是一項挑戰。此外，它們難以表示平面表面，因為它們在空間中擴散。在沒有手工設計的正則化器的情況下，它們往往會在實際表面周圍不規則地分散。為了避免這些問題，我們引入了一種新方法，名為三維凸飛濺（3DCS），它利用三維平滑凸形作為基元素，從多視圖圖像中建模幾何有意義的輝度場。平滑的凸形形狀比高斯更具靈活性，可以更好地表示具有硬邊緣和密集體積的三維場景，並使用更少的基元素。憑藉我們高效的基於CUDA的光柵化器，3DCS在Mip-NeRF360、Tanks and Temples和Deep Blending等基準測試中實現了優異性能。具體而言，我們的方法在PSNR和LPIPS方面相對於3DGS取得了高達0.81和0.026的改善，同時保持高渲染速度並減少所需基元素的數量。我們的結果突顯了三維凸飛濺成為高質量場景重建和新視角合成的新標準的潛力。項目頁面：convexsplatting.github.io。

English

Recent advances in radiance field reconstruction, such as 3D Gaussian Splatting (3DGS), have achieved high-quality novel view synthesis and fast rendering by representing scenes with compositions of Gaussian primitives. However, 3D Gaussians present several limitations for scene reconstruction. Accurately capturing hard edges is challenging without significantly increasing the number of Gaussians, creating a large memory footprint. Moreover, they struggle to represent flat surfaces, as they are diffused in space. Without hand-crafted regularizers, they tend to disperse irregularly around the actual surface. To circumvent these issues, we introduce a novel method, named 3D Convex Splatting (3DCS), which leverages 3D smooth convexes as primitives for modeling geometrically-meaningful radiance fields from multi-view images. Smooth convex shapes offer greater flexibility than Gaussians, allowing for a better representation of 3D scenes with hard edges and dense volumes using fewer primitives. Powered by our efficient CUDA-based rasterizer, 3DCS achieves superior performance over 3DGS on benchmarks such as Mip-NeRF360, Tanks and Temples, and Deep Blending. Specifically, our method attains an improvement of up to 0.81 in PSNR and 0.026 in LPIPS compared to 3DGS while maintaining high rendering speeds and reducing the number of required primitives. Our results highlight the potential of 3D Convex Splatting to become the new standard for high-quality scene reconstruction and novel view synthesis. Project page: convexsplatting.github.io.