3DGS-DET：為了3D物體偵測加強3D高斯飛濺技術，並結合邊界引導和以框為焦點的取樣。

3DGS-DET: Empower 3D Gaussian Splatting with Boundary Guidance and Box-Focused Sampling for 3D Object Detection

October 2, 2024

作者: Yang Cao, Yuanliang Jv, Dan Xu

cs.AI

摘要

神經輻射場（NeRF）被廣泛應用於新視角合成，並已為三維物體檢測（3DOD）進行了調整，提供了一種有前途的通過視角合成表示進行3DOD的方法。然而，NeRF面臨著固有限制：（i）由於其隱式性質，對於3DOD的表徵能力有限，以及（ii）渲染速度緩慢。最近，三維高斯飛濺（3DGS）作為一種明確的三維表示出現，解決了這些限制。受到這些優勢的啟發，本文首次將3DGS引入3DOD，確定了兩個主要挑戰：（i）高斯斑點的模糊空間分佈：3DGS主要依賴於2D像素級監督，導致高斯斑點的三維空間分佈不清晰，對象和背景之間的區分不清，這妨礙了3DOD；（ii）過多的背景斑點：2D圖像通常包含眾多背景像素，導致密集重建的3DGS具有許多表示背景的噪聲高斯斑點，對檢測產生負面影響。為應對挑戰（i），我們利用3DGS重建源自2D圖像的事實，提出了一個優雅且高效的解決方案，通過將2D邊界引導納入其中，顯著增強了高斯斑點的空間分佈，使對象和其背景之間的區分更加清晰。為應對挑戰（ii），我們提出了一種使用2D框框的框焦點採樣策略，以在三維空間中生成對象概率分佈，實現在三維空間中的有效概率採樣，保留更多對象斑點並減少噪聲背景斑點。由於我們的設計，我們的3DGS-DET在ScanNet數據集上明顯優於SOTA NeRF-based方法NeRF-Det，[email protected]提高了+6.6，[email protected]提高了+8.1，在ARKITScenes數據集上[email protected]驚人提高了+31.5。

English

Neural Radiance Fields (NeRF) are widely used for novel-view synthesis and have been adapted for 3D Object Detection (3DOD), offering a promising approach to 3DOD through view-synthesis representation. However, NeRF faces inherent limitations: (i) limited representational capacity for 3DOD due to its implicit nature, and (ii) slow rendering speeds. Recently, 3D Gaussian Splatting (3DGS) has emerged as an explicit 3D representation that addresses these limitations. Inspired by these advantages, this paper introduces 3DGS into 3DOD for the first time, identifying two main challenges: (i) Ambiguous spatial distribution of Gaussian blobs: 3DGS primarily relies on 2D pixel-level supervision, resulting in unclear 3D spatial distribution of Gaussian blobs and poor differentiation between objects and background, which hinders 3DOD; (ii) Excessive background blobs: 2D images often include numerous background pixels, leading to densely reconstructed 3DGS with many noisy Gaussian blobs representing the background, negatively affecting detection. To tackle the challenge (i), we leverage the fact that 3DGS reconstruction is derived from 2D images, and propose an elegant and efficient solution by incorporating 2D Boundary Guidance to significantly enhance the spatial distribution of Gaussian blobs, resulting in clearer differentiation between objects and their background. To address the challenge (ii), we propose a Box-Focused Sampling strategy using 2D boxes to generate object probability distribution in 3D spaces, allowing effective probabilistic sampling in 3D to retain more object blobs and reduce noisy background blobs. Benefiting from our designs, our 3DGS-DET significantly outperforms the SOTA NeRF-based method, NeRF-Det, achieving improvements of +6.6 on [email protected] and +8.1 on [email protected] for the ScanNet dataset, and impressive +31.5 on [email protected] for the ARKITScenes dataset.