邁向學習完成LiDAR中的任何任務

Towards Learning to Complete Anything in Lidar

April 16, 2025

作者: Ayca Takmaz, Cristiano Saltori, Neehar Peri, Tim Meinhardt, Riccardo de Lutio, Laura Leal-Taixé, Aljoša Ošep

cs.AI

摘要

我們提出了CAL（Complete Anything in Lidar）方法，用於基於激光雷達的野外形狀補全。這與基於激光雷達的語義/全景場景補全密切相關。然而，現有方法僅能從現有激光雷達數據集中標註的封閉詞彙表中完成和識別物體。與此不同，我們的零樣本方法利用多模態傳感器序列的時間上下文來挖掘觀測物體的形狀和語義特徵。這些特徵隨後被提煉成一個僅依賴激光雷達的實例級補全與識別模型。儘管我們只挖掘了部分形狀補全，但我們發現，通過數據集中多個此類部分觀測，我們的提煉模型能夠學習推斷出完整的物體形狀。我們展示了該模型可在語義和全景場景補全的標準基準上進行提示，將物體定位為（無模態）3D邊界框，並識別超出固定類別詞彙表的物體。我們的項目頁面為https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar。

English

We propose CAL (Complete Anything in Lidar) for Lidar-based shape-completion in-the-wild. This is closely related to Lidar-based semantic/panoptic scene completion. However, contemporary methods can only complete and recognize objects from a closed vocabulary labeled in existing Lidar datasets. Different to that, our zero-shot approach leverages the temporal context from multi-modal sensor sequences to mine object shapes and semantic features of observed objects. These are then distilled into a Lidar-only instance-level completion and recognition model. Although we only mine partial shape completions, we find that our distilled model learns to infer full object shapes from multiple such partial observations across the dataset. We show that our model can be prompted on standard benchmarks for Semantic and Panoptic Scene Completion, localize objects as (amodal) 3D bounding boxes, and recognize objects beyond fixed class vocabularies. Our project page is https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar