迈向学习如何完成激光雷达中的全方位感知

摘要

我们提出了CAL（Complete Anything in Lidar）方法，用于基于激光雷达的野外形状补全。这与基于激光雷达的语义/全景场景补全密切相关。然而，现有方法仅能从已有激光雷达数据集中标注的封闭词汇表中补全和识别物体。与之不同，我们的零样本方法利用多模态传感器序列中的时间上下文，挖掘观测物体的形状和语义特征。这些特征随后被提炼为一个仅依赖激光雷达的实例级补全与识别模型。尽管我们仅挖掘了部分形状补全，但我们发现，通过数据集中的多个此类部分观测，我们的提炼模型能够学习推断出完整的物体形状。我们展示了该模型可在语义和全景场景补全的标准基准上进行提示，将物体定位为（非模态）三维边界框，并识别超出固定类别词汇表的物体。项目页面请访问：https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar。

English

We propose CAL (Complete Anything in Lidar) for Lidar-based shape-completion in-the-wild. This is closely related to Lidar-based semantic/panoptic scene completion. However, contemporary methods can only complete and recognize objects from a closed vocabulary labeled in existing Lidar datasets. Different to that, our zero-shot approach leverages the temporal context from multi-modal sensor sequences to mine object shapes and semantic features of observed objects. These are then distilled into a Lidar-only instance-level completion and recognition model. Although we only mine partial shape completions, we find that our distilled model learns to infer full object shapes from multiple such partial observations across the dataset. We show that our model can be prompted on standard benchmarks for Semantic and Panoptic Scene Completion, localize objects as (amodal) 3D bounding boxes, and recognize objects beyond fixed class vocabularies. Our project page is https://research.nvidia.com/labs/dvl/projects/complete-anything-lidar

迈向学习如何完成激光雷达中的全方位感知

Towards Learning to Complete Anything in Lidar

摘要

Summary

Support

Support