機器人學中的神經場：一項調查

Neural Fields in Robotics: A Survey

October 26, 2024

作者: Muhammad Zubair Irshad, Mauro Comi, Yen-Chen Lin, Nick Heppert, Abhinav Valada, Rares Ambrus, Zsolt Kira, Jonathan Tremblay

cs.AI

摘要

神經場已成為在計算機視覺和機器人領域中進行3D場景表示的一種革命性方法，實現從2D數據中準確推斷幾何、3D語義和動態。利用可微渲染，神經場涵蓋了連續隱式和顯式神經表示，實現高保真度的3D重建，整合多模式感測數據，並生成新的視角。本調查探討了神經場在機器人領域中的應用，強調它們提升感知、規劃和控制能力的潛力。神經場的緊湊性、記憶效率和可微性，以及與基礎和生成模型的無縫整合，使其成為實時應用的理想選擇，提高了機器人的適應性和決策能力。本文對機器人領域中的神經場進行了全面回顧，對超過200篇論文進行了分類，評估了它們的優勢和局限性。首先，我們介紹了四個關鍵的神經場框架：佔據網絡、符號距離場、神經輻射場和高斯點陣化。其次，我們詳細介紹了神經場在五個主要機器人領域的應用：姿態估計、操作、導航、物理和自動駕駛，突出了重要作品，並討論了收穫和面臨的挑戰。最後，我們概述了神經場在機器人領域中目前的局限性，並提出了未來研究的有前景的方向。項目頁面：https://robonerf.github.io

English

Neural Fields have emerged as a transformative approach for 3D scene representation in computer vision and robotics, enabling accurate inference of geometry, 3D semantics, and dynamics from posed 2D data. Leveraging differentiable rendering, Neural Fields encompass both continuous implicit and explicit neural representations enabling high-fidelity 3D reconstruction, integration of multi-modal sensor data, and generation of novel viewpoints. This survey explores their applications in robotics, emphasizing their potential to enhance perception, planning, and control. Their compactness, memory efficiency, and differentiability, along with seamless integration with foundation and generative models, make them ideal for real-time applications, improving robot adaptability and decision-making. This paper provides a thorough review of Neural Fields in robotics, categorizing applications across various domains and evaluating their strengths and limitations, based on over 200 papers. First, we present four key Neural Fields frameworks: Occupancy Networks, Signed Distance Fields, Neural Radiance Fields, and Gaussian Splatting. Second, we detail Neural Fields' applications in five major robotics domains: pose estimation, manipulation, navigation, physics, and autonomous driving, highlighting key works and discussing takeaways and open challenges. Finally, we outline the current limitations of Neural Fields in robotics and propose promising directions for future research. Project page: https://robonerf.github.io