机器人学中的神经场：一项调查

Neural Fields in Robotics: A Survey

October 26, 2024

作者: Muhammad Zubair Irshad, Mauro Comi, Yen-Chen Lin, Nick Heppert, Abhinav Valada, Rares Ambrus, Zsolt Kira, Jonathan Tremblay

cs.AI

摘要

神经场已经成为计算机视觉和机器人领域中三维场景表示的一种革命性方法，能够从姿态2D数据中准确推断几何、三维语义和动态。利用可微渲染，神经场涵盖了连续隐式和显式神经表示，实现了高保真的三维重建、多模态传感器数据的集成以及新视角的生成。本调查探讨了它们在机器人领域的应用，强调了它们提升感知、规划和控制能力的潜力。神经场的紧凑性、内存效率和可微性，以及与基础和生成模型的无缝集成，使其非常适合实时应用，提高了机器人的适应性和决策能力。本文全面审视了神经场在机器人领域的应用，根据200多篇论文对其在各个领域的应用进行了分类，并评估了其优势和局限性。首先，我们介绍了四个关键的神经场框架：占据网络、有符号距离场、神经辐射场和高斯飞溅。其次，我们详细介绍了神经场在五个主要机器人领域的应用：姿态估计、操作、导航、物理和自动驾驶，重点介绍了关键作品，并讨论了经验教训和面临的挑战。最后，我们概述了神经场在机器人领域的当前局限性，并提出了未来研究的有前景的方向。项目页面：https://robonerf.github.io

English

Neural Fields have emerged as a transformative approach for 3D scene representation in computer vision and robotics, enabling accurate inference of geometry, 3D semantics, and dynamics from posed 2D data. Leveraging differentiable rendering, Neural Fields encompass both continuous implicit and explicit neural representations enabling high-fidelity 3D reconstruction, integration of multi-modal sensor data, and generation of novel viewpoints. This survey explores their applications in robotics, emphasizing their potential to enhance perception, planning, and control. Their compactness, memory efficiency, and differentiability, along with seamless integration with foundation and generative models, make them ideal for real-time applications, improving robot adaptability and decision-making. This paper provides a thorough review of Neural Fields in robotics, categorizing applications across various domains and evaluating their strengths and limitations, based on over 200 papers. First, we present four key Neural Fields frameworks: Occupancy Networks, Signed Distance Fields, Neural Radiance Fields, and Gaussian Splatting. Second, we detail Neural Fields' applications in five major robotics domains: pose estimation, manipulation, navigation, physics, and autonomous driving, highlighting key works and discussing takeaways and open challenges. Finally, we outline the current limitations of Neural Fields in robotics and propose promising directions for future research. Project page: https://robonerf.github.io