CANVAS：具備常識感知的直观人机导航系统

CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot Interaction

October 2, 2024

作者: Suhwan Choi, Yongjun Cho, Minchan Kim, Jaeyoon Jung, Myunchul Joe, Yubeen Park, Minseo Kim, Sungwoong Kim, Sungjae Lee, Hwiseong Park, Jiwan Chung, Youngjae Yu

cs.AI

摘要

現實生活中的機器人導航不僅僅是到達目的地；它需要在解決特定情境目標的同時優化移動。人類表達這些目標的直觀方式是通過抽象提示，如口頭命令或粗略草圖。這樣的人類引導可能缺乏細節或存在噪音。儘管如此，我們期望機器人能按照預期進行導航。為了讓機器人能夠解釋並執行這些與人類期望一致的抽象指令，它們必須與人類共享對基本導航概念的共同理解。為此，我們介紹了CANVAS，一個結合視覺和語言指令的常識感知導航新框架。它的成功來自於模仿學習，使機器人能夠從人類導航行為中學習。我們提出了COMMAND，一個包含人類標註的導航結果的全面數據集，涵蓋48小時和219公里，旨在訓練在模擬環境中的常識感知導航系統。我們的實驗表明，CANVAS在所有環境中均優於強大的基於規則的系統ROS NavStack，展現出對於噪音指令的優越性能。值得注意的是，在果園環境中，ROS NavStack記錄了0%的總成功率，而CANVAS實現了67%的總成功率。CANVAS還在未知環境中與人類示範和常識約束密切一致。此外，CANVAS的實際部署展示了令人印象深刻的Sim2Real轉移，總成功率達到69%，突顯了從模擬環境中學習人類示範對於現實應用的潛力。

English

Real-life robot navigation involves more than just reaching a destination; it requires optimizing movements while addressing scenario-specific goals. An intuitive way for humans to express these goals is through abstract cues like verbal commands or rough sketches. Such human guidance may lack details or be noisy. Nonetheless, we expect robots to navigate as intended. For robots to interpret and execute these abstract instructions in line with human expectations, they must share a common understanding of basic navigation concepts with humans. To this end, we introduce CANVAS, a novel framework that combines visual and linguistic instructions for commonsense-aware navigation. Its success is driven by imitation learning, enabling the robot to learn from human navigation behavior. We present COMMAND, a comprehensive dataset with human-annotated navigation results, spanning over 48 hours and 219 km, designed to train commonsense-aware navigation systems in simulated environments. Our experiments show that CANVAS outperforms the strong rule-based system ROS NavStack across all environments, demonstrating superior performance with noisy instructions. Notably, in the orchard environment, where ROS NavStack records a 0% total success rate, CANVAS achieves a total success rate of 67%. CANVAS also closely aligns with human demonstrations and commonsense constraints, even in unseen environments. Furthermore, real-world deployment of CANVAS showcases impressive Sim2Real transfer with a total success rate of 69%, highlighting the potential of learning from human demonstrations in simulated environments for real-world applications.