WildLMa：野外環境中的長時間視覺-操作整合

摘要

「野外」移動操作旨在在各種真實世界環境中部署機器人，這需要機器人具備以下能力：（1）具有可以應用於不同物體配置的技能；（2）能夠在多樣環境中執行長期任務；以及（3）執行超越拾取和放置的複雜操作。帶有操縱裝置的四足機器人有望擴展工作空間並實現強大的運動能力，但現有結果並未探討這種能力。本文提出了WildLMa，包括三個組件來解決這些問題：（1）適應於虛擬實境全身遠程操作和可穿越性的學習低層控制器；（2）WildLMa-Skill -- 通過模仿學習或啟發式獲得的通用視覺運動技能庫；以及（3）WildLMa-Planner -- 一個介面，使學習技能可以協調長期任務的低層控制器規劃器。我們通過僅使用數十個示範，在高質量訓練數據的重要性上取得了比現有強化學習基準更高的抓取成功率。WildLMa利用CLIP進行語言條件的模仿學習，從實證上推廣到訓練示範中未見的物體。除了廣泛的定量評估外，我們還在質量上展示了實際的機器人應用，例如在大學走廊或戶外地形中清理垃圾，操作關節物體，以及整理書架上的物品。

English

`In-the-wild' mobile manipulation aims to deploy robots in diverse real-world environments, which requires the robot to (1) have skills that generalize across object configurations; (2) be capable of long-horizon task execution in diverse environments; and (3) perform complex manipulation beyond pick-and-place. Quadruped robots with manipulators hold promise for extending the workspace and enabling robust locomotion, but existing results do not investigate such a capability. This paper proposes WildLMa with three components to address these issues: (1) adaptation of learned low-level controller for VR-enabled whole-body teleoperation and traversability; (2) WildLMa-Skill -- a library of generalizable visuomotor skills acquired via imitation learning or heuristics and (3) WildLMa-Planner -- an interface of learned skills that allow LLM planners to coordinate skills for long-horizon tasks. We demonstrate the importance of high-quality training data by achieving higher grasping success rate over existing RL baselines using only tens of demonstrations. WildLMa exploits CLIP for language-conditioned imitation learning that empirically generalizes to objects unseen in training demonstrations. Besides extensive quantitative evaluation, we qualitatively demonstrate practical robot applications, such as cleaning up trash in university hallways or outdoor terrains, operating articulated objects, and rearranging items on a bookshelf.

WildLMa：野外環境中的長時間視覺-操作整合

WildLMa: Long Horizon Loco-Manipulation in the Wild

摘要

Summary

Support