WildLMa：野外长时程定位操作

摘要

“野外”移动操作旨在将机器人部署在不同真实环境中，这要求机器人具备以下能力：（1）具有适用于各种物体配置的技能；（2）能够在不同环境中执行长期任务；以及（3）执行超越拾取和放置的复杂操作。具有操纵器的四足机器人有望扩展工作空间并实现强大的移动能力，但现有结果并未探究这种能力。本文提出了WildLMa，包括三个组成部分来解决这些问题：（1）为VR启用的全身遥操作和可穿越性而调整学习的低级控制器；（2）WildLMa-Skill——通过模仿学习或启发式获得的通用视觉运动技能库；以及（3）WildLMa-Planner——一个接口，允许LLM规划器协调长期任务所需的技能。我们通过仅使用少量演示，在高质量训练数据的重要性上取得了比现有RL基线更高的抓取成功率。WildLMa利用CLIP进行语言条件的模仿学习，经验性地推广到训练演示中未见的物体。除了广泛的定量评估外，我们还在定性上展示了实际的机器人应用，例如清理大学走廊或户外地形中的垃圾，操作关节对象，以及整理书架上的物品。

English

`In-the-wild' mobile manipulation aims to deploy robots in diverse real-world environments, which requires the robot to (1) have skills that generalize across object configurations; (2) be capable of long-horizon task execution in diverse environments; and (3) perform complex manipulation beyond pick-and-place. Quadruped robots with manipulators hold promise for extending the workspace and enabling robust locomotion, but existing results do not investigate such a capability. This paper proposes WildLMa with three components to address these issues: (1) adaptation of learned low-level controller for VR-enabled whole-body teleoperation and traversability; (2) WildLMa-Skill -- a library of generalizable visuomotor skills acquired via imitation learning or heuristics and (3) WildLMa-Planner -- an interface of learned skills that allow LLM planners to coordinate skills for long-horizon tasks. We demonstrate the importance of high-quality training data by achieving higher grasping success rate over existing RL baselines using only tens of demonstrations. WildLMa exploits CLIP for language-conditioned imitation learning that empirically generalizes to objects unseen in training demonstrations. Besides extensive quantitative evaluation, we qualitatively demonstrate practical robot applications, such as cleaning up trash in university hallways or outdoor terrains, operating articulated objects, and rearranging items on a bookshelf.

WildLMa：野外长时程定位操作

WildLMa: Long Horizon Loco-Manipulation in the Wild

摘要

Summary

Support

Support