ManipTrans：通过残差学习实现高效灵巧双手操作迁移

ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning

March 27, 2025

作者: Kailin Li, Puhao Li, Tengyu Liu, Yuyang Li, Siyuan Huang

cs.AI

摘要

人类双手在交互中扮演着核心角色，这推动了对灵巧机器人操控研究的不断深入。数据驱动的具身AI算法需要精确、大规模且类人的操作序列，而传统的强化学习或现实世界的遥操作难以满足这一需求。为此，我们提出了ManipTrans，一种新颖的两阶段方法，旨在高效地将人类双手技能迁移至模拟环境中的灵巧机器人手。ManipTrans首先预训练一个通用轨迹模仿器以模拟手部运动，随后在交互约束下微调特定的残差模块，从而实现复杂双手任务的高效学习与精准执行。实验表明，ManipTrans在成功率、保真度和效率上均超越了现有最先进方法。借助ManipTrans，我们将多个手-物体数据集迁移至机器人手，构建了DexManipNet，这是一个大规模数据集，涵盖了诸如笔帽盖合和瓶盖旋开等先前未探索的任务。DexManipNet包含3.3K个机器人操控片段，且易于扩展，为灵巧手的进一步策略训练提供了便利，并支持实际场景的部署应用。

English

Human hands play a central role in interacting, motivating increasing research in dexterous robotic manipulation. Data-driven embodied AI algorithms demand precise, large-scale, human-like manipulation sequences, which are challenging to obtain with conventional reinforcement learning or real-world teleoperation. To address this, we introduce ManipTrans, a novel two-stage method for efficiently transferring human bimanual skills to dexterous robotic hands in simulation. ManipTrans first pre-trains a generalist trajectory imitator to mimic hand motion, then fine-tunes a specific residual module under interaction constraints, enabling efficient learning and accurate execution of complex bimanual tasks. Experiments show that ManipTrans surpasses state-of-the-art methods in success rate, fidelity, and efficiency. Leveraging ManipTrans, we transfer multiple hand-object datasets to robotic hands, creating DexManipNet, a large-scale dataset featuring previously unexplored tasks like pen capping and bottle unscrewing. DexManipNet comprises 3.3K episodes of robotic manipulation and is easily extensible, facilitating further policy training for dexterous hands and enabling real-world deployments.