RobustDexGrasp：基于单视角感知的通用物体稳健灵巧抓取

RobustDexGrasp: Robust Dexterous Grasping of General Objects from Single-view Perception

April 7, 2025

作者: Hui Zhang, Zijian Wu, Linyi Huang, Sammy Christen, Jie Song

cs.AI

摘要

从单视角感知中稳健抓取各类物体是灵巧机器人的基础能力。以往研究多依赖于完全可观测的物体、专家示范或静态抓取姿态，这限制了其泛化能力和对外界干扰的适应性。本文提出了一种基于强化学习的框架，实现了从单视角感知对多种未见物体的零样本动态灵巧抓取，同时能对外界干扰做出自适应动作。我们采用以手为中心的物体表征方法进行形状特征提取，着重关注与交互相关的局部形状，从而增强对形状变化和不确定性的鲁棒性。为使手部在有限观测条件下有效适应干扰，我们提出了一种混合课程学习策略：首先利用模仿学习提炼出基于特权实时视觉-触觉反馈训练的策略，然后逐步过渡到强化学习，在观测噪声和动态随机化引起的干扰下学习自适应动作。实验结果表明，该方法在随机姿态下抓取未见物体时展现出强大的泛化能力，在247,786个模拟物体上取得了97.0%的成功率，在512个真实物体上达到了94.6%的成功率。通过定量和定性评估，我们还验证了该方法对各类干扰（包括未观测到的物体移动和外部力）的鲁棒性。项目页面：https://zdchan.github.io/Robust_DexGrasp/

English

Robust grasping of various objects from single-view perception is fundamental for dexterous robots. Previous works often rely on fully observable objects, expert demonstrations, or static grasping poses, which restrict their generalization ability and adaptability to external disturbances. In this paper, we present a reinforcement-learning-based framework that enables zero-shot dynamic dexterous grasping of a wide range of unseen objects from single-view perception, while performing adaptive motions to external disturbances. We utilize a hand-centric object representation for shape feature extraction that emphasizes interaction-relevant local shapes, enhancing robustness to shape variance and uncertainty. To enable effective hand adaptation to disturbances with limited observations, we propose a mixed curriculum learning strategy, which first utilizes imitation learning to distill a policy trained with privileged real-time visual-tactile feedback, and gradually transfers to reinforcement learning to learn adaptive motions under disturbances caused by observation noises and dynamic randomization. Our experiments demonstrate strong generalization in grasping unseen objects with random poses, achieving success rates of 97.0% across 247,786 simulated objects and 94.6% across 512 real objects. We also demonstrate the robustness of our method to various disturbances, including unobserved object movement and external forces, through both quantitative and qualitative evaluations. Project Page: https://zdchan.github.io/Robust_DexGrasp/