TokenHSI：通过任务标记化实现物理人-场景交互的统一合成

摘要

合成多样且物理合理的人-场景交互（HSI）对于计算机动画和具身人工智能都至关重要。尽管已取得令人鼓舞的进展，但现有方法主要集中于开发独立的控制器，每个控制器专精于特定的交互任务。这极大地限制了处理多种需要整合多项技能的复杂HSI任务的能力，例如在携带物品的同时坐下。为解决这一问题，我们提出了TokenHSI，一个基于Transformer的统一策略，能够实现多技能统一与灵活适应。其核心洞见是将人形机器人的本体感知建模为独立的共享令牌，并通过掩码机制与不同的任务令牌相结合。这种统一策略促进了技能间的有效知识共享，从而推动了多任务训练。此外，我们的策略架构支持可变长度输入，使得已学技能能够灵活适应新场景。通过训练额外的任务令牌生成器，我们不仅能调整交互目标的几何形态，还能协调多项技能以应对复杂任务。实验表明，我们的方法在多种HSI任务中显著提升了通用性、适应性和可扩展性。网站：https://liangpan99.github.io/TokenHSI/

English

Synthesizing diverse and physically plausible Human-Scene Interactions (HSI) is pivotal for both computer animation and embodied AI. Despite encouraging progress, current methods mainly focus on developing separate controllers, each specialized for a specific interaction task. This significantly hinders the ability to tackle a wide variety of challenging HSI tasks that require the integration of multiple skills, e.g., sitting down while carrying an object. To address this issue, we present TokenHSI, a single, unified transformer-based policy capable of multi-skill unification and flexible adaptation. The key insight is to model the humanoid proprioception as a separate shared token and combine it with distinct task tokens via a masking mechanism. Such a unified policy enables effective knowledge sharing across skills, thereby facilitating the multi-task training. Moreover, our policy architecture supports variable length inputs, enabling flexible adaptation of learned skills to new scenarios. By training additional task tokenizers, we can not only modify the geometries of interaction targets but also coordinate multiple skills to address complex tasks. The experiments demonstrate that our approach can significantly improve versatility, adaptability, and extensibility in various HSI tasks. Website: https://liangpan99.github.io/TokenHSI/

TokenHSI：通过任务标记化实现物理人-场景交互的统一合成

TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenization

摘要

Summary

Support

Support