教授具有体现式强化学习代理的能力：语言使用的信息量和多样性

Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use

October 31, 2024

作者: Jiajun Xi, Yinong He, Jianing Yang, Yinpei Dai, Joyce Chai

cs.AI

摘要

在现实场景中，对于具有实体的代理来说，利用人类语言获取明确或隐含知识以进行学习任务是可取的。尽管最近取得了一些进展，但大多数先前的方法采用简单的低级指令作为语言输入，这可能无法反映自然的人类交流。如何整合丰富的语言使用以促进任务学习尚不清楚。为了解决这个问题，本文研究了不同类型的语言输入在促进强化学习（RL）实体代理中的作用。更具体地，我们考察了语言信息量的不同级别（即，对过去行为的反馈和对未来指导）以及多样性（即，语言表达的变化）如何影响代理学习和推理。基于四个RL基准的实证结果表明，接受多样化和信息丰富的语言反馈训练的代理能够实现增强的泛化能力，并快速适应新任务。这些发现突显了语言在教导具有实体的代理在开放世界中学习新任务中的关键作用。项目网站：https://github.com/sled-group/Teachable_RL

English

In real-world scenarios, it is desirable for embodied agents to have the ability to leverage human language to gain explicit or implicit knowledge for learning tasks. Despite recent progress, most previous approaches adopt simple low-level instructions as language inputs, which may not reflect natural human communication. It's not clear how to incorporate rich language use to facilitate task learning. To address this question, this paper studies different types of language inputs in facilitating reinforcement learning (RL) embodied agents. More specifically, we examine how different levels of language informativeness (i.e., feedback on past behaviors and future guidance) and diversity (i.e., variation of language expressions) impact agent learning and inference. Our empirical results based on four RL benchmarks demonstrate that agents trained with diverse and informative language feedback can achieve enhanced generalization and fast adaptation to new tasks. These findings highlight the pivotal role of language use in teaching embodied agents new tasks in an open world. Project website: https://github.com/sled-group/Teachable_RL