教導具體化強化學習智能體：語言使用的資訊性和多樣性

Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use

October 31, 2024

作者: Jiajun Xi, Yinong He, Jianing Yang, Yinpei Dai, Joyce Chai

cs.AI

摘要

在現實世界的情境中，期望具體化的代理能夠利用人類語言以獲取明確或隱含的知識，用於學習任務。儘管近年取得了進展，大多數先前的方法採用簡單的低層指令作為語言輸入，這可能無法反映自然的人類溝通。如何融入豐富的語言使用以促進任務學習並不明確。為了解答這個問題，本文研究了不同類型的語言輸入如何促進強化學習（RL）具體化代理。更具體地，我們檢視了不同程度的語言信息量（即過去行為的反饋和未來指導）以及多樣性（即語言表達的變化）如何影響代理學習和推理。我們基於四個RL基準的實證結果顯示，接受多樣和信息豐富的語言反饋訓練的代理能夠實現增強的泛化能力，並快速適應新任務。這些發現凸顯了語言在教導具體化代理在開放世界中學習新任務中的關鍵作用。項目網站：https://github.com/sled-group/Teachable_RL

English

In real-world scenarios, it is desirable for embodied agents to have the ability to leverage human language to gain explicit or implicit knowledge for learning tasks. Despite recent progress, most previous approaches adopt simple low-level instructions as language inputs, which may not reflect natural human communication. It's not clear how to incorporate rich language use to facilitate task learning. To address this question, this paper studies different types of language inputs in facilitating reinforcement learning (RL) embodied agents. More specifically, we examine how different levels of language informativeness (i.e., feedback on past behaviors and future guidance) and diversity (i.e., variation of language expressions) impact agent learning and inference. Our empirical results based on four RL benchmarks demonstrate that agents trained with diverse and informative language feedback can achieve enhanced generalization and fast adaptation to new tasks. These findings highlight the pivotal role of language use in teaching embodied agents new tasks in an open world. Project website: https://github.com/sled-group/Teachable_RL