交互指导优化：基于大语言模型的并行执行者-推理者框架，用于提升自动驾驶车辆交互能力

Interact, Instruct to Improve: A LLM-Driven Parallel Actor-Reasoner Framework for Enhancing Autonomous Vehicle Interactions

March 1, 2025

作者: Shiyu Fang, Jiaqi Liu, Chengkai Xu, Chen Lv, Peng Hang, Jian Sun

cs.AI

摘要

自动驾驶车辆（AVs）已步入商业化阶段，但其在交互与意图表达方面的能力局限，仍使其在与人类驾驶车辆（HVs）的互动中面临挑战。近期，大型语言模型（LLMs）的进展实现了双向人机沟通，然而推理速度缓慢与实时决策需求之间的矛盾，对实际部署构成了难题。针对这些问题，本文提出了一种并行执行者-推理者框架，旨在跨多种场景实现明确的AV-HV双向交互。首先，通过在训练过程中促进LLM驱动的推理者与异构模拟HVs之间的交互，建立了一个交互记忆数据库，即执行者。随后，通过引入记忆分区模块和双层记忆检索模块，显著增强了执行者处理异构HVs的能力。消融研究及与其他决策方法的对比表明，所提出的执行者-推理者框架在安全性和效率上均有显著提升。最后，结合从推理者推理得出的外部人机界面（eHMI）信息与从执行者检索到的可行行动方案，在多场景实地交互中验证了该执行者-推理者框架的有效性。我们的代码已发布于https://github.com/FanGShiYuu/Actor-Reasoner。

English

Autonomous Vehicles (AVs) have entered the commercialization stage, but their limited ability to interact and express intentions still poses challenges in interactions with Human-driven Vehicles (HVs). Recent advances in large language models (LLMs) enable bidirectional human-machine communication, but the conflict between slow inference speed and the need for real-time decision-making challenges practical deployment. To address these issues, this paper introduces a parallel Actor-Reasoner framework designed to enable explicit bidirectional AV-HV interactions across multiple scenarios. First, by facilitating interactions between the LLM-driven Reasoner and heterogeneous simulated HVs during training, an interaction memory database, referred to as the Actor, is established. Then, by introducing the memory partition module and the two-layer memory retrieval module, the Actor's ability to handle heterogeneous HVs is significantly enhanced. Ablation studies and comparisons with other decision-making methods demonstrate that the proposed Actor-Reasoner framework significantly improves safety and efficiency. Finally, with the combination of the external Human-Machine Interface (eHMI) information derived from Reasoner's reasoning and the feasible action solutions retrieved from the Actor, the effectiveness of the proposed Actor-Reasoner is confirmed in multi-scenario field interactions. Our code is available at https://github.com/FanGShiYuu/Actor-Reasoner.