交互指导优化:基于大语言模型的并行执行者-推理者框架,用于提升自动驾驶车辆交互能力
Interact, Instruct to Improve: A LLM-Driven Parallel Actor-Reasoner Framework for Enhancing Autonomous Vehicle Interactions
March 1, 2025
作者: Shiyu Fang, Jiaqi Liu, Chengkai Xu, Chen Lv, Peng Hang, Jian Sun
cs.AI
摘要
自动驾驶车辆(AVs)已步入商业化阶段,但其在交互与意图表达方面的能力局限,仍使其在与人类驾驶车辆(HVs)的互动中面临挑战。近期,大型语言模型(LLMs)的进展实现了双向人机沟通,然而推理速度缓慢与实时决策需求之间的矛盾,对实际部署构成了难题。针对这些问题,本文提出了一种并行执行者-推理者框架,旨在跨多种场景实现明确的AV-HV双向交互。首先,通过在训练过程中促进LLM驱动的推理者与异构模拟HVs之间的交互,建立了一个交互记忆数据库,即执行者。随后,通过引入记忆分区模块和双层记忆检索模块,显著增强了执行者处理异构HVs的能力。消融研究及与其他决策方法的对比表明,所提出的执行者-推理者框架在安全性和效率上均有显著提升。最后,结合从推理者推理得出的外部人机界面(eHMI)信息与从执行者检索到的可行行动方案,在多场景实地交互中验证了该执行者-推理者框架的有效性。我们的代码已发布于https://github.com/FanGShiYuu/Actor-Reasoner。
English
Autonomous Vehicles (AVs) have entered the commercialization stage, but their
limited ability to interact and express intentions still poses challenges in
interactions with Human-driven Vehicles (HVs). Recent advances in large
language models (LLMs) enable bidirectional human-machine communication, but
the conflict between slow inference speed and the need for real-time
decision-making challenges practical deployment. To address these issues, this
paper introduces a parallel Actor-Reasoner framework designed to enable
explicit bidirectional AV-HV interactions across multiple scenarios. First, by
facilitating interactions between the LLM-driven Reasoner and heterogeneous
simulated HVs during training, an interaction memory database, referred to as
the Actor, is established. Then, by introducing the memory partition module and
the two-layer memory retrieval module, the Actor's ability to handle
heterogeneous HVs is significantly enhanced. Ablation studies and comparisons
with other decision-making methods demonstrate that the proposed Actor-Reasoner
framework significantly improves safety and efficiency. Finally, with the
combination of the external Human-Machine Interface (eHMI) information derived
from Reasoner's reasoning and the feasible action solutions retrieved from the
Actor, the effectiveness of the proposed Actor-Reasoner is confirmed in
multi-scenario field interactions. Our code is available at
https://github.com/FanGShiYuu/Actor-Reasoner.Summary
AI-Generated Summary