ChatPaper.aiChatPaper

FLAG-Trader:融合大语言模型与梯度强化学习的智能金融交易代理

FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Trading

February 17, 2025
作者: Guojun Xiong, Zhiyang Deng, Keyi Wang, Yupeng Cao, Haohang Li, Yangyang Yu, Xueqing Peng, Mingquan Lin, Kaleb E Smith, Xiao-Yang Liu, Jimin Huang, Sophia Ananiadou, Qianqian Xie
cs.AI

摘要

基于多模态金融数据微调的大型语言模型(LLMs)已在多种金融任务中展现出卓越的推理能力。然而,在交互式金融市场中,如交易这类需要复杂代理策略以优化决策的多步骤、目标导向场景中,它们往往表现欠佳。为此,我们提出了FLAG-Trader,一种统一架构,它将语言处理(通过LLMs)与梯度驱动的强化学习(RL)策略优化相结合。在此架构中,部分微调的LLM充当策略网络,既利用预训练知识,又通过参数高效微调适应金融领域。通过交易奖励驱动的策略梯度优化,我们的框架不仅提升了LLM在交易中的表现,还改善了其在其他金融领域任务上的成果。我们提供了详尽的实证证据来验证这些改进。
English
Large language models (LLMs) fine-tuned on multimodal financial data have demonstrated impressive reasoning capabilities in various financial tasks. However, they often struggle with multi-step, goal-oriented scenarios in interactive financial markets, such as trading, where complex agentic approaches are required to improve decision-making. To address this, we propose FLAG-Trader, a unified architecture integrating linguistic processing (via LLMs) with gradient-driven reinforcement learning (RL) policy optimization, in which a partially fine-tuned LLM acts as the policy network, leveraging pre-trained knowledge while adapting to the financial domain through parameter-efficient fine-tuning. Through policy gradient optimization driven by trading rewards, our framework not only enhances LLM performance in trading but also improves results on other financial-domain tasks. We present extensive empirical evidence to validate these enhancements.

Summary

AI-Generated Summary

PDF312February 19, 2025