利用开源模型为用户偏好生成系统消息

System Message Generation for User Preferences using Open-Source Models

February 17, 2025

作者: Minbyul Jeong, Jungho Cho, Minsoo Khang, Dawoon Jung, Teakgyu Hong

cs.AI

摘要

系统消息在与大型语言模型（LLMs）的交互中发挥着至关重要的作用，通常用作启动对话的提示。通过系统消息，用户可以指定特定角色，执行预期任务，整合背景信息，指定各种输出格式和沟通风格。尽管具有如此多样性，公开可用的数据往往缺乏系统消息，并受到行业领域严格的许可限制。使用符合用户指令的系统消息手动标记公开可用数据需要大量资源。鉴于这些挑战，我们的工作引入了SysGen，这是一个从带有系统消息的监督微调数据集中生成系统消息的管道，以获得更好与助手响应对齐的结果。在SysGen数据上进行训练已经显示出模型响应与系统消息和用户指令对齐方面的显著改进，这在Multifacet基准测试中得到了证明，同时对其他未见基准测试（如Open LLM Leaderboard 2）的影响最小。我们的定性分析突显了多样化系统消息的重要性，以确保在不同情境下更好地适应。

English

System messages play a crucial role in interactions with large language models (LLMs), often serving as prompts to initiate conversations. Through system messages, users can assign specific roles, perform intended tasks, incorporate background information, specify various output formats and communication styles. Despite such versatility, publicly available data are often lack system messages and subject to strict license constraints in the industry field. Manual labeling of publicly available data with system messages that align with user instructions demands significant resources. In view of such challenges, our work introduces SysGen, a pipeline for generating system messages with better aligned assistant responses from the supervised fine-tuning dataset without system messages. Training on SysGen data has demonstrated substantial improvements in the alignment of model responses with system messages and user instructions, as demonstrated across various open-source models on the Multifacet benchmark, while maintaining minimal impact on other unseen benchmarks such as Open LLM Leaderboard 2. Our qualitative analysis highlights the importance of diverse system messages to ensure better adaptability across different contexts.