PAFT：与提示无关的微调

PAFT: Prompt-Agnostic Fine-Tuning

February 18, 2025

作者: Chenxing Wei, Yao Shu, Mingwen Ou, Ying Tiffany He, Fei Richard Yu

cs.AI

摘要

尽管大型语言模型（LLMs）在微调后能很好地适应下游任务，但这种适应性往往以牺牲提示的鲁棒性为代价，即便是微小的提示变化也可能显著降低模型性能。为解决这一问题，我们提出了提示无关微调（Prompt-Agnostic Fine-Tuning, PAFT），这是一种简单而有效的方法，在微调过程中动态调整提示。该方法促使模型学习任务的基本原理，而非过度拟合特定的提示表述。PAFT分两个阶段进行：首先，构建一组多样且有意义的人工合成候选提示；其次，在微调过程中，从该集合中随机抽取提示以生成动态的训练输入。跨多种数据集和LLMs的广泛实验表明，采用PAFT训练的模型在包括未见过的提示在内的广泛提示范围内展现出强大的鲁棒性和泛化能力。这种增强的鲁棒性不仅提升了模型性能，还加快了推理速度，同时保持了训练效率。消融研究进一步验证了PAFT的有效性。

English

While Large Language Models (LLMs) adapt well to downstream tasks after fine-tuning, this adaptability often compromises prompt robustness, as even minor prompt variations can significantly degrade performance. To address this, we propose Prompt-Agnostic Fine-Tuning(PAFT), a simple yet effective approach that dynamically adjusts prompts during fine-tuning. This encourages the model to learn underlying task principles rather than overfitting to specific prompt formulations. PAFT operates in two stages: First, a diverse set of meaningful, synthetic candidate prompts is constructed. Second, during fine-tuning, prompts are randomly sampled from this set to create dynamic training inputs. Extensive experiments across diverse datasets and LLMs demonstrate that models trained with PAFT exhibit strong robustness and generalization across a wide range of prompts, including unseen ones. This enhanced robustness improves both model performance and inference speed while maintaining training efficiency. Ablation studies further confirm the effectiveness of PAFT.