語言模型可以自我延長以生成長文。

摘要

最近大型語言模型（LLMs）的進步顯著增強了其處理長文本的能力，但在生成長且對齊的輸出方面仍存在明顯差距。這一限制源於訓練中的差距，即預訓練缺乏對於生成長文本的有效指導，而後訓練數據主要包含短的查詢-回應對。目前的方法，如指導反向翻譯和行為模仿，面臨著數據質量、版權問題以及專有模型使用限制等挑戰。本文介紹了一種創新的迭代訓練框架，稱為Self-Lengthen，利用LLMs的內在知識和技能，無需輔助數據或專有模型。該框架由兩個角色組成：生成器和擴展器。生成器生成初始回應，然後由擴展器進行分割和擴展。這個過程產生了一個新的、更長的回應，用於迭代地訓練生成器和擴展器。通過這個過程，模型逐步訓練以處理越來越長的回應。在基準測試和人類評估上的實驗表明，Self-Lengthen在應用於Qwen2和LLaMA3等頂級開源LLMs時，在長文本生成方面優於現有方法。我們的代碼公開在https://github.com/QwenLM/Self-Lengthen。

English

Recent advancements in Large Language Models (LLMs) have significantly enhanced their ability to process long contexts, yet a notable gap remains in generating long, aligned outputs. This limitation stems from a training gap where pre-training lacks effective instructions for long-text generation, and post-training data primarily consists of short query-response pairs. Current approaches, such as instruction backtranslation and behavior imitation, face challenges including data quality, copyright issues, and constraints on proprietary model usage. In this paper, we introduce an innovative iterative training framework called Self-Lengthen that leverages only the intrinsic knowledge and skills of LLMs without the need for auxiliary data or proprietary models. The framework consists of two roles: the Generator and the Extender. The Generator produces the initial response, which is then split and expanded by the Extender. This process results in a new, longer response, which is used to train both the Generator and the Extender iteratively. Through this process, the models are progressively trained to handle increasingly longer responses. Experiments on benchmarks and human evaluations show that Self-Lengthen outperforms existing methods in long-text generation, when applied to top open-source LLMs such as Qwen2 and LLaMA3. Our code is publicly available at https://github.com/QwenLM/Self-Lengthen.

語言模型可以自我延長以生成長文。

Language Models can Self-Lengthen to Generate Long Texts

摘要

Summary

Support

Support