Fietje：一個開放、高效的荷蘭語LLM

Fietje: An open, efficient LLM for Dutch

December 19, 2024

作者: Bram Vanroy

cs.AI

摘要

本文介紹了Fietje，這是一個專為荷蘭語設計的小型語言模型（SLMs）系列。該模型基於Phi 2，一個擁有27億參數的以英語為中心的模型。Fietje在推出時展示了與更大語言模型競爭力的結果。本文的核心重點在於透明度和可重現性：Fietje是完全開源的，模型權重、數據集、訓練和評估代碼都是公開可訪問的。 本文討論了Fietje和許多其他模型在推理、情感分析、世界知識、語言可接受性和詞義消歧等廣泛評估基準上的性能。評估結果展示了在LLM領域中的快速進展，最近的小型模型優於為荷蘭語微調的舊的更大模型。這一趨勢預示著荷蘭語處理領域的美好未來，表明即使是緊湊的LLMs也變得越來越強大。 此外，將LLMs調整為荷蘭語的持續和未來努力將進一步增強這些模型，擴大其應用範圍和可訪問性。Fietje只是改善荷蘭語言技術對用戶的可訪問性的中間步驟。

English

This paper introduces Fietje, a family of small language models (SLMs) specifically designed for the Dutch language. The model is based on Phi 2, an English-centric model of 2.7 billion parameters. Fietje demonstrated competitive results with larger language models upon its release. A core emphasis of this work is transparency and reproducibility: Fietje is fully open-source, with model weights, datasets, training, and evaluation code all publicly accessible. The paper discusses the performance of Fietje and many other models on an extensive evaluation suite of benchmarks on reasoning, sentiment analysis, world knowledge, linguistic acceptability and word sense disambiguation. Evaluation results illustrate the rapid progress in the field of LLMs, where recent small models outperform older, larger models that were fine-tuned for Dutch. This trend signals an exciting future for Dutch language processing, suggesting that even compact LLMs are becoming increasingly capable. Furthermore, ongoing and future efforts to adapt LLMs to Dutch are poised to enhance these models even further, broadening their applicability and accessibility. Fietje is only an intermediate step in improving accessibility to language technology for users of the Dutch language.