Fietje：一种面向荷兰语的开放高效LLM

Fietje: An open, efficient LLM for Dutch

December 19, 2024

作者: Bram Vanroy

cs.AI

摘要

本文介绍了 Fietje，一系列专为荷兰语设计的小语言模型（SLMs）。该模型基于 Phi 2，一个参数为 27 亿的以英语为中心的模型。Fietje 在发布时展示了与更大语言模型竞争力的结果。本文的核心重点是透明度和可复现性：Fietje 是完全开源的，模型权重、数据集、训练和评估代码都可以公开获取。 本文讨论了 Fietje 和许多其他模型在推理、情感分析、世界知识、语言可接受性和词义消歧等广泛评估基准上的表现。评估结果展示了在LLMs领域的快速进展，最近的小模型胜过了为荷兰语进行微调的旧的更大模型。这一趋势预示着荷兰语处理领域的光明未来，表明即使是紧凑的LLMs也变得越来越有能力。 此外，将LLMs调整为荷兰语的持续和未来努力将进一步增强这些模型，拓宽它们的适用性和可访问性。Fietje 只是改进荷兰语言技术对用户可访问性的中间步骤。

English

This paper introduces Fietje, a family of small language models (SLMs) specifically designed for the Dutch language. The model is based on Phi 2, an English-centric model of 2.7 billion parameters. Fietje demonstrated competitive results with larger language models upon its release. A core emphasis of this work is transparency and reproducibility: Fietje is fully open-source, with model weights, datasets, training, and evaluation code all publicly accessible. The paper discusses the performance of Fietje and many other models on an extensive evaluation suite of benchmarks on reasoning, sentiment analysis, world knowledge, linguistic acceptability and word sense disambiguation. Evaluation results illustrate the rapid progress in the field of LLMs, where recent small models outperform older, larger models that were fine-tuned for Dutch. This trend signals an exciting future for Dutch language processing, suggesting that even compact LLMs are becoming increasingly capable. Furthermore, ongoing and future efforts to adapt LLMs to Dutch are poised to enhance these models even further, broadening their applicability and accessibility. Fietje is only an intermediate step in improving accessibility to language technology for users of the Dutch language.