Xmodel-1.5：一個十億規模的多語言LLM

Xmodel-1.5: An 1B-scale Multilingual LLM

November 15, 2024

作者: Wang Qun, Liu Yang, Lin Qingquan, Jiang Ling

cs.AI

摘要

我們介紹了 Xmodel-1.5，一個新穎的 10 億參數多語言大型模型，預先訓練了約 2 兆個標記。該模型在多種語言中展現出強大的性能，特別是在泰語、阿拉伯語和法語方面表現突出，同時在中文和英文中也表現出有效性。此外，我們通過釋出一個泰語評估數據集，該數據集包含由占佔良大學綜合創新學院學生註釋的數百個問題，為研究社區做出了貢獻。儘管結果令人鼓舞，我們承認仍有改進的空間。我們希望這項工作推動多語言人工智慧研究的持續努力，促進各種自然語言處理任務中更好的跨語言理解。我們的模型和代碼在 GitHub 上公開可用，網址為 https://github.com/XiaoduoAILab/XmodelLM。

English

We introduce Xmodel-1.5, a novel 1-billion-parameter multilingual large model pretrained on approximately 2 trillion tokens. The model demonstrates strong performance across several languages, with particularly notable results in Thai, Arabic, and French, alongside its effectiveness in Chinese and English. In addition, we contribute to the research community by releasing a Thai evaluation dataset, which includes hundreds of questions annotated by students from Chulalongkorn University's School of Integrated Innovation. While the results are promising, we acknowledge that there is still room for improvement. We hope this work advances ongoing efforts in multilingual AI research and promotes better cross-linguistic understanding in various natural language processing tasks. Our models and code are publicly available on GitHub at https://github.com/XiaoduoAILab/XmodelLM.