RoMath：羅馬尼亞語數學推理基準

RoMath: A Mathematical Reasoning Benchmark in Romanian

September 17, 2024

作者: Adrian Cosma, Ana-Maria Bucur, Emilian Radoi

cs.AI

摘要

數學長期以來主要通過自然語言傳達，以便人類理解。隨著機械化數學和證明助手的興起，越來越需要理解非正式數學文本，然而大多數現有的基準測試僅專注於英語，忽略了其他語言。本文介紹了RoMath，這是一個羅馬尼亞數學推理基準套件，包括三個數據集：RoMath-Baccalaureate、RoMath-Competitions和RoMath-Synthetic，涵蓋了各種數學領域和難度水平，旨在改進非英語語言模型並促進多語言人工智能的發展。通過專注於羅馬尼亞語，這是一種資源稀缺且具有獨特語言特徵的語言，RoMath解決了以英語為中心的模型的限制，並強調了除了簡單的自動翻譯之外，對專用資源的需求。我們對幾個開放權重語言模型進行基準測試，突顯了為代表性不足的語言創建資源的重要性。我們提供代碼和數據集。

English

Mathematics has long been conveyed through natural language, primarily for human understanding. With the rise of mechanized mathematics and proof assistants, there is a growing need to understand informal mathematical text, yet most existing benchmarks focus solely on English, overlooking other languages. This paper introduces RoMath, a Romanian mathematical reasoning benchmark suite comprising three datasets: RoMath-Baccalaureate, RoMath-Competitions and RoMath-Synthetic, which cover a range of mathematical domains and difficulty levels, aiming to improve non-English language models and promote multilingual AI development. By focusing on Romanian, a low-resource language with unique linguistic features, RoMath addresses the limitations of Anglo-centric models and emphasizes the need for dedicated resources beyond simple automatic translation. We benchmark several open-weight language models, highlighting the importance of creating resources for underrepresented languages. We make the code and dataset available.