mAceReason-Math: A Dataset of High-Quality Multilingual Math Problems Ready For RLVR

Konstantin Dobler; Simon Lehnerer; Federico Scozzafava; Jonathan Janke; Mohamed Ali

cs.CLMar 11, 2026

mAceReason-Math: A Dataset of High-Quality Multilingual Math Problems Ready For RLVR

Authors: Konstantin Dobler, Simon Lehnerer, Federico Scozzafava, Jonathan Janke, Mohamed Ali

Organizations: Apple · Hasso Plattner Institute & ELLIS Unit Potsdam

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) has been successfully applied to significantly boost the capabilities of pretrained large language models, especially in the math and logic problem domains. However, current research and available training datasets remain English-centric. While multilingual training data and benchmarks have been created in the past, they were not created with RLVR and current model capability in mind, and their level of difficulty is often too low to provide appropriate training signals for current models. To address this gap, we provide mAceReason-Math, a dataset of high-quality translations of challenging math problems sourced from a corpus specifically curated for RLVR (AceReason-Math). We further take specific care to clean and improve our translations, resulting in a coverage of 14 languages with more than 10,000 samples per language. We release the dataset to facilitate multilingual RLVR research and benchmarking in the research community.

Figures & tables

English (Original)	Translated (Language)
How many integers are there between $(11.2)^{3}$ and $(11.3)^{3}$ ?	German: Wie viele ganze Zahlen gibt es zwischen $(11{,}2)^{3}$ und $(11{,}3)^{3}$ ?
Given the function $f(x)=x^{2-m}$ is defined on the interval $[-3-m,m^{2}-m]$ and is an odd function, then $f(m)=$ ?	Italian: Data la funzione $f(x)=x^{2-m}$ definita nell’intervallo $[-3-m,m^{2}-m]$ che è una funzione dispari, allora $f(m)=$ ?
A circle with radius $R$ is inscribed in an isosceles trapezoid. The upper base of the trapezoid is half of its height. Find the area of the trapezoid.	Japanese: 半径 $R$ の円が二等辺台形に内接している。この台形の上底は高さの半分である。台形の面積を求めよ。
Given that an office at a school needs to arrange a duty roster from the 1st to the 6th day with six designated individuals participating, find the total number of different arrangements possible, given that person A and person B cannot be adjacent, and person C and person D also cannot be adjacent.	Chinese: 学校某办公室需要安排6个指定人员在第1天到第6天进行值班，已知甲和乙不能相邻，丙和丁也不能相邻，求共有多少种不同的安排方案。
Given a function $f(x)=\log_{a}\left(\sqrt{x^{2}+1}+x\right)+\frac{1}{a^{x}-1}+\frac{3}{2}$ , where $a>0$ and $a\neq 1$ . If $f\left(\log_{3}b\right)=5$ for $b>0$ and $b\neq 1$ , find the value of $f\left(\log_{\frac{1}{3}}b\right)$ .	Russian: Дана функция $f(x)=\log_{a}\left(\sqrt{x^{2}+1}+x\right)+\frac{1}{a^{x}-1}+\frac{3}{2}$ , где $a>0$ и $a\neq 1$ . Если $f\left(\log_{3}b\right)=5$ для $b>0$ и $b\neq 1$ , найди значение $f\left(\log_{\frac{1}{3}}b\right)$ .
If 105 customers purchased a total of 300 cans of soda, with every customer purchasing at least one can of soda, determine the maximum possible median number of cans of soda bought per customer that day.	Brazilian Portuguese: Se 105 clientes compraram um total de 300 latas de refrigerante, sendo que cada cliente comprou pelo menos uma lata de refrigerante, determine a mediana máxima possível do número de latas de refrigerante compradas por cliente naquele dia.

\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishTable 1: Examples of original samples and their translations to selected target languages from our final dataset.

Model	en	zh	de	es	fr	it	pt	ru	ja	ko	th	bn	te	sw	Average
Open-weight models (avg@8 evaluation)
\fontspec_if_language:nTF ENG \addfontfeature Language=Englishgpt-oss-20b	79.2	78.6	77.5	75.0	78.3	75.8	76.6	79.1	78.2	75.5	78.2	73.8	77.8	64.8	76.3 $\pm$ 3.4
\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishSmolLM3-3B	77.9	53.6	69.5	72.8	72.0	70.0	71.8	69.2	58.8	55.3	59.2	28.0	16.8	20.9	56.8 $\pm$ 18.9
\fontspec_if_language:nTF ENG \addfontfeature Language=Englishgemma-3-12b-it	58.3	51.0	48.3	53.9	54.1	53.9	52.8	52.4	48.0	48.2	52.4	42.5	42.8	42.3	50.1 $\pm$ 4.6
\fontspec_if_language:nTF ENG \addfontfeature Language=Englishgemma-3-4b-it	29.5	35.3	31.0	45.6	42.6	43.1	41.8	39.9	24.1	32.8	35.1	24.6	25.2	23.8	33.9 $\pm$ 7.2
\fontspec_if_language:nTF ENG \addfontfeature Language=Englishgemma-3-1b-it	19.4	9.1	9.3	11.4	10.3	11.1	10.9	10.0	7.8	6.0	6.6	7.0	4.5	3.3	9.1 $\pm$ 3.6

\fontspec_if_language:nTF ENG \addfontfeature Language=English Table 2 : Accuracy (%) of different models across languages on our dataset. We report pass@1 with a single sampled response per prompt for closed models (classic accuracy) and the average accuracy over eight sampled responses per prompt (avg@8) for open-weight models. The best result for each language within the open- and closed-model categories is marked in bold . The Average column reports mean

\pm

std across languages.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

\fontspec_if_language:nTF ENG \addfontfeature Language=English Figure 1 : Number of accepted samples per improvement stage in the translation pipeline. Most samples are accepted after the initial translation or one additional round of LLM-based refinement.

Language	Train (Parallel)	Test (Parallel)	Train (All)
English	7,620	190	12,245
German	7,620	190	11,151
French	7,620	190	11,007
Spanish	7,620	190	11,346
Chinese	7,620	190	10,470
Russian	7,620	190	11,237

Appendix

\fontspec_if_language:nTF ENG \addfontfeature Language=English Table 3 : Number of samples in mAceReason-Math per split and language.

Primary Category	Count	Subcategory	Illustrative Examples
Missing Context	~2%	Missing Visual	“The diagram shows three rectangles and three straight lines. What is the value of p + q + r […]” (No diagram provided)
	~2%	Missing Definitions	“Define a regular n-pointed star as described in the original problem […]” (Critical definitions not provided)
Content and Extraction Issues	~1.5%	Answer Leakage	“[…] Answer: decreased by 6%” (Solution provided in problem)
		Processing Artifacts	“[…] text above has been translated as requested […]” (Leftover data-processing artifacts)
		Truncated Content	“[…] We know that for any pair of distinct numbers from this set there is” with answer: “50” (Cuts off before finishing question, answer is just “50”)
Ill-posed Question	~0.5%	Free-text answer	“ How can you cut a 5×5 square […] into 50 equal squares?” (No methodology provided, answer is just a single integer)

Appendix

\fontspec_if_language:nTF ENG \addfontfeature Language=English Table 4 : Overview of “critical issues” identified in the original data. Examples abbreviated with […] for space constraints. Bold text highlights key corruption indicators within the quoted examples.

Model	en	zh	de	es	fr	it	pt	ru	ja	ko	th	bn	te	sw	Average
Open-weight models (avg@8 evaluation)
\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishQwen3-14B-Base	31.4	29.5	24.9	27.3	25.2	21.7	25.9	28.2	21.2	20.2	18.8	13.2	12.1	5.7	21.8 $\pm$ 6.8
\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishQwen3-8B-Base	25.8	23.2	17.8	21.8	20.7	18.6	22.9	23.9	15.3	14.4	12.5	9.0	12.6	2.8	17.2 $\pm$ 6.1
\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishQwen3-4B-Base	15.5	15.3	12.1	13.6	13.6	13.6	14.5	13.3	10.3	8.3	7.9	8.1	8.4	3.0	11.2 $\pm$ 3.4
\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishQwen3-1.7B-Base	8.0	5.0	3.0	3.8	4.0	3.2	4.2	4.1	2.3	2.6	2.4	0.9	0.9	0.5	3.2 $\pm$ 1.8
\fontspec_if_language:nTF ENG \addfontfeature Language=EnglishQwen3-14B	88.8	89.9	91.6	91.3	90.9	92.0	91.1	87.5	89.3	89.2	90.5	90.6	87.6	69.7	88.6 $\pm$ 5.2

Appendix

\fontspec_if_language:nTF ENG \addfontfeature Language=English Table 5 : Accuracy (%) of different models across languages on our dataset. We report pass@1 with a single sampled response per prompt for closed models (classic accuracy) and the average accuracy over eight sampled responses per prompt (avg@8) for open-weight models. Best result per language is marked in bold . The Average column reports mean

\pm

std across languages.

LLM Mathematical Reasoning

Large language model mathematical reasoning studies how AI systems solve, explain, verify, and generalize across mathematical problems and proofs.

See all 269 papers

PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
Jul 7, 2026Daryna Dementieva, Nikolay Babakov, Kathy Hämmerl +14Mathematical Reasoning Benchmarks Multilingual Language Model Evaluation
Technical University of Munich (TUM) · Munich Center for Machine Learning (MCML) · Independent Researcher +7
MATH-PT: A Math Reasoning Benchmark for European and Brazilian Portuguese
Apr 1, 2026Tiago Teixeira, Ana Carolina Erthal, Juan Belieni +5Mathematical Reasoning Benchmarks Multilingual Language Model Evaluation
Instituto Superior Técnico, Universidade de Lisboa · Fundação Getulio Vargas · Instituto de Telecomunicações +2
ResearchMath-14K: Scaling Research-Level Mathematics via Agents
May 27, 2026Guijin Son, Seungyeop Yi, Minju Gwak +3Supervised Fine-Tuning LLM Mathematical Reasoning
Seoul National University · OneLineAI · Yonsei University

Jul 7, 2026J/K move · Enter open · S save

PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages

Daryna Dementieva, Nikolay Babakov, Kathy Hämmerl, Ilseyar Alimova+13

Technical University of Munich (TUM) · Munich Center for Machine Learning (MCML) · Independent Researcher+7

Mathematical Reasoning Benchmarks Multilingual Language Model Evaluation Mathematical Reasoning Low-Resource Language Processing LLM Mathematical Reasoning

Open paper AskPDF arXiv

Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al., 2025) dataset represents a significant step forward, yet its coverage is still limited to 18 only high-resource languages. To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional {underrepresented languages spanning 6 language families -- ranging from mid-resource to extreme low-resource settings. We constructed the dataset through a human-curated pipeline, where native speakers thoroughly validated pre-computed translations. Using PluraMath, we then benchmark 27 reasoning LLMs across four model scales -- small, mid-size, large, and closed-source ensembles -- probing the multilingual mathematical reasoning capabilities of state-of-the-art models under diverse linguistic conditions. Our fine-grained analysis confirms a persistent gap in mathematical reasoning performance between high-resource and underrepresented languages, with stronger results largely associated with better instruction-following ability. We fully open-source our dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to multilingual benchmark development for underrepresented communities.

Abstract

LLM Mathematical Reasoning

Explore similar work

PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages

MATH-PT: A Math Reasoning Benchmark for European and Brazilian Portuguese

ResearchMath-14K: Scaling Research-Level Mathematics via Agents