cs.CYMar 28, 2026

RoMathExam: A Longitudinal Dataset of Romanian Math Exams (1895-2025) with a Seven-Decade Core (1957-2025)

Authors: Luca-Ncolae Cuclea, Sabin-Codrut Badea, Adrian-Marius Dumitran

Organizations: University of Bucharest, Faculty of Mathematics and Computer Science, Bucharest, Romania · Cu Drag si Sport SRL, Bucharest, Romania · Softbinator Technologies, Bucharest, Romania

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

AI in Education research increasingly relies on authentic, curriculum-grounded assessment data, yet large, well-structured exam corpora remain scarce for many languages and educational systems. We introduce RoMathExam, a longitudinal dataset of Romanian high-school mathematics exams spanning 1895-2025, with a robust standardized core for 1957-2025. The dataset contains 10,592 mathematics problems organized into 600+ complete exam sets across multiple tracks (M1-M4), covering both official national examination sessions and ministry-published training variants. Beyond high-fidelity digitization and a unified JSON schema with traceable provenance, RoMathExam is enriched with curriculum-aligned topic tags and dense text embeddings, enabling variant detection, deduplication, and similarity-based retrieval. To overcome the lack of historical psychometric data, we propose and validate a solution complexity metric as a scalable intrinsic proxy for difficulty. Our evaluation across three frontier reasoning models (GPT-5-mini, DeepSeek-R1, and Qwen3-235B-Thinking) reveals high cross-model synchronization (r > 0.72), confirming the metric's ability to isolate intrinsic mathematical depth from stochastic generation noise. We demonstrate the dataset's utility through a longitudinal analysis that quantifies a "regime shift" from volatile historical formats to a standardized, algebra-dominant modern curriculum. RoMathExam provides a foundation for reproducible research in difficulty modeling, curriculum analytics, and LLM evaluation in low-resource linguistic contexts.

Figures & tables

MetricValue
Jaccard Similarity0.8847
Precision0.8811
Recall0.9662
F1 Score0.9217
Cohen’s Kappa0.9477
Table 1: Similarity metrics between Expert and LLM annotations (N=100)
Figure 1: Temporal Similarity Matrices comparing year-to-year consistency.
Figure 2: Relative Share of Major Mathematical Categories (1957–2025). The plot highlights the gradual dominance of Algebra in the modern curriculum.
Figure 3: Cross-model complexity trends (2010–2025). High synchronization ( r>0.72r>0.72 ) confirms intrinsic difficulty spikes between the models.
Figure 4: The Stabilization of Difficulty Metrics. Historical exams (pre-2000) display significant stochastic variance (a), often driven by isolated, high-complexity outliers. The modern standardized era (2010–2025) demonstrates a convergence toward stability: while individual item complexity has decreased (b), the aggregate cognitive load is maintained through a higher volume of items, resulting in a consistent total output.
Query Problem (Student Input)Retrieved Isomorphic Variant
2025, Special Session, M2: Rezolva t , i în mul t , imea numerelor reale ecua t , ia 2−x=x2+x−1\sqrt{2-x}=\sqrt{x^{2}+x-1} .2011, Model, M2: Rezolva t , i în mul t , imea numerelor reale ecua t , ia 2−2−x=x2-\sqrt{2-x}=x .
2023, Special Session, M2: În reperul cartezian xOyxOy se consideră punctele A(1,−5)A(1,-5) s , i B(5,5)B(5,5) . Determina t , i distan t , a de la punctul OO la mijlocul segmentului ABAB .2010, Model, M2: În reperul cartezian xOyxOy se consideră punctele A(−1,−2), B(1,2)A(-1,-2),\ B(1,2) şi C(2,−1)C(2,-1) . Calcula t , i distan t , a de la punctul CC la mijlocul segmentului ABAB .
Table 2: Examples of Isomorphic Retrieval. The system identifies historical variants that share the same underlying logic as the query problem, enabling targeted student practice.
More in this topic

Item Difficulty Estimation

See all 17 papers

Explore similar work

CardsList
Apr 1, 2026

MATH-PT: A Math Reasoning Benchmark for European and Brazilian Portuguese

The use of large language models (LLMs) for complex mathematical reasoning is an emergent area of research, with fast progress in methods, models, and benchmark datasets. However, most mathematical reasoning evaluations exhibit a significant linguistic bias, with the vast majority of benchmark datasets being exclusively in English or (at best) translated from English. We address this limitation by introducing {\sc Math-PT}, a novel dataset comprising 1,729 mathematical problems written in European and Brazilian Portuguese. {\sc Math-PT} is curated from a variety of high-quality native sources, including mathematical Olympiads, competitions, and exams from Portugal and Brazil. We present a comprehensive benchmark of current state-of-the-art LLMs on {\sc Math-PT}, revealing that frontier reasoning models achieve strong performance in multiple choice questions compared to open weight models, but that their performance decreases for questions with figures or open-ended questions. To facilitate future research, we release the benchmark dataset and model outputs.

Tiago Teixeira, Ana Carolina Erthal, Juan Belieni +5

Instituto Superior Técnico, Universidade de Lisboa · Fundação Getulio Vargas · Instituto de Telecomunicações +2

Mathematical Reasoning BenchmarksMultilingual Language Model EvaluationCross-Lingual Reasoning in Language Models
AskPDF
May 9, 2026

Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs

Following the recent achievement of gold-medal performance on the IMO by frontier LLMs, the community is searching for the next meaningful and challenging target for measuring LLM reasoning. Whereas olympiad-style problems measure step-by-step reasoning alone, research-level problems use such reasoning to advance the frontier of mathematical knowledge itself, emerging as a compelling alternative. Yet research-level math benchmarks remain scarce because such problems are difficult to source (e.g., Riemann Bench and FrontierMath-Tier 4 contain 25 and 50 problems, respectively). To support reliable evaluation of next-generation frontier models, we introduce Soohak, a 439-problem benchmark newly authored from scratch by 64 mathematicians. Soohak comprises two subsets. On the Challenge subset, frontier models including Gemini-3-Pro, GPT-5, and Claude-Opus-4.5 reach 30.4%, 26.4%, and 10.4% respectively, leaving substantial headroom, while leading open-weight models such as Qwen3-235B, GPT-OSS-120B, and Kimi-2.5 remain below 15%. Notably, beyond standard problem solving, Soohak introduces a refusal subset that probes a capability intrinsic to research mathematics: recognizing ill-posed problems and pausing rather than producing confident but unjustified answers. On this subset, no model exceeds 50%, identifying refusal as a new optimization target that current models do not directly address. To prevent contamination, the dataset will be publicly released in late 2026, with model evaluations available upon request in the interim.

Guijin Son, Seungone Kim, Catherine Arnett +73

Mathematical Reasoning BenchmarksBenchmark DesignLLM Mathematical Reasoning
AskPDF
Mar 11, 2026

mAceReason-Math: A Dataset of High-Quality Multilingual Math Problems Ready For RLVR

Reinforcement Learning with Verifiable Rewards (RLVR) has been successfully applied to significantly boost the capabilities of pretrained large language models, especially in the math and logic problem domains. However, current research and available training datasets remain English-centric. While multilingual training data and benchmarks have been created in the past, they were not created with RLVR and current model capability in mind, and their level of difficulty is often too low to provide appropriate training signals for current models. To address this gap, we provide mAceReason-Math, a dataset of high-quality translations of challenging math problems sourced from a corpus specifically curated for RLVR (AceReason-Math). We further take specific care to clean and improve our translations, resulting in a coverage of 14 languages with more than 10,000 samples per language. We release the dataset to facilitate multilingual RLVR research and benchmarking in the research community.

Konstantin Dobler, Simon Lehnerer, Federico Scozzafava +2

Apple · Hasso Plattner Institute & ELLIS Unit Potsdam

LLM Mathematical ReasoningMathematical ReasoningRL for Language Model Reasoning
AskPDF