Mathematical Reasoning Benchmarks

Follow this topic Ask about this topic

Latest papers 108

11 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

All topics

A dataset of formalised theorem statements from the Annals of Mathematics
Oct 13, 2026Thomas Browning, Katerina Hristova, David Ledvinka +4Autoformalization Mathematical Reasoning Benchmarks
Department of Mathematics, Imperial College London · Department of Philosophy, Carnegie Mellon University
SqueezeBench: Saturation-Resilient Evaluation of Mathematical Capability
Oct 9, 2026Pablo Manrique, Hai Xia, Stefan SzeiderMathematical Reasoning Benchmarks Benchmark Design
Algorithms and Complexity Group TU Wien, Vienna, Austria
MathVis: Can AI Reason through Drawing? A Benchmark for Mathematical Figure Generation
Oct 9, 2026Jingjing Wei, Hao Zhao, Xinke Li +3Mathematical Reasoning Benchmarks LLM Mathematical Reasoning
Zhihu (zhihu.com) · Tsinghua University (tsinghua.edu.cn) · City University of Hong Kong (cityu.edu.hk) +1
OpenProblemBench: Benchmarking AI on Open Problems in the Foundational Theoretical Sciences
Oct 8, 2026Zhiyi Li, Sihan Hu, Tianning Xiao +4AI for Science Scientific Reasoning in Language Models
Department of Modern Physics, University of Science and Technology of China, Hefei, Anhui 230026, China · CAS Key Laboratory of Theoretical Physics, Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing 100190, China · Hefei National Laboratory, University of Science and Technology of China, Hefei 230088, China +3
AIProver: Agentic Auto-Formalization of Mathematical Research via Certificate-Driven Evolving Harness
Oct 4, 2026Prithwish Jana, Viet Bach Hoang, Logan Luna +9Mathematical Reasoning Benchmarks Automated Theorem Proving
Georgia Institute of Technology, USA · University of Pennsylvania, USA · Foothill College, USA +3
Multilingual GSM-Symbolic: What determines capability transfer across languages?
Oct 2, 2026Kenneth Enevoldsen, Riley Herchert, Sofie Mosegaard +22Mathematical Reasoning Benchmarks Multilingual Language Model Evaluation
Aarhus University · Danish Foundation Models · University of Alabama +13
The Missing Primitive: Diagnosing and Repairing Mathematical Reasoning in Large Language Models
Oct 1, 2026Shuo Xing, Zilin Dai, Chengyuan Qian +7Mathematical Reasoning Benchmarks Self-Distillation
Texas A&M University · Harvard University · Vanderbilt University +3
Budget Boundary Effects in Test-Time Mathematical Reasoning
Sep 30, 2026Guilin Zhang, Ziqi Tan, Wulan Guo +5Mathematical Reasoning Benchmarks Test-Time Optimization
Yantong AI
TCSAlgBench: Benchmarking Automated Proving for Research-Level Theoretical Computer Science
Sep 28, 2026Chutong Yang, Xiyuan Zhang, Yu Huang +7Mathematical Reasoning Benchmarks Automated Theorem Proving
Department of Computer Science, The University of Texas at Austin · Amazon · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania
Solving Every Step Is Not Enough: Milestone Oracles Reveal a Composition Gap in LLM Math Reasoning
Sep 26, 2026Zhuohan Wang, Haoran Ma, Tianyu Wu +3LLM Mathematical Reasoning LLM Evaluation
Harvard University · Rutgers University · The Hong Kong University of Science and Technology
Math Reasoning in LLMs is Organized by Approach, Not Topic
Sep 22, 2026Sajad Goudarzi, Samaneh Zamanifard, Moloud Nasiri +1Mathematical Reasoning Benchmarks LLM Interpretability
Clemson University
The Endless Exam: Mathematical Constructions from Today's Models toward Superintelligence
Sep 21, 2026Muhan ZhangMathematical Reasoning Benchmarks Benchmark Construction
The University of Texas at Arlington
ChitraMiti: Benchmarking Visual Grounding and Modality Reliance in Bengali Geometric Reasoning
Sep 11, 2026Khan Raiyan Ibne Reza, Sanjana Aktar Maria, Sumaiya Tabassum Nimi +1Mathematical Reasoning Benchmarks Multilingual VLM Evaluation
North South University
An Open Recipe for IMO Gold: Training Nemotron for Olympiad Mathematics
Sep 9, 2026Ivan Moshkov, Stephen Ge, George Armstrong +3Mathematical Reasoning Benchmarks Test-Time Scaling
NVIDIA
StochBench: A Domain-Specific Benchmark for Stochastic Processes in Lean
Sep 8, 2026Idan Davidovich, Debargha Ganguly, Vikash Singh +1Mathematical Reasoning Benchmarks Lean Theorem Proving
Case Western Reserve University
FrontierMath Erdős
Sep 6, 2026Tom Adamczewski, Thomas F. BloomMathematical Reasoning Benchmarks LLM Mathematical Reasoning
Epoch AI · University of Manchester
RePro: Proof-Verified Benchmark Rewriting for Reliable Evaluation of LLM Mathematical Problem Solving
Aug 30, 2026Xiyuan Zhou, Zhuoqi Li, Xinlei Wang +6Mathematical Reasoning Benchmarks LLM Mathematical Reasoning
Nanyang Technological University · The Chinese University of Hong Kong, Shenzhen · INSAIT, Sofia University “St. Kliment Ohridski” +2
Where Induction Runs Out: Description-Length Difficulty and the Memorisation Gap in Integer-Sequence Benchmarks
Aug 29, 2026Sabilashan GaneshanMathematical Reasoning Benchmarks LLM Evaluation
Independent Researcher
From Atomic to Agentic: Towards Interpretable Evaluation of LLMs' Agentic Mathematical Capabilities
Aug 27, 2026Jiayi Kuang, Yinghui Li, Yunze Song +11LLM Agent Evaluation Mathematical Reasoning Benchmarks
Sun Yat-sen University · Tencent Youtu Lab · Pengcheng Laboratory +1
MathAdv: What Theorem Provers Know, Reason, Formalize, and Generalize
Aug 26, 2026Jiaxin Yuan, Connor Martinez Lockhart, Xiaoyu Liu +11Mathematical Reasoning Benchmarks Automated Theorem Proving
University of Maryland, College Park · Georgia Institute of Technology · Cornell University +2
MetaReason: Precise Interleaved Multimodal Reasoning via Editing Meta Information for Solving Geometry Problems
Aug 15, 2026Penghao Yin, Haomin Wang, Qihong Tang +3Multimodal Reasoning Mathematical Reasoning
Tsinghua University · Shanghai AI Laboratory · Shanghai Jiao Tong University +1
Ask-E: An Environment for Calibrated Question Generation
Aug 7, 2026Sarah Pratt, Jae Sung Park, Scott Geng +1Mathematical Reasoning Benchmarks Synthetic Benchmark Generation
University of Washington · 2Allen Institute for AI
TREAT: Evaluating Access to Formal Knowledge across Equivalent Mathematical Representations
Jul 29, 2026Fateme Mazdarani, Carlos ToxtliMathematical Reasoning Benchmarks Language Model Robustness
School of Computing Clemson University Clemson, SC, USA
A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models
Jul 28, 2026Vivek Shukla, Varun Shukla, Atul +2CoT Faithfulness Mathematical Reasoning Benchmarks
Allenhouse Institute of Technology Kanpur, India
PatiGonit22K: A Comprehensive Dataset for Solving Complex Bengali MWPs
Jul 24, 2026Swastika Kundu, Azizul Hakim Fayaz, Tashreef MuhammadMathematical Reasoning Benchmarks Mathematical Reasoning
Department of Computer Science and Engineering Ahsanullah University of Science and Technology, Dhaka, Bangladesh · Department of Computer Science and Engineering Southeast University, Dhaka, Bangladesh
AIMO Interpretability Challenge
Jul 15, 2026Michal Štefánik, Philipp Mondorf, Andreas Waldis +11Mathematical Reasoning Benchmarks LLM Interpretability
National Institute of Informatics, Japan · Munich Center for Machine Learning / MaiNLP LMU · University of Tübingen +6
GSM-Plus-BN: A Perturbation-Based Benchmark for Bangla Mathematical Reasoning in Large Language Models
Jul 14, 2026Bidyarthi Paul, Nahida Jannat Mayouree, Md. Asif Karim +2Mathematical Reasoning Benchmarks Multilingual Language Model Evaluation
Computer Science and Engineering Southeast University Dhaka, Bangladesh · Computer Science and Engineering Ahsanullah University of Science and Technology Dhaka, Bangladesh
AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification
Jul 13, 2026Lingkai Kong, Zijian Wu, Yuzhe Gu +11Mathematical Reasoning Benchmarks Automated Theorem Proving
Shanghai AI Laboratory · Shanghai Jiao Tong University · MMLab, The Chinese University of Hong Kong +1
MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning
Jul 8, 2026Charbel Al Bateh, Samer SaabMathematical Reasoning Benchmarks Synthetic Benchmark Generation
Department of Electrical and Computer Engineering Lebanese American University Byblos, Lebanon
Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics
Jul 7, 2026Pavel Snopov, German MagaiMathematical Reasoning Benchmarks Knowledge Augmentation for Language Models
School of Mathematical and Statistical Sciences, The University of Texas Rio Grande Valley, USA · Noeon Research, Tokyo, Japan.

Oct 13, 2026J/K move · Enter open · S save

A dataset of formalised theorem statements from the Annals of Mathematics

Thomas Browning, Katerina Hristova, David Ledvinka, Justus Springer+3

Department of Mathematics, Imperial College London · Department of Philosophy, Carnegie Mellon University

Autoformalization Mathematical Reasoning Benchmarks Lean Theorem Proving

Open paper AskPDF arXiv

Recent progress in autoformalisation of mathematics using AI has been impressive. Measuring this progress requires datasets of formal statements that are both hard and trustworthy. We present AnnalsChallenge: 50 formalised theorem statements written by hand in Lean 4 using Mathlib. Each is the main result of a paper published in the Annals of Mathematics since 2020. Every formalisation was manually reviewed and then audited by AI. We release them without formalised proofs.