LLM Mathematical Reasoning

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

19 papers in the last four weeks, up 375% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 194

All topics

ZAYA1-8B Technical Report
May 6, 2026Robert Washbourne, Rishi Iyer, Tomas Figliolia +15Mixture-of-Experts Language Models RL for Language Model Reasoning
Zyphra San Francisco, CA
Automated Formal Proofs of Combinatorial Identities via Wilf-Zeilberger Guidance and LLMs
May 6, 2026Beibei Xiong, Hangyu Lv, Junqi Liu +5Automated Theorem Proving Lean Theorem Proving
School of Software Engineering, East China Normal University, Shanghai, China · Institute of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China · School of Computer and Information Engineering, Henan University, Kaifeng, China
Probing Structural Mathematical Reasoning in Language Models with Algebraic Trapdoors
May 5, 2026Igor RivinMathematical Reasoning Benchmarks Mathematical Reasoning
Igor Rivin
RAG over Thinking Traces Can Improve Reasoning Tasks
May 5, 2026Negar Arabzadeh, Wenjie Ma, Sewon Min +1Retrieval-Augmented Generation LLM Mathematical Reasoning
University of California, Berkeley
Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
May 1, 2026Sheridan Feucht, Tal Haklay, Usha Bhalla +9Numerical Reasoning in Language Models LLM Interpretability
aNortheastern University · bTechnion IIT · cHarvard University +1
Evaluating the Architectural Reasoning Capabilities of LLM Provers via the Obfuscated Natural Number Game
May 1, 2026Lixing LiMathematical Reasoning Benchmarks Lean Theorem Proving
St Anne’s College, Oxford, OX26HS, United Kingdom
Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs
May 1, 2026Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger +4Mathematical Reasoning Benchmarks LLM Evaluation
ETH Zurich · INSAIT · Sofia University "St. Kliment Ohridski"
When to Vote, When to Rewrite: Disagreement-Guided Strategy Routing for Test-Time Scaling
Apr 29, 2026Zhimin Lin, Yixin Ji, Jinpeng Li +5Test-Time Scaling LLM Routing
School of Computer Science and Technology, Soochow University · Department of Foundation Model, 2012 Labs, Huawei · 3Harbin Institute of Technology, Shenzhen (HITSZ)
QED: An Open-Source Multi-Agent System for Generating Mathematical Proofs on Open Problems
Apr 27, 2026Chenyang An, Qihao Ye, Minghao Pan +1AI Agents for Scientific Discovery Automated Theorem Proving
University of California, San Diego, La Jolla, USA · California Institute of Technology, Pasadena, USA · Amazon Web Services
MathDuels: Evaluating LLMs as Problem Posers and Solvers
Apr 23, 2026Zhiqiu Xu, Shibo Jin, Shreya Arya +1Mathematical Reasoning Benchmarks LLM Evaluation
Department of Computer and Information Science, University of Pennsylvania · Rabdos AI · Department of Mathematics, University of Pennsylvania
Thinking with Reasoning Skills: Fewer Tokens, More Accuracy
Apr 23, 2026Guangxiang Zhao, Qilong Shi, Xusen Xiao +3Efficient Language Model Reasoning Efficient Language Model Inference
Qiyuan Tech · Tsinghua University · The University of Hong Kong +1
How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning
Apr 21, 2026Zhiyuan Zhai, Xinkai You, Wenjing Yan +1LLM Inference Efficiency RL for Language Model Reasoning
Fudan University · The Chinese University of Hong Kong
OLLM: Options-based Large Language Models
Apr 21, 2026Shashank Sharma, Janina Hoffmann, Vinay NamboodiriLLM Alignment RL for Language Model Reasoning
Department of Computer Science University of Bath · Department of Psychology University of Bath
Decompose, Structure, and Repair: A Neuro-Symbolic Framework for Autoformalization via Operator Trees
Apr 21, 2026Xiaoyang Liu, Zineng Dong, Yifan Bai +3LLM Self-Correction Neuro-Symbolic Reasoning
School of Mathematical Sciences, Shanghai Jiao Tong University · Zhiyuan College, Shanghai Jiao Tong University · Institute of Natural Sciences, MOE-LSC, CMA-Shanghai, Shanghai Jiao Tong University
Less Is More: Cognitive Load and the Single-Prompt Ceiling in LLM Mathematical Reasoning
Apr 20, 2026Manuel Israel CazaresPrompt Sensitivity LLM Prompting
Bytepro AI Mazatlán, Sinaloa, Mexico
MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval
Apr 20, 2026Shaden Alshammari, Kevin Wen, Abrar Zainal +5Mathematical Reasoning Benchmarks Multilingual Language Model Evaluation
MIT · HUMAIN · Individual Researcher +1
TPS-CalcBench: A Benchmark and Diagnostic Evaluation Framework for LLM Analytical Calculation Competence in Hypersonic Thermal Protection System Engineering
Apr 20, 2026Jinglai Zheng, Chuhan Qiao, Haiming HuangSynthetic Benchmark Generation LLM Safety Evaluation
School of Civil Engineering, Beijing Jiaotong University, Beijing 100044, China
Yanasse: Finding New Proofs from Deep Vision's Analogies, Part 1
Apr 19, 2026Alexandre LinharesAutomated Theorem Proving Analogical Reasoning
Getulio Vargas Foundation · ARGO LABS
Learning to Reason with Insight for Informal Theorem Proving
Apr 17, 2026Yunhe Li, Hao Shi, Bowen Deng +8Automated Theorem Proving RL for Language Model Reasoning
City University of Hong Kong · Tsinghua University · Ke Holdings Inc. +2
Disentangling Mathematical Reasoning in LLMs: A Methodological Investigation of Internal Mechanisms
Apr 17, 2026Tanja Baeumel, Josef van Genabith, Simon OstermannLLM Interpretability Mechanistic Interpretability
German Research Center for Artificial Intelligence (DFKI) · Saarland University · Center for European Research in Trusted AI (CERTAIN)
Placing Puzzle Pieces Where They Matter: A Question Augmentation Framework for Reinforcement Learning
Apr 17, 2026Yangyi Fang, Jiaye Lin, Xiaoliang Fu +2RL for Language Model Reasoning LLM Mathematical Reasoning
Tsinghua University · Fudan University · Peking University
CoTEvol: Self-Evolving Chain-of-Thoughts for Data Synthesis in Mathematical Reasoning
Apr 16, 2026Zhuo Wang, Zhuo Zhang, Yafu Li +3Evolutionary Optimization LLM Training
1Fudan University · 5Independent Researcher · 3The Chinese University of Hong Kong +3
Riemann-Bench: A Benchmark for Moonshot Mathematics
Apr 8, 2026Suhaas Garre, Erik Knutsen, Sushant Mehta +1Mathematical Reasoning Benchmarks Benchmark Design
Surge AI
LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
Apr 2, 2026Linyang He, Qiyao Yu, Hanze Dong +5Mathematical Reasoning Benchmarks LLM Evaluation
Columbia University · Microsoft Research · University of Amsterdam
LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematics
Feb 27, 2026Antoine Peyronnet, Fabian Gloeckle, Amaury HayatMathematical Reasoning Benchmarks Automated Theorem Proving
1Ecole Normale Superieure de Rennes · 2Ecole des Ponts Paris · 3Korean Institute for Advanced Study
Ontology-Guided Neuro-Symbolic Inference: Grounding Language Models with Mathematical Domain Knowledge
Feb 19, 2026Marcelo LabreRetrieval-Augmented Generation LLM Grounding
Advanced Institute for Artificial Intelligence (AI2)
MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation
Jan 29, 2026Tianyi Xu, Kosei Uemura, Alfred Malengo Kondoro +6Numerical Reasoning in Language Models Mathematical Reasoning Benchmarks
McGill University · Mila-Quebec AI Institute · University of Toronto +7
The Effect of Scripts and Formats on LLM Numeracy
Jan 21, 2026Varshini Reddy, Craig W. Schmidt, Seth Ebner +3Numerical Reasoning in Language Models Multilingual Language Model Evaluation
Kensho Technologies · Ben-Gurion University · MIT Cambridge, MA
MINIF2F-DAFNY: LLM-Guided Mathematical Theorem Proving via Auto-Active Verification
Dec 11, 2025Mantas Baksys, Stefan Zetzsche, Olivier Bouissou +1Automated Theorem Proving Formal Verification
University of Cambridge, Cambridge, UK · Amazon Web Services, London, UK · Amazon Web Services, Boston, USA
ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth
Dec 2, 2025Salman Rahman, Sruthi Gorantla, Arpit Gupta +3Process Reward Models Reinforcement Learning with Verifiable Rewards
2UCLA · Work done while as an intern at Amazon AGI. · 1Amazon AGI

May 6, 2026, cs.AIJ/K move · Enter open · S save

ZAYA1-8B Technical Report

Robert Washbourne, Rishi Iyer, Tomas Figliolia, Henry Zheng+14

Zyphra San Francisco, CA

Mixture-of-Experts Language Models RL for Language Model Reasoning Test-Time Scaling Efficient Language Model Reasoning LLM Mathematical Reasoning Large Reasoning Models

Open paper AskPDF arXiv

We present ZAYA1-8B, a reasoning-focused mixture-of-experts (MoE) model with 700M active and 8B total parameters, built on Zyphra's MoE++ architecture. ZAYA1-8B's core pretraining, midtraining, and supervised fine-tuning (SFT) were performed on a full-stack AMD compute, networking, and software platform. With under 1B active parameters, ZAYA1-8B matches or exceeds DeepSeek-R1-0528 on several challenging mathematics and coding benchmarks, and remains competitive with substantially larger open-weight reasoning models. ZAYA1-8B was trained from scratch for reasoning, with reasoning data included from pretraining onward using an answer-preserving trimming scheme. Post-training uses a four-stage RL cascade: reasoning warmup on math and puzzles; a 400-task RLVE-Gym curriculum; math and code RL with test-time compute traces and synthetic code environments built from competitive-programming references; and behavioral RL for chat and instruction following. We also introduce Markovian RSA, a test-time compute method that recursively aggregates parallel reasoning traces while carrying forward only bounded-length reasoning tails between rounds. In TTC evaluation, Markovian RSA raises ZAYA1-8B to 91.9% on AIME'25 and 89.6% on HMMT'25 while carrying forward only a 4K-token tail, narrowing the gap to much larger reasoning models including Gemini-2.5 Pro, DeepSeek-V3.2, and GPT-5-High.

Momentum

Related topics

Latest papers 194