Numerical Reasoning in Language Models

Overview

Numerical reasoning asks whether language models can read quantities, perform arithmetic, compare values, and use numerical evidence in mathematical and practical settings. Researchers test these abilities with math benchmarks such as GSM8K and MATH, as well as table, spreadsheet, financial, scientific, clinical, and multilingual tasks. Approaches include digit-aware representations, chain-of-thought and code execution, calculator and retrieval tools, symbolic solvers, and supervised or reinforcement-learning training with verifiable rewards. Mechanistic analyses examine whether numbers are merely represented or actually used, while tests of formatting, distractors, long contexts, units, and distribution shifts reveal reliability gaps; a central tension is fluent explanations versus consistently correct, auditable computation.

Latest papers

CardsList

168 papers14 papers in the last four weeks, down 62% on the four weeks before. 0.1% of all new papers.

  1. Transferring an analysis of human brain organization to language models

    Nov 18, 2026Donghyeok Lee, Sangwon Yu, Sangkwon Park +3Concept Localization in Language ModelsNumerical Reasoning in Language Models

  2. CivilBench: Benchmarking Vision-Language Models for Multimodal Information Integration in Civil Engineering

    Nov 16, 2026Dan Yuan, Yue Wen, Zhili He +3VLM EvaluationNumerical Reasoning in Language Models

  3. Linear Probes to Bayesian Manifolds: Geometry of Arithmetic in Large Language Models

    Nov 8, 2026Anshul Kumar, Neha Prasad, Manoj Kumar +2LLM InterpretabilityRepresentation Geometry in Language Models

  4. Localization Does Not Imply Steerability: A Mechanistic Study of Sign Errors in LLM Mathematics

    Nov 5, 2026Shradha Agarwal, Tariq J., Deepak RajbharLLM InterpretabilityCircuits in Language Models

  5. "Close enough?" An investigation into multiplication errors in LLMs

    Oct 28, 2026William Bolton, Bezawit Tilaye, Huw DayNumerical Reasoning in Language ModelsLLM Reliability

  6. Assessing Language Models for Scientific Applications: Inertial Confinement Fusion as a Use Case

    Oct 27, 2026P. B. Radha, D. D. Meyerhofer, M. Bhattarai +11Inertial Confinement FusionScientific Reasoning in Language Models

  7. Understanding Large Language Models as Numerical Optimizers

    Oct 27, 2026Lingyun Guo, Ali BereyhiLarge Language Model-Guided OptimizationNumerical Reasoning in Language Models

  8. Are Linguistic and Arithmetic Circuits Dissociated in LLMs?

    Oct 20, 2026Riku Kisako, Tatsuki Kuribayashi, Ryohei SasanoModular Language Model ArchitecturesCircuits in Language Models

  9. TIRE: Top-Down Formula Instantiation with Reliable Execution for Physics Problem Synthesis

    Oct 18, 2026Xiaoyu Wu, Zhaoxin Yu, Xinglin Xiao +1Synthetic Data Generation for Language ModelsScientific Reasoning in Language Models

  10. P2N: Reusing Deep Representations for Greater Effective Depth

    Oct 18, 2026Yiqin Hao, Boyi Zeng, Zitong Wang +7Efficient Language Model InferenceRecurrent Transformers

  11. CME-SpectrumBench: Can LLMs Analyze Condensed Matter Spectral Data?

    Oct 16, 2026Jin Gene Wong, Anjney Midha, Joseph Mathew Tennyson +3LLM EvaluationScientific Reasoning in Language Models

  12. Bellman-Consistent Trajectory RL for LLM Arithmetic Reasoning

    Oct 15, 2026Lianfa LiRL for Language Model ReasoningReinforcement Learning

  13. INTACT: Inducing Latent Reasoning on a Frozen Backbone

    Oct 14, 2026Kutay Tire, Atula Tejaswi, Haris Vikalo +1Efficient Language Model ReasoningImplicit Reasoning in Language Models

  14. Dynamic K-Head: Difficulty-Supervised Budget Selection for Continuous Latent Reasoning

    Oct 13, 2026Ying Li, Yuqi Chen, Sixuan Li +1Efficient Language Model ReasoningInference-Time Compute Allocation

  15. TPPO: Tracking-Value PPO For Language Model Reasoning

    Oct 12, 2026Futing Wang, Chenxi Li, Jianhao Yan +6RL for Language ModelsRL for Language Model Reasoning

  16. MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis

    Oct 8, 2026Ximing Lu, Aviral Kumar, Jan Kautz +2Reinforcement Learning with Verifiable RewardsSynthetic Data Generation for Language Models

  17. COMPASS: Finding Where Reasoning Lives in Language Models

    Oct 5, 2026Pratyay Dutta, Kowshik Thopalli, Vivek NarayanaswamyEfficient Language Model ReasoningActivation Steering