LLM Mathematical Reasoning

LLM: Large Language Model

The field investigates whether language models can carry out arithmetic, multi-step problem solving, and proof construction reliably—not just produce correct-looking answers. Common approaches include chain-of-thought prompting, supervised and reinforcement learning, test-time search and voting, tool use with code or computer algebra, and formal verification with proof assistants such as Lean. Evaluation spans word-problem and contest benchmarks such as GSM8K, MATH, and AIME, as well as formal proof tasks like miniF2F; newer tests probe robustness to equivalent transformations, multilingual inputs, and research-level problems. Models can perform strongly on familiar contest tasks, but generalization, faithful reasoning, proof quality, and reliable verification remain unresolved tensions, especially on open-ended research mathematics.

Latest papers

273 papers20 papers in the last four weeks, down 5% on the four weeks before. 0.1% of all new papers.

Multidimensional Consistency Improves Reasoning in Language Models
Mar 4, 2025Huiyuan Lai, Xiao Zhang, Malvina NissimReasoning Consistency in Language Models LLM Mathematical Reasoning
CLCG, University of Groningen / The Netherlands
The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer
Feb 21, 2025Marthe Ballon, Andres Algaba, Vincent GinisCoT Reasoning Efficient Language Model Reasoning
Data Analytics Lab, Vrije Universiteit Brussel, Brussel 1050, Belgium · School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA
Benchmarking LLMs' Mathematical Reasoning with Unseen Random Variables Questions
Jan 20, 2025Zijin Hong, Hao Wu, Su Dong +8Mathematical Reasoning Benchmarks Language Model Generalization
The Hong Kong Polytechnic University, Hong Kong SAR, China · University of Electronic Science and Technology of China, Chengdu, China · Tencent Youtu Lab, Shanghai, China +2

Mar 4, 2025J/K move · Enter open · S save

Multidimensional Consistency Improves Reasoning in Language Models

Huiyuan Lai, Xiao Zhang, Malvina Nissim

CLCG, University of Groningen / The Netherlands

Reasoning Consistency in Language Models LLM Mathematical Reasoning Prompt Sensitivity

Open paper AskPDF arXiv

While Large language models (LLMs) have proved able to address some complex reasoning tasks, we also know that they are highly sensitive to input variation, which can lead to different solution paths and final answers. Answer consistency across input variations can thus be taken as a sign of stronger confidence. Leveraging this aspect, we introduce a framework where models are systematically pushed to diversify solution paths towards a final answer, thereby testing them for answer consistency across multiple input variations. Focusing on math problems, we induce variations in (i) order of shots in prompt, (ii) problem phrasing, and (iii) languages used. Experiments on a wide range of open-source LLMs of various sizes show that reasoning consistency differs by variation dimension, and that by aggregating consistency across dimensions, our framework enhances mathematical reasoning performance on monolingual datasets GSM8K and MATH500, and the multilingual dataset MGSM. We also discuss the different efficacy of the types of variations induced and how they could be further leveraged in future work.

Latest papers

Multidimensional Consistency Improves Reasoning in Language Models

The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer

Benchmarking LLMs' Mathematical Reasoning with Unseen Random Variables Questions