cs.LGNov 18, 2026

Atomic MathReasoner: Instance-Level Composition of Atomic Reasoning Skills for Mathematical LLMs

Authors: Jiashu Li, Tianyu Gao, Qinglong Liu, Jinping Wang, Ye Wei, Zhiqiang Gao

Abstract

Most existing approaches enhance mathematical reasoning in large language models (LLMs) as a monolithic capability, despite the fact that different mathematical problems require different reasoning skills. Recent work has explored a more fine-grained perspective by decomposing mathematical reasoning into atomic skills and developing specialized modules for individual capabilities. However, how to effectively utilize these atomic abilities remains underexplored. To address this challenge, we propose Atomic MathReasoner (AMR), an instance-level framework for composing atomic reasoning skills. AMR first trains a library of capability-specific LoRA experts and performs gradient-free optimization over held-out validation subsets to discover a diverse set of low-loss composition prototypes. It evaluates the preference of each prototype using the token-level negative log-likelihood of reference solutions, filters ambiguous assignments through a confidence margin, and distills the resulting pseudo-labels into a lightweight router. During inference, the router selects an appropriate composition prototype for each problem, enabling problem-specific utilization of atomic skills without costly online optimization. Experiments on the AtomicMath benchmark across general and mathematics-specialized LLMs demonstrate consistent improvements. On Qwen2.5-Math-Instruct-7B, AMR increases average field accuracy from 64.4% to 68.1% at Level 1 and from 59.3% to 63.7% at Level 2, while improving average logical reasoning performance from 39.4% to 46.8%. These results demonstrate that effectively composing specialized atomic skills provides a promising approach to strengthening mathematical reasoning in LLMs.

Figures & tables

Explore similar work

CardsList
  1. PyraMathBench: Evaluating and Improving Mathematical Capability in Large Language Models

    Jun 2, 2026Zetian Ouyang, Linlin Wang, Gerard de Melo +1Numerical Reasoning in Language ModelsMathematical Reasoning Benchmarks

  2. Adaptive Multi-Expert Reasoning via Difficulty-Aware Routing and Uncertainty-Guided Aggregation

    Apr 11, 2026Mohamed Ehab, Ali HamdiLLM Mathematical ReasoningEfficient Language Model Reasoning

  3. The Missing Primitive: Diagnosing and Repairing Mathematical Reasoning in Large Language Models

    Oct 1, 2026Shuo Xing, Zilin Dai, Chengyuan Qian +7Mathematical Reasoning BenchmarksSelf-Distillation