cs.AIJul 6, 2025

SmartThinker: Learning High-Density Reasoning via Step-Level Token Allocation

Authors: Xingyang He, Xiao Ling, Jie Liu

Organizations: College of Artificial Intelligence, Nankai University

Abstract

Large reasoning models (LRMs) have exhibited remarkable reasoning capabilities through inference-time scaling, but this progress has also introduced considerable redundancy and inefficiency into their reasoning processes, resulting in substantial computational waste. Previous work has attempted to mitigate this issue by penalizing the overall length of generated samples during reinforcement learning (RL), with the goal of encouraging a more concise chains of Efficient reasoning is critical for large reasoning models (LRMs) deployed in agent systems with strict context constraints. Existing methods predominantly rely on global length penalties that overlook step-level structure, inadvertently compressing critical derivations while retaining low-value detours. Hence, we argue for a principled, step-aware alternative that preserves critical steps and compresses low-value detours. However, it raises three fundamental challenges: the difficulty of obtaining step-level supervision, the need for reward signals that induce differentiated length pressure, and the requirement to balance training stability with exploration on complex problems. We propose SmartThinker, a step-level token allocation framework that learns to selectively preserve or compress individual reasoning steps. At its core is Step-Level Length Control Policy Optimization (SCPO), which provides label-free feedback by online estimation of each step's contribution from the change in correct-answer likelihood after removing it. We further introduce reasoning density, defined as contribution per token, to relax length pressure on high-value steps while intensifying it on low-density ones. Finally, SCPO stabilizes training through length-bias-corrected advantages and difficulty-adaptive clipping, achieving superior accuracy-efficiency trade-offs across nine in-domain and out-of-domain benchmarks.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SAT: Balancing Reasoning Accuracy and Efficiency with Stepwise Adaptive Thinking

    Apr 9, 2026Weiyang Huang, Xuefeng Bai, Kehai Chen +4Efficient Language Model ReasoningOverthinking in Language Models

  2. Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning

    Feb 27, 2026Xintong Li, Sha Li, Rongmei Lin +10Efficient Language Model ReasoningRL for Language Model Reasoning