Large reasoning models (LRMs) have exhibited remarkable reasoning capabilities through inference-time scaling, but this progress has also introduced considerable redundancy and inefficiency into their reasoning processes, resulting in substantial computational waste. Previous work has attempted to mitigate this issue by penalizing the overall length of generated samples during reinforcement learning (RL), with the goal of encouraging a more concise chains of Efficient reasoning is critical for large reasoning models (LRMs) deployed in agent systems with strict context constraints. Existing methods predominantly rely on global length penalties that overlook step-level structure, inadvertently compressing critical derivations while retaining low-value detours. Hence, we argue for a principled, step-aware alternative that preserves critical steps and compresses low-value detours. However, it raises three fundamental challenges: the difficulty of obtaining step-level supervision, the need for reward signals that induce differentiated length pressure, and the requirement to balance training stability with exploration on complex problems. We propose SmartThinker, a step-level token allocation framework that learns to selectively preserve or compress individual reasoning steps. At its core is Step-Level Length Control Policy Optimization (SCPO), which provides label-free feedback by online estimation of each step's contribution from the change in correct-answer likelihood after removing it. We further introduce reasoning density, defined as contribution per token, to relax length pressure on high-value steps while intensifying it on low-density ones. Finally, SCPO stabilizes training through length-bias-corrected advantages and difficulty-adaptive clipping, achieving superior accuracy-efficiency trade-offs across nine in-domain and out-of-domain benchmarks.
Figures & tables
Figure 1: Comparison of reasoning traces under different length-control settings. Uncompressed reasoning wastes context on redundant detours; global length penalties shorten responses uniformly and can remove critical derivations; Step-level length control compresses low-value steps while preserving critical ones.
Figure 2: Overview of the proposed SmartThinker.
Models
Pass@k
Maj@k
AvgLen.
Red.
Dens.
DeepSeek-R1-1.5B
61.88
51.77
5768
0.0
10.7
Alpha-0.1-1.5B
65.17
53.08
4406
23.6
14.8
Thinkless-1.5B
63.69
51.93
4362
24.4
14.6
SmartThinker-1.5B
65.75
53.69
3651
36.7
18.0
DeepScaleR-1.5B
66.53
57.51
4988
0.0
13.3
L1-Max-1.5B
64.33
53.98
2841
43.0
22.6
Table 1: Average in-domain accuracy-efficiency trade-off. Averages are computed over the six in-domain benchmarks. Red. denotes token reduction relative to the corresponding original backbone; Dens. denotes average Pass@k per 1K generated tokens.
Models
4K
8K
16K
32K
DeepSeek-R1-7B
0.20
0.80
1.60
4.20
Alpha-0.1-7B
0.20
0.80
3.20
3.60
AdaptThink-7B
0.60
0.80
2.80
3.40
SmartThinker-7B
0.60
0.80
3.40
5.60
Table 2: Context-budgeted AIME24 evaluation. Each entry reports the average number of correctly solved problems before the cumulative chat context reaches budget B .
Models
LCB
GPQA
MuSR
Avg.
DeepSeek-R1-1.5B
22.00
16.67
36.80
25.16
Alpha-0.1-1.5B
28.00
21.72
49.60
33.11
Thinkless-1.5B
22.50
16.16
33.20
23.95
SmartThinker-1.5B
32.00
23.74
52.00
35.91
DeepScaleR-1.5B
31.00
22.73
48.40
34.04
L1-Max-1.5B
31.50
34.37
54.00
39.96
Table 3: Out-of-domain Pass@1 accuracy (%). LCB, GPQA, and MuSR denote Live-Code-Bench, GPQA-Diamond, and MuSR-MM, respectively.
Figure 3: Pass@5 accuracy and AvgLen at different training steps on the MATH dataset.
Figure 4: Hyperparameter selection.
Setting
Pass@k
AvgLen
Dens.
DeepSeek-R1-7B
71.51
4987
14.3
+ SFT warm-up
71.13
4001
17.8
+ Step contribution estimator
73.38
3673
20.0
+ Lightweight safeguard
74.92
3861
19.4
+ S-GAE
74.83
3452
21.7
+ Difficulty-adaptive clipping
75.26
3367
22.4
Table 4: Cumulative module-level ablation on DeepSeek-R1-Distill-Qwen-7B. Results are averaged over the six in-domain benchmarks. Each “+” row adds the corresponding component to the previous setting. Dens. denotes average Pass@k per 1K generated tokens.
Models
Pass@k
AvgLen.
Red.
Dens.
SFT+Alpha-0.1-1.5B
65.47
3991
30.8
16.4
SFT+Thinkless-1.5B
60.59
3897
32.4
15.6
SmartThinker-1.5B
65.75
3651
36.7
18.0
SFT+GRPO-ScaleR
64.07
3030
39.3
21.1
SFT+L1-Max-1.5B
63.59
2460
50.7
25.8
LAPO-I-ScaleR †
68.94
3710
25.6
18.6
Table 5: Controlled comparisons under identical SFT warm-up, training data, 4K context, and training steps. Only the RL objective differs across rows; results are averaged over the six in-domain benchmarks. † LAPO-I is the publicly released checkpoint trained without our SFT warm-up.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Reasoning-density distribution under global-level and step-level training. Higher density indicates greater answer contribution per generated token. Compared with global-level length-penalty training, SmartThinker step-level training shifts reasoning toward high-density steps and compresses low-density detours across both easy and hard questions.
Models
AIME25
AIME24
AMC23
MinervaMATH
MATH
Olympiad-Bench
Avg.
Pass@k (%)
DeepSeek-R1-1.5B
43.33
50.00
87.50
44.85
91.20
54.37
61.88
Alpha-0.1-1.5B
40.00
63.33
90.00
49.26
92.00
56.44
65.17
Thinkless-1.5B
40.00
50.00
92.50
45.59
92.00
62.07
63.69
SmartThinker-1.5B
43.33
63.33
95.00
45.22
91.00
56.59
65.75
DeepScaleR-1.5B-Preview
43.33
60.00
95.00
48.53
92.60
59.70
66.53
Appendix
Table 6: Detailed Pass@k, Maj@k, and AvgLen results for 1.5B model families on the six in-domain reasoning benchmarks.
Models
AIME25
AIME24
AMC23
MinervaMATH
MATH
Olympiad-Bench
Avg.
Pass@k (%)
DeepSeek-R1-7B
50.00
73.33
95.00
54.04
95.20
61.48
71.51
Alpha-0.1-7B
60.00
73.33
95.00
54.78
95.60
64.30
73.83
AdaptThink-7B
60.00
73.33
95.00
53.31
95.20
63.56
73.40
SmartThinker-7B
66.67
80.00
95.00
50.37
95.40
64.15
75.26
Maj@k (%)
Appendix
Table 7: Detailed Pass@k, Maj@k, and AvgLen results for 7B models on the six in-domain reasoning benchmarks.
Models
AIME25
AIME24
AMC23
MinervaMATH
MATH
Olympiad-Bench
Avg.
P@k
Tok.
P@k
Tok.
P@k
Tok.
P@k
Tok.
P@k
Tok.
P@k
Tok.
P@k
Tok.
DeepSeek-R1-Distill-Qwen-1.5B backbone
SFT+Alpha-0.1-1.5B
43.33
5925
63.33
5964
90.00
3405
47.79
2491
92.20
1987
56.15
4172
65.47
3991
SFT+Thinkless-1.5B
36.67
5948
43.33
6073
87.50
3281
45.22
2541
89.60
1711
61.19
3825
60.59
3897
SmartThinker-1.5B
43.33
5653
63.33
5310
95.00
3105
45.22
2495
91.00
1643
56.59
3700
65.75
3651
DeepScaleR-1.5B-Preview backbone
Appendix
Table 8: Per-benchmark Pass@k (%) and average response length of the controlled comparisons described in Section 5.2 . Each cell pair reports Pass@k and average token count. † LAPO-I is the publicly released checkpoint trained without our SFT warm-up and is shown only as a frontier reference.
Model
Run 1
Run 2
Run 3
Avg.
DeepSeek-R1-1.5B
0.62
0.58
0.59
0.60
DeepSeek-R1-7B
4.08
4.06
4.05
4.06
Appendix
Table 9: Wall-clock time (seconds) of one online step-contribution evaluation over 8 rollouts (up to 4,096 tokens each), measured over three runs.
Model
GRPO
SCPO
SCPO/GRPO
DeepSeek-R1-1.5B
9.35
9.81
104.92%
DeepSeek-R1-7B
34.92
37.66
107.80%
Appendix
Table 10: Average wall-clock time (seconds) per training step under identical hardware and configuration. SCPO adds less than 8% end-to-end overhead over plain GRPO on both model scales.
Models
B=4K
B=8K
B=16K
B=32K
DeepSeek-R1-7B
1.20
1.60
4.20
16.00
Alpha-0.1-7B
1.20
2.00
6.60
5.60
AdaptThink-7B
1.60
2.00
8.40
11.80
SmartThinker-7B
1.60
2.60
12.20
20.80
Appendix
Table 11: Average number of attempted AIME24 problems under each context budget. Attempts include correctly solved, incorrectly answered, and truncated problems before the run stops.
Figure 6: Effect of lightweight safeguard on reasoning behavior and Pass@k performance across different difficulty levels.
Figure 7: Entropy trends with problem difficulty during training. Because each sample batch updates the policy model for four iterations, both entropy and problem difficulty are averaged over every four steps.
Figure 8: Examples of model responses to questions with straightforward solutions across different baselines.
Figure 9: Examples of model responses to challenging questions across different baselines.
Large Reasoning Models (LRMs) have revolutionized complex problem-solving, yet they exhibit a pervasive "overthinking", generating unnecessarily long reasoning chains. While current solutions improve token efficiency, they often sacrifice fine-grained control or risk disrupting the logical integrity of the reasoning process. To address this, we introduce Stepwise Adaptive Thinking (SAT), a framework that performs step-level, difficulty-aware pruning while preserving the core reasoning structure. SAT formulates reasoning as a Finite-State Machine (FSM) with distinct thinking modes (Slow, Normal, Fast, Skip). It navigates these states dynamically using a lightweight Process Reward Model (PRM), compressing easy steps while preserving depth for hard ones. Experiments across 9 LRMs and 7 benchmarks show that SAT achieves up to 40% reduction in reasoning tokens while generally maintaining or improving accuracy.
Weiyang Huang, Xuefeng Bai, Kehai Chen +4
Harbin Institute of Technology (Shenzhen), Shenzhen, China · Huawei Technologies
Large reasoning models (LRMs) like OpenAI o1 and DeepSeek-R1 achieve high accuracy on complex tasks by adopting long chain-of-thought (CoT) reasoning paths. However, the inherent verbosity of these processes frequently results in redundancy and overthinking. To address this issue, existing works leverage Group Relative Policy Optimization (GRPO) to reduce LRM output length, but their static length reward design cannot dynamically adapt according to the relative problem difficulty and response length distribution, causing over-compression and compromised accuracy. Therefore, we propose SmartThinker, a novel GRPO-based efficient reasoning method with progressive CoT length calibration. SmartThinker makes a two-fold contribution: First, it dynamically estimates the optimal length with peak accuracy during training and guides overlong responses toward it to reduce response length while sustaining accuracy. Second, it dynamically modulates the length reward coefficient to avoid the unwarranted penalization of correct reasoning paths. Extensive experiment results show that SmartThinker achieves up to 52.5% average length compression with improved accuracy, and achieves up to 16.6% accuracy improvement on challenging benchmarks like AIME25. The source code can be found at https://github.com/SJTU-RTEAS/SmartThinker.
Chenzhi Hu, Qinzhe Hu, Yuhang Xu +6
Shanghai Jiao Tong University · Beihang University
Large reasoning models improve with more test-time computation, but often overthink, producing unnecessarily long chains-of-thought that raise cost without improving accuracy. Prior reinforcement learning approaches typically rely on a single outcome reward with trajectory-level length penalties, which cannot distinguish essential from redundant reasoning steps and therefore yield blunt compression. Although recent work incorporates step-level signals, such as offline pruning, supervised data construction, or verifier-based intermediate rewards, reasoning length is rarely treated as an explicit step-level optimization objective during RL. We propose Step-wise Adaptive Penalization (SWAP), a fine-grained framework that allocates length reduction across steps based on intrinsic contribution. We estimate step importance from the model's on-policy log-probability improvement toward the correct answer, then treat excess length as a penalty mass redistributed to penalize low-importance steps more heavily while preserving high-importance reasoning. We optimize with a unified outcome-process advantage within group-relative policy optimization. Extensive experiments demonstrate that SWAP reduces reasoning length by 64.3% on average while improving accuracy by 5.7% relative to the base model.