Large reasoning models (LRMs) have exhibited remarkable reasoning capabilities through inference-time scaling, but this progress has also introduced considerable redundancy and inefficiency into their reasoning processes, resulting in substantial computational waste. Previous work has attempted to mitigate this issue by penalizing the overall length of generated samples during reinforcement learning (RL), with the goal of encouraging a more concise chains of Efficient reasoning is critical for large reasoning models (LRMs) deployed in agent systems with strict context constraints. Existing methods predominantly rely on global length penalties that overlook step-level structure, inadvertently compressing critical derivations while retaining low-value detours. Hence, we argue for a principled, step-aware alternative that preserves critical steps and compresses low-value detours. However, it raises three fundamental challenges: the difficulty of obtaining step-level supervision, the need for reward signals that induce differentiated length pressure, and the requirement to balance training stability with exploration on complex problems. We propose SmartThinker, a step-level token allocation framework that learns to selectively preserve or compress individual reasoning steps. At its core is Step-Level Length Control Policy Optimization (SCPO), which provides label-free feedback by online estimation of each step's contribution from the change in correct-answer likelihood after removing it. We further introduce reasoning density, defined as contribution per token, to relax length pressure on high-value steps while intensifying it on low-density ones. Finally, SCPO stabilizes training through length-bias-corrected advantages and difficulty-adaptive clipping, achieving superior accuracy-efficiency trade-offs across nine in-domain and out-of-domain benchmarks.
Figures & tables
Figure 1: Comparison of reasoning traces under different length-control settings. Uncompressed reasoning wastes context on redundant detours; global length penalties shorten responses uniformly and can remove critical derivations; Step-level length control compresses low-value steps while preserving critical ones.
Figure 2: Overview of the proposed SmartThinker.
Models
Pass@k
Maj@k
AvgLen.
Red.
Dens.
DeepSeek-R1-1.5B
61.88
51.77
5768
0.0
10.7
Alpha-0.1-1.5B
65.17
53.08
4406
23.6
14.8
Thinkless-1.5B
63.69
51.93
4362
24.4
14.6
SmartThinker-1.5B
65.75
53.69
3651
36.7
18.0
DeepScaleR-1.5B
66.53
57.51
4988
0.0
13.3
L1-Max-1.5B
64.33
53.98
2841
43.0
22.6
Table 1: Average in-domain accuracy-efficiency trade-off. Averages are computed over the six in-domain benchmarks. Red. denotes token reduction relative to the corresponding original backbone; Dens. denotes average Pass@k per 1K generated tokens.
Models
4K
8K
16K
32K
DeepSeek-R1-7B
0.20
0.80
1.60
4.20
Alpha-0.1-7B
0.20
0.80
3.20
3.60
AdaptThink-7B
0.60
0.80
2.80
3.40
SmartThinker-7B
0.60
0.80
3.40
5.60
Table 2: Context-budgeted AIME24 evaluation. Each entry reports the average number of correctly solved problems before the cumulative chat context reaches budget B .
Models
LCB
GPQA
MuSR
Avg.
DeepSeek-R1-1.5B
22.00
16.67
36.80
25.16
Alpha-0.1-1.5B
28.00
21.72
49.60
33.11
Thinkless-1.5B
22.50
16.16
33.20
23.95
SmartThinker-1.5B
32.00
23.74
52.00
35.91
DeepScaleR-1.5B
31.00
22.73
48.40
34.04
L1-Max-1.5B
31.50
34.37
54.00
39.96
Table 3: Out-of-domain Pass@1 accuracy (%). LCB, GPQA, and MuSR denote Live-Code-Bench, GPQA-Diamond, and MuSR-MM, respectively.
Figure 3: Pass@5 accuracy and AvgLen at different training steps on the MATH dataset.
Figure 4: Hyperparameter selection.
Setting
Pass@k
AvgLen
Dens.
DeepSeek-R1-7B
71.51
4987
14.3
+ SFT warm-up
71.13
4001
17.8
+ Step contribution estimator
73.38
3673
20.0
+ Lightweight safeguard
74.92
3861
19.4
+ S-GAE
74.83
3452
21.7
+ Difficulty-adaptive clipping
75.26
3367
22.4
Table 4: Cumulative module-level ablation on DeepSeek-R1-Distill-Qwen-7B. Results are averaged over the six in-domain benchmarks. Each “+” row adds the corresponding component to the previous setting. Dens. denotes average Pass@k per 1K generated tokens.
Models
Pass@k
AvgLen.
Red.
Dens.
SFT+Alpha-0.1-1.5B
65.47
3991
30.8
16.4
SFT+Thinkless-1.5B
60.59
3897
32.4
15.6
SmartThinker-1.5B
65.75
3651
36.7
18.0
SFT+GRPO-ScaleR
64.07
3030
39.3
21.1
SFT+L1-Max-1.5B
63.59
2460
50.7
25.8
LAPO-I-ScaleR †
68.94
3710
25.6
18.6
Table 5: Controlled comparisons under identical SFT warm-up, training data, 4K context, and training steps. Only the RL objective differs across rows; results are averaged over the six in-domain benchmarks. † LAPO-I is the publicly released checkpoint trained without our SFT warm-up.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Reasoning-density distribution under global-level and step-level training. Higher density indicates greater answer contribution per generated token. Compared with global-level length-penalty training, SmartThinker step-level training shifts reasoning toward high-density steps and compresses low-density detours across both easy and hard questions.
Models
AIME25
AIME24
AMC23
MinervaMATH
MATH
Olympiad-Bench
Avg.
Pass@k (%)
DeepSeek-R1-1.5B
43.33
50.00
87.50
44.85
91.20
54.37
61.88
Alpha-0.1-1.5B
40.00
63.33
90.00
49.26
92.00
56.44
65.17
Thinkless-1.5B
40.00
50.00
92.50
45.59
92.00
62.07
63.69
SmartThinker-1.5B
43.33
63.33
95.00
45.22
91.00
56.59
65.75
DeepScaleR-1.5B-Preview
43.33
60.00
95.00
48.53
92.60
59.70
66.53
Appendix
Table 6: Detailed Pass@k, Maj@k, and AvgLen results for 1.5B model families on the six in-domain reasoning benchmarks.
Models
AIME25
AIME24
AMC23
MinervaMATH
MATH
Olympiad-Bench
Avg.
Pass@k (%)
DeepSeek-R1-7B
50.00
73.33
95.00
54.04
95.20
61.48
71.51
Alpha-0.1-7B
60.00
73.33
95.00
54.78
95.60
64.30
73.83
AdaptThink-7B
60.00
73.33
95.00
53.31
95.20
63.56
73.40
SmartThinker-7B
66.67
80.00
95.00
50.37
95.40
64.15
75.26
Maj@k (%)
Appendix
Table 7: Detailed Pass@k, Maj@k, and AvgLen results for 7B models on the six in-domain reasoning benchmarks.
Models
AIME25
AIME24
AMC23
MinervaMATH
MATH
Olympiad-Bench
Avg.
P@k
Tok.
P@k
Tok.
P@k
Tok.
P@k
Tok.
P@k
Tok.
P@k
Tok.
P@k
Tok.
DeepSeek-R1-Distill-Qwen-1.5B backbone
SFT+Alpha-0.1-1.5B
43.33
5925
63.33
5964
90.00
3405
47.79
2491
92.20
1987
56.15
4172
65.47
3991
SFT+Thinkless-1.5B
36.67
5948
43.33
6073
87.50
3281
45.22
2541
89.60
1711
61.19
3825
60.59
3897
SmartThinker-1.5B
43.33
5653
63.33
5310
95.00
3105
45.22
2495
91.00
1643
56.59
3700
65.75
3651
DeepScaleR-1.5B-Preview backbone
Appendix
Table 8: Per-benchmark Pass@k (%) and average response length of the controlled comparisons described in Section 5.2 . Each cell pair reports Pass@k and average token count. † LAPO-I is the publicly released checkpoint trained without our SFT warm-up and is shown only as a frontier reference.
Model
Run 1
Run 2
Run 3
Avg.
DeepSeek-R1-1.5B
0.62
0.58
0.59
0.60
DeepSeek-R1-7B
4.08
4.06
4.05
4.06
Appendix
Table 9: Wall-clock time (seconds) of one online step-contribution evaluation over 8 rollouts (up to 4,096 tokens each), measured over three runs.
Model
GRPO
SCPO
SCPO/GRPO
DeepSeek-R1-1.5B
9.35
9.81
104.92%
DeepSeek-R1-7B
34.92
37.66
107.80%
Appendix
Table 10: Average wall-clock time (seconds) per training step under identical hardware and configuration. SCPO adds less than 8% end-to-end overhead over plain GRPO on both model scales.
Models
B=4K
B=8K
B=16K
B=32K
DeepSeek-R1-7B
1.20
1.60
4.20
16.00
Alpha-0.1-7B
1.20
2.00
6.60
5.60
AdaptThink-7B
1.60
2.00
8.40
11.80
SmartThinker-7B
1.60
2.60
12.20
20.80
Appendix
Table 11: Average number of attempted AIME24 problems under each context budget. Attempts include correctly solved, incorrectly answered, and truncated problems before the run stops.
Figure 6: Effect of lightweight safeguard on reasoning behavior and Pass@k performance across different difficulty levels.
Figure 7: Entropy trends with problem difficulty during training. Because each sample batch updates the policy model for four iterations, both entropy and problem difficulty are averaged over every four steps.
Figure 8: Examples of model responses to questions with straightforward solutions across different baselines.
Figure 9: Examples of model responses to challenging questions across different baselines.