Large Reasoning Models (LRMs) produce explicit chains of thought to improve transparency, but they frequently overthink, generating long yet inefficient reasoning chains that inflate computational costs and risk introducing errors. While numerous reinforcement learning methods struggle to penalize verbosity to achieve conciseness, they typically apply the penalty uniformly across lengthy sequences, which leads to credit confounding, that is, the failure to distinguish essential reasoning steps from superfluous ones. In this paper, we study a measurable proxy for this confounding: high-entropy tokens that often coincide with surface transition points where reasoning re-enters verification. Based on this insight, we propose Adaptive Entropy-Sparing (AES), which selectively spares high-entropy tokens that occur before an efficient in-group reference, while penalizing overlength high-entropy suffix tokens associated with superfluous branching. Our AES thereby navigates the suppression-exploration trade-off, reducing overthinking while preserving average reasoning accuracy. We also present an entropy-length model that motivates why local uncertainty can serve as a proxy for future reasoning cost. Across eight reasoning benchmarks, AES reduces reasoning length by up to 52.9% while increasing mean accuracy by up to 3.8% over the base models, consistently outperforming existing efficiency-oriented methods. Our code is available at https://github.com/MobiusDai/AES.
Figures & tables
Figure 1: Top-20 most frequent tokens in high-entropy reasoning segments.
Figure 2: Impact of reasoning trajectory truncation on model performance. The plot compares the percentage of token savings against the accuracy drop relative to the baseline.
Figure 3: Overview of Adaptive Entropy-Sparing (AES). AES navigates the suppression-exploration trade-off by selectively penalizing overlength high-entropy suffix tokens associated with superfluous branching, while preserving the original learning signal for concise correct trajectories.
Model
Method
GSM8K
MATH-500
AMC
OlympiadBench
Mean
Acc
Len
Acc
Len
Acc
Len
Acc
Len
Acc
Len
R1-1.5B
Base Model
82.1
1813
83.5
5736
49.3
6820
44.8
11497
64.9
6467
Efficient Reason
78.1
324
81.8
2522
46.6
3345
45.2
6302
62.9
3123
LC-R1
81.2
745
80.2
2505
50.3
3432
43.0
5836
63.7
3130
Laser
87.0
1778
84.8
3321
51.4
4259
49.4
5819
68.2
3794
ThinkPrune
85.7
1131
83.7
2791
51.1
3527
46.3
5520
66.7
3242
Table 1: Main results on mathematical reasoning benchmarks. We report accuracy (%) and average response length (Len). In the Mean columns, bold and underlined values indicate the best and second-best aggregate values among rows shown, respectively.
Model
Method
AIME24
AIME25
LSAT
GPQA
Mean
Acc
Len
Acc
Len
Acc
Len
Acc
Len
Acc
Len
R1-1.5B
Base Model
27.9
16645
23.1
16623
22.8
14549
34.1
10090
27.0
14477
Efficient Reason
29.2
10517
22.1
9755
23.2
7694
33.0
6037
26.9
8501
LC-R1
24.6
8684
22.1
8526
24.4
11711
35.5
8575
26.7
9374
Laser
34.4
8492
22.5
8368
24.1
8059
35.5
5842
29.1
7690
ThinkPrune
30.4
7690
22.5
7750
25.5
6969
34.6
5785
28.3
7049
Table 2: Results on competition-level mathematics and out-of-domain benchmarks. We report accuracy (%) and average response length (Len). In the Mean columns, bold and underlined values indicate the best and second-best aggregate values among rows shown, respectively.
Figure 4: Performance Comparison under Varying Token Budgets. We compare the pass rates of AES and the baseline model under budget forcing, with maximum output lengths ranging from 512 to 4,096 tokens.
Variant
Accuracy
Tokens
Full AES
91.7
2103
w/o Scaling (Trajectory-Level)
91.1
2271
w/o Scaling (Token-Level)
90.0
2546
Random Selection
88.5
1734
Table 3: We evaluate the contribution of each design choice in AES on MATH-500. “Random Selection” penalizes randomly selected tokens with the same budget as the high-entropy set. “w/o Scaling (Trajectory-Level)” applies uniform penalty weights across all overthinking trajectories. “w/o Scaling (Token-Level)” uses identical penalty weights for all high-entropy tokens within each trajectory.
Setting
Hard/OOD Acc
Hard/OOD Len
Math Acc
Math Len
βAES=1.0
47.2
6544
75.2
3016
βAES=1.5
46.9
6045
75.1
2667
βAES=2.0
47.1
5940
75.2
2510
βAES=2.5
46.8
5432
74.7
2273
q=0.60
46.1
5537
74.3
2061
q=0.75
47.0
5770
74.9
2149
Table 4: Sensitivity to AES hyperparameters on R1-7B. We report mean accuracy (%) and average response length on the mathematical benchmark group and the hard/OOD benchmark group.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Dataset
Mean entropy diff.
90th-pct. entropy diff.
AIME25
+0.066
+0.160
AIME24
+0.031
+0.092
AMC
+0.056
+0.133
Appendix
Table 5: Entropy comparison between longer correct trajectories and the shortest correct trajectory in the same activated sampling group (R1-7B, G=16 ). Each entry is the mean paired difference (longer − shortest) of the per-trajectory mean token entropy or 90th-percentile token entropy.