Large Reasoning Models (LRMs) produce explicit chains of thought to improve transparency, but they frequently overthink, generating long yet inefficient reasoning chains that inflate computational costs and risk introducing errors. While numerous reinforcement learning methods struggle to penalize verbosity to achieve conciseness, they typically apply the penalty uniformly across lengthy sequences, which leads to credit confounding, that is, the failure to distinguish essential reasoning steps from superfluous ones. In this paper, we study a measurable proxy for this confounding: high-entropy tokens that often coincide with surface transition points where reasoning re-enters verification. Based on this insight, we propose Adaptive Entropy-Sparing (AES), which selectively spares high-entropy tokens that occur before an efficient in-group reference, while penalizing overlength high-entropy suffix tokens associated with superfluous branching. Our AES thereby navigates the suppression-exploration trade-off, reducing overthinking while preserving average reasoning accuracy. We also present an entropy-length model that motivates why local uncertainty can serve as a proxy for future reasoning cost. Across eight reasoning benchmarks, AES reduces reasoning length by up to 52.9% while increasing mean accuracy by up to 3.8% over the base models, consistently outperforming existing efficiency-oriented methods. Our code is available at https://github.com/MobiusDai/AES.
Figures & tables
Figure 1: Top-20 most frequent tokens in high-entropy reasoning segments.
Figure 2: Impact of reasoning trajectory truncation on model performance. The plot compares the percentage of token savings against the accuracy drop relative to the baseline.
Figure 3: Overview of Adaptive Entropy-Sparing (AES). AES navigates the suppression-exploration trade-off by selectively penalizing overlength high-entropy suffix tokens associated with superfluous branching, while preserving the original learning signal for concise correct trajectories.
Model
Method
GSM8K
MATH-500
AMC
OlympiadBench
Mean
Acc
Len
Acc
Len
Acc
Len
Acc
Len
Acc
Len
R1-1.5B
Base Model
82.1
1813
83.5
5736
49.3
6820
44.8
11497
64.9
6467
Efficient Reason
78.1
324
81.8
2522
46.6
3345
45.2
6302
62.9
3123
LC-R1
81.2
745
80.2
2505
50.3
3432
43.0
5836
63.7
3130
Laser
87.0
1778
84.8
3321
51.4
4259
49.4
5819
68.2
3794
ThinkPrune
85.7
1131
83.7
2791
51.1
3527
46.3
5520
66.7
3242
Table 1: Main results on mathematical reasoning benchmarks. We report accuracy (%) and average response length (Len). In the Mean columns, bold and underlined values indicate the best and second-best aggregate values among rows shown, respectively.
Model
Method
AIME24
AIME25
LSAT
GPQA
Mean
Acc
Len
Acc
Len
Acc
Len
Acc
Len
Acc
Len
R1-1.5B
Base Model
27.9
16645
23.1
16623
22.8
14549
34.1
10090
27.0
14477
Efficient Reason
29.2
10517
22.1
9755
23.2
7694
33.0
6037
26.9
8501
LC-R1
24.6
8684
22.1
8526
24.4
11711
35.5
8575
26.7
9374
Laser
34.4
8492
22.5
8368
24.1
8059
35.5
5842
29.1
7690
ThinkPrune
30.4
7690
22.5
7750
25.5
6969
34.6
5785
28.3
7049
Table 2: Results on competition-level mathematics and out-of-domain benchmarks. We report accuracy (%) and average response length (Len). In the Mean columns, bold and underlined values indicate the best and second-best aggregate values among rows shown, respectively.
Figure 4: Performance Comparison under Varying Token Budgets. We compare the pass rates of AES and the baseline model under budget forcing, with maximum output lengths ranging from 512 to 4,096 tokens.
Variant
Accuracy
Tokens
Full AES
91.7
2103
w/o Scaling (Trajectory-Level)
91.1
2271
w/o Scaling (Token-Level)
90.0
2546
Random Selection
88.5
1734
Table 3: We evaluate the contribution of each design choice in AES on MATH-500. “Random Selection” penalizes randomly selected tokens with the same budget as the high-entropy set. “w/o Scaling (Trajectory-Level)” applies uniform penalty weights across all overthinking trajectories. “w/o Scaling (Token-Level)” uses identical penalty weights for all high-entropy tokens within each trajectory.
Setting
Hard/OOD Acc
Hard/OOD Len
Math Acc
Math Len
βAES=1.0
47.2
6544
75.2
3016
βAES=1.5
46.9
6045
75.1
2667
βAES=2.0
47.1
5940
75.2
2510
βAES=2.5
46.8
5432
74.7
2273
q=0.60
46.1
5537
74.3
2061
q=0.75
47.0
5770
74.9
2149
Table 4: Sensitivity to AES hyperparameters on R1-7B. We report mean accuracy (%) and average response length on the mathematical benchmark group and the hard/OOD benchmark group.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Dataset
Mean entropy diff.
90th-pct. entropy diff.
AIME25
+0.066
+0.160
AIME24
+0.031
+0.092
AMC
+0.056
+0.133
Appendix
Table 5: Entropy comparison between longer correct trajectories and the shortest correct trajectory in the same activated sampling group (R1-7B, G=16 ). Each entry is the mean paired difference (longer − shortest) of the per-trajectory mean token entropy or 90th-percentile token entropy.
Reasoning in Large Language Models incurs significant inference-time compute, yet the token-level information structure of reasoning traces remains underexplored. We observe that reasoning tokens split into two functional types: low-entropy structural tokens (recurring phrases that scaffold the reasoning process) and higher-entropy organic tokens (problem-specific content that drives toward a solution). This asymmetry motivates a simple, model-agnostic compression pipeline: apply cross-word BPE merges on a model's own reasoning traces to derive supertokens that capture frequent structural patterns, then teach the model to adopt them via supervised fine-tuning. Across three model families and five mathematical reasoning benchmarks, our approach shortens reasoning traces by 8.1% on average with no statistically significant accuracy loss on any model--benchmark pair. Beyond compression, supertokens act as interpretable reasoning-move annotations (backtracking, verification, strategy shifts), exposing the model's high-level strategy at a glance. Analyzing transitions between structural categories reveals systematic differences between correct and incorrect traces: correct traces show productive recovery (backtracking followed by strategy shifts and verification), while incorrect traces are dominated by confusion cycles (repeated hedging and unresolved contradictions). These diagnostic signals suggest applications in reward shaping and early stopping for RL-based reasoning training.
Large Reasoning Models (LRMs) have revolutionized complex problem-solving, yet they exhibit a pervasive "overthinking", generating unnecessarily long reasoning chains. While current solutions improve token efficiency, they often sacrifice fine-grained control or risk disrupting the logical integrity of the reasoning process. To address this, we introduce Stepwise Adaptive Thinking (SAT), a framework that performs step-level, difficulty-aware pruning while preserving the core reasoning structure. SAT formulates reasoning as a Finite-State Machine (FSM) with distinct thinking modes (Slow, Normal, Fast, Skip). It navigates these states dynamically using a lightweight Process Reward Model (PRM), compressing easy steps while preserving depth for hard ones. Experiments across 9 LRMs and 7 benchmarks show that SAT achieves up to 40% reduction in reasoning tokens while generally maintaining or improving accuracy.
Weiyang Huang, Xuefeng Bai, Kehai Chen +4
Harbin Institute of Technology (Shenzhen), Shenzhen, China · Huawei Technologies
Large reasoning models, such as OpenAI o1 and DeepSeek-R1, tend to become increasingly verbose as their reasoning capabilities improve. These inflated Chain-of-Thought (CoT) trajectories often exceed what the underlying problems require, wasting compute, latency, and context budgets. While introducing length-based efficiency rewards during reinforcement learning offers a natural remedy, existing methods struggle with two fundamental challenges: the optimal balance between correctness and efficiency is non-stationary throughout training, and intrinsic reasoning budgets vary drastically across problems. Relying on static reward weights and global length constraints inevitably forces a compromise between degraded accuracy and unrealized compression. To overcome these limitations, we propose LEAD (Length-Efficient Adaptive and Dynamic reasoning), a method that replaces static heuristics with online, self-adaptive mechanisms. LEAD dynamically calibrates the correctness--efficiency trade-off at each step using a Potential-Scaled Instability, directing optimization capacity to the most informative learning signal. Furthermore, it estimates an adaptive per-problem target length online based on the model's own correct rollouts, applying a symmetric efficiency reward that penalizes both overthinking and over-compression. Evaluated on five mathematical reasoning benchmarks, LEAD achieves the highest accuracy and Accuracy-Efficiency Score among RL-trained efficient-reasoning methods in our matched comparisons while producing substantially shorter outputs than the base model.
Songtao Wei, Yi Li, Zhikai Li +7
University of Texas at Dallas · Emory University · University of Texas at Arlington +2