Efficient Reasoning with Balanced Thinking

Yulin Li; Tengyao Tu; Li Ding; Junjie Wang; Huiling Zhen; Yixin Chen; Yong Li; Zhuotao Tian

cs.AIMar 12, 2026

Efficient Reasoning with Balanced Thinking

Authors: Yulin Li, Tengyao Tu, Li Ding, Junjie Wang, Huiling Zhen, Yixin Chen, Yong Li, Zhuotao Tian

Organizations: Harbin Institute of Technology (Shenzhen) · Zhongguancun Academy · Huawei Noah’s Ark Lab · The Chinese University of Hong Kong · Tsinghua University · Shenzhen Loop Area Institute

Abstract

Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they often suffer from overthinking, expending redundant computational steps on simple problems, or underthinking, failing to explore sufficient reasoning paths despite inherent capabilities. These issues lead to inefficiencies and potential inaccuracies, limiting practical deployment in resource-constrained settings. Existing methods to mitigate overthinking, such as suppressing reflective keywords or adjusting reasoning length, may inadvertently induce underthinking, compromising accuracy. Therefore, we propose ReBalance, a training-free framework that achieves efficient reasoning with balanced thinking. ReBalance leverages confidence as a continuous indicator of reasoning dynamics, identifying overthinking through high confidence variance and underthinking via consistent overconfidence. By aggregating hidden states from a small-scale dataset into reasoning mode prototypes, we compute a steering vector to guide LRMs' reasoning trajectories. A dynamic control function modulates this vector's strength and direction based on real-time confidence, pruning redundancy during overthinking, and promoting exploration during underthinking. Extensive experiments conducted on four models ranging from 0.5B to 32B, and across nine benchmarks in math reasoning, general question answering, and coding tasks demonstrate that ReBalance effectively reduces output redundancy while improving accuracy, offering a general, training-free, and plug-and-play strategy for efficient and robust LRM deployment. Project page and code are available at https://rebalance-ai.github.io .

Figures & tables

Panel of Figure 1 — Figure 1: Qualitative and quantitative comparisons with previous state-of-the-art methods for mitigating overthinking. (a) Given the question “For what real values of $x$ is $-4<x^{4}+4x^{2}<21$ ?”, the model first obtains intervals $(-\sqrt{3},0)$ and $(0,\sqrt{3})$ , and then verifies if $x=0$ is included. However, the baseline ( Guo et al., 2025 ) redundantly checks irrelevant values after correctly validating $x=0$ , causing overthinking. Current mitigation methods ( Yang et al., 2025b ) overly suppress necessary reflection, leading to underthinking. Our method dynamically controls the reasoning state, effectively balancing these two extremes. (b) ReBalance outperforms previous state-of-the-art method ( Chen et al., 2025b ) across multiple mathematical reasoning datasets and model scales (0.5B–32B), reducing reasoning length while simultaneously improving accuracy.

Figure 2: (a) Effects of overthinking mitigation on reasoning modes. We compare the distributions of reasoning lengths for correct and incorrect predictions before and after applying overthinking mitigation methods. The reduction in reasoning lengths for correct and incorrect predictions indicates the degree to which overthinking is alleviated and underthinking is introduced, respectively. Existing methods significantly introduce underthinking, whereas our method effectively achieves a balanced reduction of both. (b) Correlation between confidence and reasoning modes. We observe that the overthinking samples exhibit higher confidence variance compared to normal samples, while underthinking samples show persistently high confidence levels.

Figure 3: Illustration of the ReBalance framework. We first perform offline one-pass data collection on a small-scale seen dataset. At each step, the steering vector is extracted at the first token of the specified layer based on confidence, and a dynamic function is fitted according to model behaviors. During deployment, the dynamic function outputs steering weights based on the model’s real-time confidence online, thus balancing between overthinking and underthinking.

	MATH-500		AIME24		AIME25		GSM8K		AMC23		Olympiad
Method	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$
DeepSeek-R1-Distill-Qwen-1.5B
Baseline ( Guo et al., 2025 )	79.6	4516	23.3	12451	20.0	12739	76.0	1018	62.5	8239	41.2	8785
CoD ( Xu et al., 2025a )	80.2	3512	33.3	11894	13.3	10941	69.5	531	62.5	6812	35.2	7160
DEER ( Yang et al., 2025b )	67.0	2251	20.0	8135	23.3	8719	69.2	684	57.5	5132	35.4	5982
NoThinking ( Ma et al., 2025b )	75.0	1582	6.7	6998	16.6	7473	61.6	285	60.0	3267	37.3	3485

Table 1: Performance on math reasoning benchmarks. Metrics include Pass@1 (↑) and #Tokens (↓) on six math reasoning benchmarks. Changes are shown in orange for Pass@1 and blue for #Tokens. FlashThink and TrimR are reproduced according to the paper.

	SCIENCE		COMMONSENSE		PROGRAMMING
	GPQA-D		StrategyQA		LiveCodeBench
Method	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$
DeepSeek-R1-Distill-Qwen-1.5B
Baseline	$17.1$	$8\,727$	$63.2$	$435$	$19.5$	$12\,509$
Ours	$21.7$ ( $+4.6$ )	$6\,902$ ( $-20.9$ %)	$67.7$ ( $+4.5$ )	$401$ ( $-7.8$ %)	$22.5$ ( $+3.0$ )	$11\,622$ ( $-7.1$ %)
DeepSeek-R1-Distill-Qwen-7B

Table 2: Generalization capabilities on other non-math tasks. Metrics include Pass@1 (↑) and #Tokens (↓). Changes are shown in orange for Pass@1 and blue for #Tokens.

Method	Math500		GSM8K		Olympiad
Method	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$
ReBalance (Ours)	83.0	3474	78.3	765	43.9	7235
$\|\mathcal{W}_{s}\|=5$	82.4	3686	78.2	910	42.2	7343
$\|\mathcal{W}_{s}\|=10$	81.0	4084	77.8	940	44.7	7679
$B_{u}=0$	82.0	3343	78.1	761	39.4	7147
$B_{u}=0.2$	83.6	3600	80.5	850	44.2	7923

Table 3: Ablations on the R1-1.5B backbone across difficulty levels. We analyze performance changes on three math benchmarks with varying difficulty: Math500 (medium), GSM8K (easy), and Olympiad (hard). Metrics are Pass@1 accuracy (%) and generated token numbers. Arrows indicate change relative to our original ReBalance : Acc.

\uparrow

increase,

\downarrow

decrease; Tokens

\downarrow

decrease,

\uparrow

increase.

Figure 4: (a-b) Layerwise Performance of MATH-500 for (a) R1–7B and (b) QwQ–32B. (c) Sensitivity to sample size for steering vector extraction. (d) Performance with cross-domain vectors.

Gating Functions	MATH-500		GSM8K		Olympiad		GPQA-D
Gating Functions	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$
DeepSeek-R1-Distill-Qwen-1.5B
Baseline	79.6	4516	76.0	1018	41.2	8785	17.1	8727
Sigmoid Gate	83.0	3474	78.3	765	43.9	7235	21.7	6902
Linear Gate	83.6	3600	79.2	800	42.5	6794	17.7	7019
Hard-Step Gate	81.2	3830	79.7	820	41.8	7528	17.2	6537

Table 5: Performance comparison of different gating functions across benchmarks. We evaluate both DeepSeek-R1-Distill-Qwen-1.5B and DeepSeek-R1-Distill-Qwen-7B on four reasoning datasets. Metrics are Pass@1 accuracy (%) and the number of generated tokens.

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Mitigating Overthinking Without Inducing Underthinking. Evaluation on the MATH dataset with DeepSeek-R1-Distill-Qwen-1.5B. Each panel plots output length vs. reference length (words), restricted to reference

\leq 300

and output

\leq 8000

. Top: correct examples. Bottom: incorrect examples. Competing methods reduce overthinking at the cost of underthinking, whereas Our Method mitigates overthinking without inflating underthinking.

Model	Words vs Min confidence			Words vs Confidence variance
Model	$\bm{\rho}$	CI_lo	CI_hi	$\bm{\rho}$	CI_lo	CI_hi
DeepSeek-R1-Distill-Qwen-1.5B	$-0.733$	$-0.777$	$-0.684$	$0.602$	$0.542$	$0.655$
DeepSeek-R1-Distill-Qwen-7B	$-0.624$	$-0.681$	$-0.560$	$0.458$	$0.385$	$0.527$
QwQ-32B	$-0.801$	$-0.835$	$-0.765$	$0.698$	$0.643$	$0.746$
Qwen3-14B	$-0.681$	$-0.730$	$-0.628$	$0.623$	$0.561$	$0.682$

Appendix

Table 6: Spearman correlations (

\rho

) between word count length and confidence statistics on MATH-500 . The 95% bootstrap percentile confidence intervals (CI_lo and CI_hi, with

B=2000

) are reported in separate columns. All correlation coefficients are significant at

p<0.001

Model	$P(H{\to}H)$	$P(L{\to}L)$	SameRate	OR	CI_lo	CI_hi
DeepSeek-R1-Distill-Qwen-1.5B	$0.666$	$0.665$	$0.666$	$3.96$	$3.83$	$4.10$
DeepSeek-R1-Distill-Qwen-7B	$0.653$	$0.651$	$0.652$	$3.51$	$3.38$	$3.65$
QwQ-32B	$0.670$	$0.673$	$0.672$	$4.19$	$4.03$	$4.35$
Qwen3-14B	$0.657$	$0.659$	$0.658$	$3.70$	$3.56$	$3.86$

Appendix

Table 7: Adjacent state persistence in step-level confidence using a median threshold for binarization. Rows report the same state transition probabilities and diagonal odds ratios (OR) with ninety-five percent Woolf confidence intervals. Significance codes (Sig.): *

p<0.05

, **

p<0.01

, ***

p<0.001

(two sided Fisher exact test).

Figure 8: t - SNE projections of hidden states sampled immediately after \n\n ; colors encode sentence-level confidence (0–1). (a) DeepSeek - R1 - Distill - Qwen - 1.5B — layers 5, 20, 28. (b) DeepSeek - R1 - Distill - Qwen - 7B — layers 5, 20, 28. (c) Qwen3 - 14B — layers 5, 20, 35. (d) QwQ - 32B — layers 5, 20, 58.

Figure 9: Layer-wise linear probe decodability ( $R^{2}$ ) of confidence. Mid-to-late layers achieve the highest scores, motivating the steering-layer choice.

Model	Original Dim	PCA Dim	Retained Var.
DeepSeek-R1-Distill-Qwen-1.5B	$1\,536$	$64$	$0.889\,3$
DeepSeek-R1-Distill-Qwen-7B	$3\,584$	$64$	$0.857\,3$
QwQ-32B	$5\,120$	$64$	$0.833\,5$
Qwen3-14B	$5\,120$	$64$	$0.900\,0$

Appendix

Table 8: Cumulative explained variance retained by PCA (

k=64

) across models. Higher retained variance indicates a stronger low-dimensional linear structure amenable to probing.

Figure 10: Linear-confidence probe with PCA. Given an input sequence, we freeze the language model (LM) and extract layer-wise hidden states. The representations are projected by PCA and passed to a linear predictor to estimate step-level confidence. We repeat this procedure across layers to obtain layer-wise test-set $R^{2}$ scores.

Category	Vocabulary
NoWait (suppress)	wait , alternatively , hmm , but , however , alternative , another , check , double - check , oh , maybe , verify , other , again , now , ah , any
SEAL—Transition	alternatively , think differently , another way , another approach , another method , another solution , another strategy , another technique
SEAL—Reflection	wait , verify , make sure , hold on , think again , ’s correct , ’s incorrect , let me check , seems right

Appendix

Table 9: Unified vocabularies for NoWait (keyword suppression) and SEAL transition/reflection cues.

Figure 12: Left: heatmap of $g(c_{s},v_{s})$ with $v_{s}$ on a log scale; Right: 3D surface of $g(c_{s},v_{s})$ . Dashed lines mark $q_{25}^{c},q_{75}^{c}$ and $q_{25}^{v},q_{75}^{v}$ .

Method	Math500		GSM8K		Olympiad
Method	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$
Full [ $g(c,v)$ ]	83.0	3474	78.3	765	43.9	7235
$g(c)$	82.6 $\downarrow$	3387 $\downarrow$	78.1 $\downarrow$	890 $\uparrow$	41.7 $\downarrow$	6612 $\downarrow$
$g(v)$	76.6 $\downarrow$	3596 $\uparrow$	78.0 $\downarrow$	658 $\downarrow$	39.6 $\downarrow$	6876 $\downarrow$

Appendix

Table 11: Axis-wise ablations on the R1-1.5B backbone across difficulty levels. We analyze performance changes on three math benchmarks with varying difficulty: Math500 (medium), GSM8K (easy), and Olympiad (hard). Metrics are Pass@1 accuracy (%) and generated token numbers. Arrows indicate change relative to Full

[\;g(c,v)\;]

: Acc.

\uparrow

increase,

\downarrow

decrease; Tokens

\downarrow

decrease,

\uparrow

increase.

Dataset	Confidence Distribution				MATH-500
Dataset	q25c	q75c	q25v	q75v	Pass@1 $\uparrow$	#Tokens $\downarrow$
QwQ-32B
MATH	70.6	92.0	0.4	7.3	95.2	3662
Level-1	74.8	92.8	0.3	5.2	94.8	3447
Level-2	72.4	92.7	0.3	6.9	94.8	3623
Level-3	71.0	92.1	0.4	6.9	95.2	3678

Appendix

Table 12: Performance Variation under Difficulty-Conditioned Control Surfaces. We investigate performance shifts induced by control surfaces derived from datasets of varying difficulty. For each difficulty tier, we extract the steering vectors and fit the associated control surface, and subsequently evaluate them on DeepSeek-R1-Distill-Qwen-1.5B and QwQ-32B. The results indicate systematic differences in Rebalance behavior, suggesting that dataset difficulty plays a non-trivial role in shaping the resulting control dynamics.

Method	AMC23		AIME2024		AIME2025
Method	Pass@20 $\uparrow$	#Tokens $\downarrow$	Pass@20 $\uparrow$	#Tokens $\downarrow$	Pass@20 $\uparrow$	#Tokens $\downarrow$
DeepSeek-R1-Distill-Qwen-1.5B
Baseline	97.5	7430	63.3	10645	43.3	10447
ReBalance	97.5	4832	63.3	9243	46.7	8652
DeepSeek-R1-Distill-Qwen-7B
Baseline	100.0	6021	76.7	11249	66.7	11379

Appendix

Table 13: Pass@20 Performance and Token Efficiency. ReBalance preserves accuracy while consistently reducing the token usage of baseline models.

Model	Avg@1	Tok@1	Avg@4 $\pm$ Std	Tok@4 $\pm$ Std
DeepSeek-R1-Distill-Qwen-1.5B	79.6	4516	79.6 $\pm$ 0.004	4620 $\pm$ 100.9
w/ ReBalance	83.0	3474	83.3 $\pm$ 0.007	3553 $\pm$ 55.5
DeepSeek-R1-Distill-Qwen-7B	89.8	3699	89.4 $\pm$ 0.007	3703 $\pm$ 56.4
w/ ReBalance	92.6	2903	92.6 $\pm$ 0.001	2894 $\pm$ 38.1
Qwen3-14B	93.8	4470	93.8 $\pm$ 0.001	4550 $\pm$ 38.8
w/ ReBalance	94.0	3641	94.1 $\pm$ 0.001	3674 $\pm$ 44.6

Appendix

Table 14: Multi-sample evaluation on MATH-500.

AIME24
Model	Avg@1	Tok@1	Avg@4 $\pm$ Std	Tok@4 $\pm$ Std	Avg@16 $\pm$ Std	Tok@16 $\pm$ Std
DeepSeek-R1-Distill-Qwen-1.5B
Baseline	23.3	12596	21.7 $\pm$ 0.02	12897 $\pm$ 830.7	19.6 $\pm$ 0.03	12931 $\pm$ 830.7
ReBalance	36.7	9040	34.2 $\pm$ 0.04	9179 $\pm$ 718.4	35.6 $\pm$ 0.05	9179 $\pm$ 602.6
DeepSeek-R1-Distill-Qwen-7B
Baseline	40.0	13994	41.7 $\pm$ 0.03	13636 $\pm$ 434.2	41.3 $\pm$ 0.04	13764 $\pm$ 418.2

Appendix

Table 15: Multi-sample evaluation on AMC23, AIME24, and AIME25.

Temperature	Confidence Distribution				Baseline		Rebalance
Temperature	$q_{25}^{c}$	$q_{75}^{c}$	$q_{25}^{v}$	$q_{75}^{v}$	Pass@1 (%)	Token	Pass@1 (%)	Token
0.2	1	1	0	0	68.4	6248	77.0	4335
0.4	0.9602	1	0	0.0006	72.6	5694	79.6	3906
0.6	0.8507	1	0	0.0039	79.4	4584	81.6	3689
0.8	0.7156	0.9845	0.0002	0.0098	81.0	4529	84.6	3485
1.0	0.5516	0.9449	0.0006	0.0164	82.4	4660	82.4	3488

Appendix

Table 16: Performance of Rebalance under different temperature settings on DeepSeek-R1-Distill-Qwen-1.5B. Rebalance consistently provides higher accuracy and shorter reasoning traces from low- to high-temperature settings, demonstrating stable generalization under varying confidence distributions.

Figure 13: (a)–(d) show the mean confidence distributions of the Baseline and Rebalance models at temperatures 0.2, 0.4, 0.8, and 1.0, respectively. The trend reveals a clear temperature-dependent effect: at lower temperatures, Rebalance systematically reduces the model’s overconfident predictions, whereas at higher temperatures, it shifts the distribution upward, counteracting the underconfidence introduced by increased sampling randomness.

DeepSeek-R1-Distill-Qwen-1.5B
Method	Reflection			Transition			Performance Change
Method	Word Count	TF	TF–IDF	Word Count	TF	TF–IDF	$\Delta$ Pass@1	$\Delta$ Tokens
Baseline	30.3	9.7	12.1	6.7	2.4	3.9	–	–
NoThinking	3.9	1.7	4.6	0.9	0.4	1.3	-4.6	-64.9%
NoWait	1.6	1.7	3.5	0.1	$1.9\times 10^{-2}$	$7.7\times 10^{-2}$	-1.6	-41.4%
Rebalance	10.5	6.4	9.3	1.7	1.0	2.3	+3.4	-23.1%

Appendix

Table 17: Semantic statistics of Transition and Reflection vocabularies across different methods and models, and corresponding performance changes. For each method on each model, we report the total word count, term frequency (TF), and TF–IDF score for both vocabularies, as well as the change in accuracy and tokens relative to the Baseline of the same model.

DeepSeek-R1-Distill-Qwen-1.5B
Method	Score		Ability
Method	Rubric	Elo	Coh	Crt	Img	Pac	Ele	Inst	Voi	Dia	Flo	SDT	Ama	Emo	Pos	Pur	Ch
Baseline	15.0	30.0	1.3	1.8	2.8	1.9	5.4	1.5	2.9	2.9	4.1	2.5	3.2	1.2	11.5	9.2	1.8
Rebalance	15.2	57.5	1.5	1.9	2.7	2.2	5.6	1.4	2.8	3.0	4.3	2.5	3.3	1.2	11.2	9.4	1.7
DeepSeek-R1-Distill-Qwen-7B
Baseline	22.9	308.9	4.0	4.4	4.8	4.8	6.6	3.1	5.0	4.2	6.6	4.0	5.4	2.7	9.9	9.6	3.4

Appendix

Table 18: Creative-writing performance on Creative Writing v3 benchmark. Ability abbreviations: Coh = Coherent; Crt = Creativity; Img = Descriptive Imagery; Pac = Pacing; Ele = Elegant Prose; Inst = Instruction Following; Voi = Consistent Voice & Tone; Dia = Strong Dialogue; Flo = Sentence Flow; SDT = Show-Don’t-Tell; Ama = Avoids Amateurish Prose; Emo = Emotional Depth; Pos = Avoids Positivity Bias; Pur = Avoids Purple Prose; Ch = Believable Characters.

Model	Normal (O-base)			Overthinking			Normal (U-base)			Underthinking
Model	Conf.	Var.	Len.	Conf.	Var.	Len.	Conf.	Var.	Len.	Conf.	Var.	Len.
DeepSeek-R1-Distill-Qwen-1.5B	80.4	18.5	1357	78.6	21.3	2386	85.1	22.2	2909	89.7	16.7	1726
DeepSeek-R1-Distill-Qwen-7B	90.0	11.0	2809	81.8	23.0	5995	82.4	22.0	3259	91.2	12.0	2752
Qwen3-14B	92.6	8.7	5763	88.1	12.1	8819	85.3	11.3	6305	9.27	8.0	5743
QwQ-32B	84.0	16.1	3712	75.4	22.1	6377	76.2	18.5	4573	78.9	1.76	3080

Appendix

Table 19: Comparison of Confidence, Variance, and Output Length Across Thinking Modes. For each model, we report statistics for Normal responses paired with Overthinking (Normal (O-base)) and Underthinking (Normal (U-base)), together with the corresponding Overthinking and Underthinking states.

QwQ-32B	MATH-500		AIME24		AIME25		GSM8K		AMC23		GPQA-D
QwQ-32B	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$	Pass@1 $\uparrow$	#Tokens $\downarrow$
TrimR
threshold=0.5	93.8	3830	56.7	8345	43.3	8827	93.7	1319	90.0	6055	63.1	6380
threshold=0.75	94.8	4048	70.0	10235	53.3	11397	96.7	1410	87.5	6778	69.2	7312
threshold=1	93.8	4241	66.7	11007	56.7	11726	95.9	1432	90.0	6890	63.7	7012
Flashthink

Appendix

Table 20: Performance comparison with TrimR and Flashthink on QwQ-32B. All experiments are run with the same sampling parameters.

DeepSeek-R1-Distill-Qwen-1.5B
Method	AMC23		AIME24		AIME25		Olympiad
Method	Pass@1	#Tokens	Pass@1	#Tokens	Pass@1	#Tokens	Pass@1	#Tokens
Baseline	55	8990	23.3	12596	16.7	14556	41.2	8785
Dynamic Temperature	75	7344	36.7	12054	23.3	11659	44.7	8538
Dynamic Steering	80	5216	36.7	9040	30.0	8140	43.9	7253
DeepSeek-R1-Distill-Qwen-7B

Appendix

Table 21: Performance comparison across three model sizes under different inference-time control methods.

Interval	1	2	3	4	5	6	7	8	9	10
Relative Position Range	[0.0, 0.1)	[0.1, 0.2)	[0.2, 0.3)	[0.3, 0.4)	[0.4, 0.5)	[0.5, 0.6)	[0.6, 0.7)	[0.7, 0.8)	[0.8, 0.9)	[0.9, 1.0]
Count of Stability Indices	27.41	16.45	10.96	9.87	6.58	3.29	2.85	2.19	3.95	16.45

Appendix

Table 22: Distribution of stability indices over relative position intervals.

Methods	$\lVert S\rVert_{2}$	MATH-500 (Pass@1)	MATH-500 (#Tokens)	AIME24 (Pass@1)	AIME24 (#Tokens)
Baseline	–	79.6	4516	23.3	12596
Vector Extraction w/ Stability Index	11.4	81.8	4715	20.0	12563
Vector Extraction w/ Confidence	62.4	83.0	3474	36.7	9040

Appendix

Table 23: Comparison of steering vector extraction variants.

Method	Prompt Modification
CoT ( Wei et al., 2022 )	Please reason step by step.
CoD ( Xu et al., 2025a )	Think step by step, but only keep a minimum draft for each thinking step, with 5 words at most.
CCoT ( Renze and Guven, 2024 )	Think step by step, Be concise.
CCoT-2-45 ( Nayab et al., 2024 )	Let’s think a bit step by step and limit the answer length to 45 words.
BTC ( Ding et al., 2024 )	Rapidly evaluate and use the most effective reasoning shortcut to answer the question.
NoThinking ( Ma et al., 2025a )	<think> Okay, I have finished thinking.</think>

Appendix

Table 24: Prompt modifications used in different reasoning strategies.

Method	TPS	TPR (s)	Additional GPU Memory (GB)	#Tokens
DeepSeek-R1-Distill-Qwen-7B
Baseline	80.2	174.6	–	13994
NoThinking	80.1	55.2	0.0	4427
CoD	80.2	145.5	0.0	11663
FlashThink	73.8	135.9	18.3	10034
TrimR	48.8	147.7	15.4	7213

Appendix

Table 25: Efficiency analysis on DeepSeek-R1-Distill-Qwen-7B and QwQ-32B.

Figure 14: A DeepSeek-R1-Distill-Qwen-1.5B inference case shows that, although the model arrives at the correct solution, its reasoning is rife with redundancy and repetition. Beyond the necessary boundary checks, it inspects numerous inconsequential points, incurring substantial token overhead.

Figure 15: In a DeepSeek-R1-Distill-Qwen-1.5B inference case, applying existing overthinking-mitigation techniques reduces token usage relative to the baseline; however, the absence of verification steps results in an incorrect answer.

Figure 16: In a DeepSeek-R1-Distill-Qwen-1.5B inference example, the incorporation of judicious verification steps yields a correct and succinct response.

Explore similar work

Sep 28, 2026

Efficient Reasoning via Constrained Optimization in Latent Space

Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they still suffer from overthinking, generating redundant reasoning steps which incur substantial token consumption. Existing methods, such as suppressing reflective keywords or forcing shorter reasoning lengths, attempt to mitigate this issue but inevitably truncate necessary steps and induce underthinking, thereby compromising performance. To address this dilemma, we investigate the latent representations and observe that efficient reasoning steps naturally cluster into a concentrated region in latent space, while those deviating from this region tend to produce verbose sequences. To leverage this, we keep reasoning focused within this region via a quadratic program which projects deviating hidden states back into the region. Then we propose a novel training-free framework to achieve efficient reasoning that reduces token generation costs without sacrificing performance. Extensive experiments conducted on four models ranging from 1.5B to 14B, and across six benchmarks in math reasoning, coding, and scientific QA, validate the effectiveness of our method, up to a 12.1% improvement in accuracy while reducing generated tokens by 11.8% to 52.8%. Codes are available at https://github.com/hzn18/Opt4Reasoning.

Zhinan Hou, Xingchen Li, Keyou You

Tsinghua University

Overthinking in Language Models Efficient Language Model Reasoning Quadratic Programming Efficient Language Model Inference Latent Space Optimization Large Reasoning Models

Ask PDF

Jul 22, 2026

EvoThink: Evolving Thinking in Large Reasoning Models via Self-Pruning and Aha-Moment Preference Optimization

Large Reasoning Models (LRMs) often suffer from overthinking due to redundant verification steps. Existing approaches for mitigating overthinking, such as fast-slow thinking switching and reasoning trajectory compression, fail to make a fine-grained distinction between beneficial and redundant steps within the LRM's reasoning process, and may thus impair reasoning capability in their pursuit of efficiency. To simultaneously improve reasoning efficiency and capability, we propose EvoThink, a framework that reduces redundant verification and encourages the exploration of new reasoning paths. EvoThink comprises two key components: Self-Pruning Training (SPT), an unsupervised method that iteratively prunes redundant reasoning steps and self-trains on the concise trajectories; and Aha-Moment Preference Optimization (AMPO), which, inspired by genetic algorithms, identifies valuable failed reasoning attempts, synthesizes from-wrong-to-right aha-moment data, and optimizes the model to internalize this reasoning pattern. Extensive evaluations across mathematical reasoning and code generation benchmarks demonstrate that EvoThink not only substantially reduces inference-time token usage but also improves the reasoning capability of LRMs.

Xinbang Dai, Zheyu Xin, Huikang Hu +7

Southeast University · Noah’s Ark Lab

Overthinking in Language Models Self-Training for Language Models Efficient Language Model Reasoning Preference Optimization Efficient Language Model Inference Large Reasoning Models

Ask PDF

Feb 18, 2026

Training Large Reasoning Models Efficiently via Progressive Thought Encoding

Large reasoning models (LRMs) excel on complex problems but face a critical barrier to efficiency: reinforcement learning (RL) training requires long rollouts for outcome-based rewards, where autoregressive decoding dominates time and memory usage. While sliding-window cache strategies can bound memory, they disrupt long-context reasoning and degrade performance. We introduce Progressive Thought Encoding, a parameter-efficient fine-tuning method that enables LRMs to reason effectively under fixed-size caches. By progressively encoding intermediate reasoning into fixed-size vector representations, our approach eliminates the need to backpropagate through full-cache rollouts, thereby reducing memory usage, while maintaining constant memory during inference. Experiments on three models, including Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, and DeepSeek-R1-Distill-Llama-8B, on six widely used challenging mathematical benchmarks show consistent gains: our method achieves +19.3% improvement over LoRA-based fine-tuning and +29.9% over LRMs without fine-tuning on average, with up to +23.4 accuracy improvement on AIME2024/2025 under the same tight cache budgets. These results demonstrate that Progressive Thought Encoding not only improves reasoning accuracy but also makes RL training of LRMs substantially more efficient and scalable under real-world memory constraints.

Zeliang Zhang, Xiaodong Liu, Hao Cheng +3

University of Rochester · Microsoft Research

Efficient Language Model Reasoning LLM Reasoning Large Reasoning Models Parameter-Efficient Fine-Tuning RL for Language Models

Ask PDF