On the Token Value Inequality in Efficient Reasoning

Runjia Zeng; Hang Hua; Yiyang Liu; Zhiqiang Tao; Ruixiang Tang; Qifan Wang; Cheng Han; Dongfang Liu

cs.CLSep 27, 2026

On the Token Value Inequality in Efficient Reasoning

Authors: Runjia Zeng, Hang Hua, Yiyang Liu, Zhiqiang Tao, Ruixiang Tang, Qifan Wang, Cheng Han, Dongfang Liu

Organizations: Rochester Institute of Technology · Purdue University · MIT-IBM Watson AI Lab · University of Missouri-Kansas City · Rutgers University · Meta AI

Abstract

Chain-of-Thought reasoning has enabled large language models to achieve substantial performance gains on complex tasks. However, these gains come at the cost of dramatically increased token consumption. This raises a fundamental question: is every token in the reasoning trace equally valuable? We present a diagnostic and optimization framework grounded in a key empirical finding: the value of tokens within a CoT reasoning sequence is highly non-uniform, and this non-uniformity can be effectively characterized by token-level log probability signals. We show that normalized log probability helps distinguish core tokens, which carry structural and decisive reasoning content, from redundant tokens, which are exploratory, low-confidence filler that contributes less directly to the final answer. Building on these findings, we formulate the TokenProbe framework around two empirical findings and one claim: findings identify token value inequality first and then establish TokenProbe as a core-token proxy, and the claim introduces an efficient GRPO objective positing that selectively compressing redundant tokens can yield Pareto improvements in the accuracy-token efficiency space. Empirically, our method preserves reasoning quality while reducing the token usage by 76% of the baseline. Under matched reasoning-length budgets, we show that it can even outperform strong flagship baselines like Gemini-3.1-Pro. Homepage: https://runjia.tech/tokenprobe/.

Figures & tables

Figure 1: Overview of TokenProbe: from token value inequality analysis § 2 to efficient reasoning § 3 . (a) Probing analysis shows that token contributions are highly unequal, and TokenProbe serves as an proxy for identifying core tokens. (b) Building on this insight, efficient GRPO improves it by preserving hidden core tokens, yielding better reasoning efficiency and Pareto performance.

Figure 2: TokenProbe as an indicator of reasoning efficiency . (a) TokenProbe increases from the think stage to the answer stage. (b) Across models, higher TokenProbe corresponds to better accuracy and lower token consumption. (c) Quartile analysis shows the same trend at the sample level.

Position	$\mathrm{NormLP}$	Semantic Role	Impact
Token [54] (early)	$+0.47$	First verification discovers misinterpretation	Productive
Token [103] (early)	$+0.22$	Mild self-correction	Productive
Token [575] (mid)	$-3.87$	Deep confusion with hesitation	Wasteful
Token [1273] (mid-late)	$-4.13$	Re-questions already-confirmed step	Wasteful (+700 tokens)

Table 1: Functional analysis of the “Wait” token at different positions within a single reasoning trace.

Figure 3: Comparison of training strategies for efficient reasoning . (a) Vanilla GRPO treats all tokens equally. (b) Selective KL Anchoring (KL) rescales the KL regularizer per token and imposes a sparsity penalty on over-budget redundant tokens. (c) Reward Shaping (RS) reshapes the token-level advantage, upweighting core tokens and penalising overlong redundant spans.

Model	Average		AIME		MATH		AMC		Olympid
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Closed-API Reasoning Models
GPT-5.4-Mini	987	69.6	1714	36.7	589	88.6	1137	73.5	1232	56.6
Claude-4.6-Sonnet	1221	61.3	1907	20.0	801	86.0	1413	51.8	1478	46.1
Grok-4.1-Fast	4945	52.3	9361	10.0	1866	74.8	4017	43.4	7144	38.5
Gemini-3.1-Pro	1586	54.4	2020	10.0	1274	79.2	1798	37.3	1771	40.1

Table 2: Main results on four in-domain reasoning benchmarks. #Tk. is the average number of generated tokens per response (lower is better). Acc. is averaged accuracy in % (higher is better).

Figure 4: TokenProbe signals during RL training .

Figure 5: Transfer and ablation of TokenProbe-based efficient reasoning . (a) Our variants match accuracy with far fewer tokens. (b) Different hypeparameter settings for KL and RS (more in § S3.2 ).

Appendix figures & tables59 assets

Supplementary material from the paper’s appendix.

Appendix

Model Name	Identifier	Link
Qwen-0.6B	Qwen/Qwen3-0.6B	HuggingFace
DeepScaleR-1.5B	agentica-org/DeepScaleR-1.5B-Preview	HuggingFace
Qwen-4B	Qwen/Qwen3-4B	HuggingFace
Phi-Reasoning-4B	microsoft/Phi-4-mini-reasoning	HuggingFace
Qwen-8B	Qwen/Qwen3-8B	HuggingFace

Appendix

Table S1: Training models used in Table 2 .

Model Name	HF Identifier	Parameters	Link
Dense Models
Qwen-4B-Instruct	Qwen/Qwen3-4B-Instruct-2507	4B	HuggingFace
Qwen-9B	Qwen/Qwen3.5-9B	9B	HuggingFace
DS-Qw-1B	deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B	1.5B	HuggingFace
Mixture-of-Experts (MoE) Models
GPT-117B-5B	openai/gpt-oss-120b	117B (5B)	HuggingFace

Appendix

Table S2: Open-source baseline models used in our experiments. Identifiers follow the HuggingFace org/model-name convention. For MoE models, parameters are listed as Total (Active).

Model Name	API Identifier	Version	Mode	Link
Qwen-3.6-Plus	qwen/qwen3.6-plus	3.6	Plus	Blog
GPT-5.4-Mini	openai/gpt-5.4-mini	5.4	Mini	Blog
Grok-4.1-Fast	x-ai/grok-4.1-fast	4.1	Fast	Blog
Gemini-3-Flash	google/gemini-3-flash-preview	3	Flash	Blog
Gemini-3.1-Pro	google/gemini-3.1-pro-preview	3.1	Pro	Blog
Gemini-3.1-Flash-Lite	google/gemini-3.1-flash-lite-preview	3.1	Flash-Lite	Blog

Appendix

Table S3: Closed-source baseline models used in our experiments. Version denotes the model version number. Links point to official announcement pages.

Mode	Accuracy	Avg. Tokens
CoT ON ( thinking=True )	77.1%	8,728
CoT OFF ( thinking=False )	53.0%	1,943
Difference	+24.1 percentage points	+6,785 (4.5 $\times$ )

Appendix

Table S4: Pure CoT contribution measured on Qwen-4B.

Model	Think TokenProbe	Answer TokenProbe	$\Delta$ (Answer $-$ Think)
DS-Qw-1B	0.716	0.915	$+0.199$
Qwen-0.6B	0.712	0.823	$+0.111$
Qwen-4B	0.775	0.815	$+0.040$
Qwen-8B	0.784	0.810	$+0.026$
Qwen-9B	0.779	0.878	$+0.099$

Appendix

Table S5: TokenProbe in the Think versus Answer stage. Across all five models, the Answer stage exhibits a systematically higher TokenProbe than the Think stage.

Model	TokenProbe	Avg Tokens	Avg Accuracy
Qwen-0.6B	72.7%	8,723	43.8%
DS-Qw-1B	74.2%	8,589	54.7%
Qwen-4B	77.4%	6,155	65.7%
Qwen-4B-Instruct	83.8%	3,573	71.8%

Appendix

Table S6: TokenProbe, average token usage, and average accuracy for the four representative models in Fig. 2 (b).

Model	AIME2025	AMC	MATH	Olympiad	Overall
Qwen-0.6B	13.3%	45.8%	75.6%	40.6%	43.8%
DS-Qw-1B	20.0%	66.3%	85.0%	47.7%	54.7%
Qwen-4B	40.0%	74.7%	90.4%	57.8%	65.7%
Qwen-4B-Instruct	50.0%	78.3%	92.6%	66.2%	71.8%

Appendix

Table S7: Per-dataset accuracy of the four models in Fig. 2 (b).

Model	AIME2025	AMC	MATH	Olympiad	Overall
Qwen-0.6B	12,569	8,934	5,064	8,323	8,723
DS-Qw-1B	12,256	8,146	4,785	9,169	8,589
Qwen-4B	7,696	6,360	4,162	6,404	6,155
Qwen-4B-Instruct	5,721	3,216	1,548	3,805	3,573

Appendix

Table S8: Per-dataset token usage of the four models in Fig. 2 (b).

Quartile	TokenProbe range	Avg TokenProbe	Accuracy	Avg Tokens
Q1	0.690 to 0.756	0.736	61.5%	10,555
Q2	0.756 to 0.781	0.770	78.3%	8,327
Q3	0.781 to 0.804	0.791	87.0%	6,592
Q4	0.804 to 0.869	0.820	92.9%	4,524

Appendix

Table S9: TokenProbe quartile analysis for Qwen-4B.

Quartile	TokenProbe range	Avg TokenProbe	Accuracy	Avg Tokens
Q1	0.682 to 0.766	0.744	60.9%	10,825
Q2	0.766 to 0.789	0.779	77.0%	8,637
Q3	0.789 to 0.811	0.800	86.6%	6,888
Q4	0.811 to 0.990	0.828	90.4%	5,017

Appendix

Table S10: TokenProbe quartile analysis for Qwen-8B.

Quartile	TokenProbe range	Accuracy	Avg Tokens
Q1	$\leq 0.688$	39.1%	9,623
Q2	0.688 to 0.745	67.7%	7,066
Q3	0.745 to 0.773	71.4%	6,933
Q4	$>0.773$	72.7%	6,271

Appendix

Table S11: TokenProbe quartile analysis for DS-Qw-1.5B.

Quartile	TokenProbe range	Accuracy	Avg Tokens
Q1	$\leq 0.698$	32.3%	8,048
Q2	0.698 to 0.731	45.7%	8,310
Q3	0.731 to 0.756	64.3%	6,873
Q4	$>0.756$	73.3%	5,554

Appendix

Table S12: TokenProbe quartile analysis for Qwen-0.6B.

Setting	TokenProbe	Avg Tokens	Accuracy
Qwen-9B (8K context, $t=0.7$ )	0.780	7,252	36.8%
Qwen-9B (16K context, $t=0.7$ )	0.782	10,009	46.5%
Qwen-9B (16K context, $t=1.0$ )	0.780	9,303	45.5%

Appendix

Table S13: Qwen-9B under three settings. TokenProbe is stable while accuracy and token usage shift considerably.

Model	AIME2025	AMC	MATH	Olympiad	Overall
Qwen-0.6B	0.716	0.729	0.739	0.726	0.727
DS-Qw-1B	0.732	0.748	0.748	0.740	0.742
Qwen-4B	0.764	0.775	0.786	0.765	0.774
Qwen-4B-Instruct	0.800	0.830	0.862	0.823	0.838

Appendix

Table S14: Per-dataset TokenProbe means for the four representative models.

Model	TokenProbe (Correct)	TokenProbe (Wrong)	$\Delta$	$N_{\text{correct}}$	$N_{\text{wrong}}$
Qwen-0.6B	0.742	0.714	$+0.028$	564	724
DS-Qw-1B	0.757	0.726	$+0.031$	705	583
Qwen-4B	0.782	0.754	$+0.028$	916	372
Qwen-4B-Instruct	0.851	0.793	$+0.059$	990	298

Appendix

Table S15: TokenProbe of correctly versus wrongly answered samples. Correct samples carry a uniformly higher TokenProbe.

Setting	Avg. Acc.	Avg. Tokens
Qwen-4B-Instruct (base, non-thinking)	$71.7$	$3{,}600$
Qwen-4B-Instruct + vanilla GRPO	$\phantom{0}1.3$	$8{,}136$
Qwen-4B-Instruct + KL ( $\beta=5{\times}10^{-3}$ , $w{=}20$ )	$52.1$	$2{,}346$
Qwen-4B-Instruct + KL ( $\beta=5{\times}10^{-4}$ , $w{=}20$ )	$41.7$	$\phantom{0,}790$

Appendix

Table S16: Compression objectives applied to a non-thinking backbone. Qwen3-4B-Instruct is at TokenProbe saturation, so there is no exploratory headroom for the compression objective to suppress. Vanilla GRPO collapses the model entirely. Our KL variant avoids the runaway expansion but cannot recover the base accuracy. Numbers are averages over the four math benchmarks of Tab. 2 .

Model	$r$ ( TokenProbe , Token Length )	$r$ ( TokenProbe , Accuracy )
DS-Qw-1.5B	$-0.227$	$+0.288$
Qwen-0.6B	$-0.379$	$+0.367$
Qwen-4B	$-0.415$	$+0.307$
Qwen-8B	$-0.367$	$+0.254$
Qwen-9B	$-0.370$	$+0.197$

Appendix

Table S17: Pearson correlations between TokenProbe and (i) token length and (ii) accuracy. The first column is uniformly negative. The second is uniformly positive.

Descriptor	Per-response definition
Log-probability distribution (absolute level)
Mean log-prob	$\mu=\tfrac{1}{T}\sum_{t=1}^{T}\ell_{t}$
Std / Var log-prob	$\sigma$ , $\sigma^{2}$
Median / Min / Max log-prob	$\operatorname{median}(\{\ell_{t}\}),\;\min_{t}\ell_{t},\;\max_{t}\ell_{t}$
Quartiles, IQR	$q_{0.25},\;q_{0.75},\;q_{0.75}-q_{0.25}$
Skewness log-prob	$\tfrac{1}{T}\sum_{t}(\ell_{t}-\mu)^{3}/\sigma^{3}$

Appendix

Table S18: Every per-response descriptor evaluated in the ablation. Per-dataset / overall numbers are obtained by averaging the per-response value across responses.

Signal (overall)	Qwen-0.6B	DS-Qw-1B	Qwen-4B	Qwen-4B-Instruct
Accuracy	43.8%	54.7%	65.7%	71.8%
Avg. Tokens	8,723	8,589	6,155	3,573
TokenProbe	0.727	0.742	0.774	0.838
$z<-1$ ratio	0.1267	0.1255	0.1089	0.0812
Mean $\|z\|$	0.6892	0.6281	0.6813	0.5187
Skewness log-prob	$-2.577$	$-3.232$	$-2.668$	$\mathbf{-4.723}$

Appendix

Table S19: Overall metric values across the four-model ladder of Fig. 2 (b). Only TokenProbe and its shape-family relatives (

z<-1

ratio, mean

|z|

, log-prob skewness/kurtosis) move monotonically with accuracy. Every absolute-level signal (mean/sum log-prob, perplexity, geometric-/arithmetic-mean prob, high/low-confidence ratio, mean/normalised entropy, std log-prob) inverts at the Qwen-0.6B

\rightarrow

DS-Qw-1B step.

Model	AIME2025	AMC	MATH	Olympiad	Overall
Qwen-0.6B	$-0.2925$	$-0.2674$	$-0.2567$	$-0.2871$	$-0.2741$
DS-Qw-1B	$-0.3413$	$-0.2807$	$-0.2836$	$-0.3084$	$-0.2978$
Qwen-4B	$-0.1932$	$-0.1729$	$-0.1535$	$-0.1845$	$-0.1719$
Qwen-4B-Instruct	$-0.1584$	$-0.1202$	$-0.0825$	$-0.1340$	$-0.1137$

Appendix

Table S20: Per-dataset mean log-probability

\mu

. Higher (closer to

0

) indicates higher absolute confidence. The DS-Qw-1B row is uniformly more negative than Qwen-0.6B despite the

+10.9

percentage points accuracy gap.

Model	AIME2025	AMC	MATH	Olympiad	Overall
Qwen-0.6B	1.3420	1.3088	1.2953	1.3362	1.3187
DS-Qw-1B	1.4144	1.3286	1.3324	1.3679	1.3526
Qwen-4B	1.2140	1.1895	1.1669	1.2041	1.1890
Qwen-4B-Instruct	1.1748	1.1298	1.0880	1.1472	1.1238

Appendix

Table S21: Per-dataset perplexity

\exp(-\mu)

. Lower indicates higher absolute confidence. DS-Qw-1B is again worse than Qwen-0.6B on every dataset.

Model	AIME2025	AMC	MATH	Olympiad	Overall
Qwen-0.6B	0.4422	0.4049	0.3888	0.4344	0.4150
DS-Qw-1B	0.5014	0.4170	0.4222	0.4561	0.4415
Qwen-4B	0.2802	0.2520	0.2247	0.2698	0.2514
Qwen-4B-Instruct	0.2365	0.1798	0.1237	0.2011	0.1705

Appendix

Table S22: Per-dataset mean step entropy

\bar{H}

(nats, top-

K{=}20

truncation). Lower indicates more decisive next-token distributions. DS-Qw-1B is uniformly more uncertain than Qwen-0.6B although it is more accurate.

Dataset	Selection	Coverage	Desert100
AIME	Non-overlap	$0.993\pm 0.012$	$0.221\pm 0.074$
	Overlap	$0.882\pm 0.145$	$0.720\pm 0.074$
AMC	Non-overlap	$0.989\pm 0.019$	$0.207\pm 0.078$
	Overlap	$0.886\pm 0.136$	$0.685\pm 0.099$

Appendix

Table S23: Exploratory comparison of non-overlapping and overlapping core-window selection at

W{=}10

and core fraction

\approx 20\%

. Desert100 denotes the fraction of 100-token sliding intervals containing no selected core token.

Dataset	Jaccard	F1	Score gap	Time / rollout
AIME	$0.837\pm 0.042$	$0.911\pm 0.025$	$0.19\%\pm 0.28$	$6.41$ ms / $31.01$ ms
AMC	$0.831\pm 0.050$	$0.907\pm 0.032$	$0.15\%\pm 0.21$	$3.72$ ms / $17.38$ ms

Appendix

Table S24: Greedy non-overlapping selection compared with the DP optimum at

W{=}10

. Score gap is

(S_{\mathrm{DP}}-S_{\mathrm{greedy}})/S_{\mathrm{DP}}

	$W{=}5$	$W{=}10$	$W{=}20$	$W{=}30$
Avg. Accuracy	$38.4$	$42.3$	$41.4$	$42.2$
Avg. Tokens	$1{,}752$	$2{,}023$	$2{,}008$	$2{,}044$

Appendix

Table S25: Training-level window-size sensitivity for the KL variant on Qwen-0.6B. Numbers are averages over the four math benchmarks.

Local range around $W{=}10$			Coarse range $W{=}20$ to $30$
$W$	$\mathbf{J(M_{W},M_{10})}$	Adjacent $J$	$W$	$\mathbf{J(M_{W},M_{10})}$	Adjacent $J$
5	$0.448$	,	20	$0.428$	$0.798$
6	$0.460$	$0.615$	21	$0.422$	$0.804$
7	$0.497$	$0.641$	22	$0.415$	$0.809$
8	$0.567$	$0.662$	23	$0.408$	$0.814$
9	$0.700$	$0.680$	24	$0.403$	$0.817$

Appendix

Table S26: Mask-level window-size diagnostics.

J(M_{W},M_{10})

measures similarity to the default mask. Adjacent

J

measures

J(M_{W},M_{W-1})

. Values are averaged over AIME and AMC rollouts.

Data	Mask	Score / span	Role	Excerpt around selected window
AIME	Core	$4.9139$ / 455–464	Symbolic content	“ $=36-D\Rightarrow S_{\mathrm{even}}=$ ”, inside the derivation of odd/even digit sums.
AIME	Core	$4.5124$ / 4141–4150	Symbolic content	“ $2^{15}-2^{12}-2^{10}+2^{8}$ ”, inside an inclusion-exclusion count.
AMC	Core	$3.5591$ / 1056–1065	Symbolic content	“ $+3y+3y^{2}$ ”, inside the expansion of $(1+y)^{3}$ .
AIME	Anti	$-18.0658$ / 3292–3301	Hedging/repetition	“Wait, is there a way that this answer could be wrong?”
AMC	Anti	$-16.2504$ / 776–785	Hedging/repetition	“Hmm, that seems complicated. Maybe there is another approach.”
AMC	Anti	$-18.6124$ / 1876–1885	Formula/theorem	“applying logarithmic identities and simplifying the terms”, a content-bearing anti-mask example.

Appendix

Table S27: Representative excerpts from the manually annotated semantic probe. Scores are cumulative

\mathrm{NormLP}

values over the selected 10-token window, and spans are token positions within the rollout.

	AIME-2025		AMC
Category	Core	Anti	Core	Anti
Symbolic content	$98.0$	$4.0$	$98.0$	$4.0$
Decisive reasoning	$0.0$	$22.0$	$2.0$	$32.0$
Formula / theorem	$2.0$	$2.0$	$0.0$	$2.0$
Boilerplate connective	$0.0$	$14.0$	$0.0$	$6.0$
Hedging / repetition	$0.0$	$58.0$	$0.0$	$56.0$

Appendix

Table S28: Semantic-role distribution of core versus anti-mask windows in a 20-rollout probe. Content is the sum of symbolic, decisive-reasoning, and formula/theorem categories. Filler is the sum of boilerplate and hedging/repetition categories.

Model	Average		AIME		MATH		AMC		OlympiadBench
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Gemini-3.1-Pro (2K)	1,586	54.4	2,020	10.0	1,274	79.2	1,798	37.3	1,771	40.1
Gemini-3.1-Pro (8K)	2,581	88.0	5,163	80.0	1,385	99.4	2,532	96.4	3,358	79.0

Appendix

Table S29: Gemini-3.1-Pro budget check. The main table keeps the original 2K closed-API comparison. This appendix table reports the single newly added 8K Gemini run.

Model	Input	Output	Output / Qwen-4B	Source
GPT-5.4-Mini	$\$ 0.75$	$\$ 4.50$	$45\times$	OpenAI
Gemini-3.1-Pro Preview ( $\leq$ 200K prompt)	$\$ 2.00$	$\$ 12.00$	$120\times$	Google
Claude-4.6-Sonnet	$\$ 3.00$	$\$ 15.00$	$150\times$	Anthropic
Our Qwen-4B serving estimate	N/A	$\$ 0.10$	$1\times$	Estimate

Appendix

Table S30: API Pricing of Closed Models. Prices are reported in USD per million tokens. Ratios compare output price against our Qwen-4B serving estimate of

\

0.10$ /M output tokens.

Backbone	Params	Base Acc.	KL Acc.	RS Acc.	RS-KL	KL #Tk.	RS #Tk.
Qwen-0.6B	0.6B	40.7	42.3	36.3	$-6.0$	2,023	1,806
DeepScaleR-1.5B	1.5B	59.6	56.1	55.9	$-0.2$	1,698	1,993
Qwen-4B	4B	62.4	61.0	62.3	$+1.3$	1,886	2,026
Phi-Reasoning-4B	4B	59.6	62.0	62.6	$+0.6$	2,919	2,992
Qwen-8B	8B	61.9	62.7	68.3	$+5.6$	1,476	2,313

Appendix

Table S31: Capacity-dependent behavior of KL and RS. Accuracy and token counts are averaged over the four in-domain reasoning benchmarks in Tab. 2 . Positive RS-KL means RS is more accurate than KL.

Trace length $T$	$\mathrm{NormLP}$ (µs)	Window sel. (µs)	Total (µs)	Frac. of GRPO step
$1{,}024$	$25.3$	$126.6$	$151.8$	$0.009\,\%$
$2{,}048$	$26.2$	$280.6$	$306.8$	$0.018\,\%$
$4{,}096$	$29.1$	$629.0$	$658.2$	$0.037\,\%$
$8{,}192$	$33.9$	$1{,}475.9$	$1{,}509.8$	$0.082\,\%$
$16{,}384$	$44.3$	$3{,}850.6$	$3{,}894.8$	$0.210\,\%$

Appendix

Table S32: Computational overhead of TokenProbe operations.

\mathrm{NormLP}

(z-score) and greedy non-overlapping window selection both run on CPU on log-probabilities that vLLM already returns. Per-call timings averaged over

2{,}000

trials. Per-step amortization assumes a

30

s GRPO step with

16

rollouts.

\mathrm{NormLP}

is essentially constant-time across

T

(single z-score over the array). Window selection grows linearly in

T\cdot K

but stays in the single-millisecond regime even on

16

K traces. Inference-time overhead is exactly zero. The trained checkpoint is decoded by an unmodified vLLM generate call.

Setting	Average			AIME-2025		MATH-500		AMC		Olympiad-Bench
Setting	#Tk.	Acc.	%Trunc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Qwen-4B base @ $8$ K cap	$6{,}132$	$62.4$	N/A	$7{,}683$	$30.0$	$4{,}195$	$92.0$	$6{,}274$	$72.3$	$6{,}376$	$55.4$
Qwen-4B base @ $2$ K hard cap ( new )	$2{,}018$	$\mathbf{31.94}$	$93.3\%$	$2{,}048$	$10.0$	$1{,}941$	$59.0$	$2{,}040$	$32.5$	$2{,}041$	$26.2$
$\Delta$ (hard cap $-$ $8$ K base)	$-4{,}114$	$\mathbf{-30.46}$	N/A	$-5{,}635$	$-20.0$	$-2{,}254$	$-33.0$	$-4{,}234$	$-39.8$	$-4{,}335$	$-29.2$
Qwen-4B + KL (ours)	$1{,}886$	$61.0$	N/A	$2{,}253$	$33.3$	$1{,}246$	$88.2$	$2{,}092$	$66.3$	$1{,}952$	$56.1$
Qwen-4B + RS (ours)	$2{,}026$	$62.3$	N/A	$2{,}532$	$30.0$	$1{,}391$	$90.2$	$2{,}154$	$71.1$	$2{,}025$	$58.1$

Appendix

Table S33: Hard-truncation strawman vs. trained variants on Qwen3-4B. Hard-capping the unmodified base model at

2{,}048

tokens collapses average accuracy by

30.5

points relative to the same model at

8

K, and truncates

93.3\%

of problems at the budget boundary. Our trained KL/RS variants reach the same

\sim

2K-token output budget while retaining accuracy within

1.4

points of the unconstrained

8

K base, demonstrating that the token reductions in Tab. 2 are not equivalent to output clipping. #Tk. is the average generated token count (lower is better). Acc. is averaged accuracy in % (higher is better). %Trunc. is the fraction of rollouts that hit the

2{,}048

-token cap. The base

@8

K and our trained variants are reproduced from Tab. 2 .

Model	Average		AIME		MATH		AMC		OlympiadBench
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Qwen-3.6-Plus	18,099	92.2	30,755	90.0	9103	99.2	19,836	98.8	23,987	86.4
Gemini-3-Flash	4844	91.4	8332	93.3	2830	98.6	5049	96.4	6156	85.3
Qwen3-Next-80B-3B	8619	89.4	16,463	86.7	4202	98.0	9279	95.2	11,462	82.4
GLM-Air-106B-12B	12,125	89.0	22,320	86.7	6564	98.4	13,338	94.0	15,643	81.5
Step-Flash-196B-11B	13,907	87.3	31,432	90.0	4502	98.6	12,583	95.2	20,258	77.9

Appendix

Table S34: Unlimited setting (256K budget).

Model	Average		AIME		MATH		AMC		OlympiadBench
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Qwen-3.6-Plus	1998	7.8	1980	0.0	1961	18.0	2041	1.2	2020	1.5
Gemini-3-Flash	1836	31.1	2048	0.0	1647	51.8	1910	22.9	1958	18.1
Qwen3-Next-80B-3B	1832	27.7	2048	0.0	1636	49.4	1957	12.0	1952	14.8
GLM-Air-106B-12B	1979	10.6	2048	0.0	1891	22.2	2034	3.6	2035	3.3
Step-Flash-196B-11B	1571	40.9	2019	3.3	1213	65.0	1755	32.5	1794	25.8

Appendix

Table S35: Token budget gradient results at 2K.

Setting	Base Model	Distill Source	Think Tags
S1	Qwen3-4B [ 80 ]	Phi-4-Reasoning [ 1 ]	Yes
S2	Qwen3-4B [ 80 ]	Qwen3-30B-A3B [ 80 ]	No
S3	Qwen3-8B [ 80 ]	Phi-4-Reasoning [ 1 ]	Yes
S4	Qwen3-8B [ 80 ]	Qwen3-30B-A3B [ 80 ]	No

Appendix

Table S36: SFT distillation experiment matrix.

Feature	Phi4 Data	Qwen Data
Sample count	25,695	29,761
Contains <think> tags	100.0%	0.0%
Degenerate opening patterns	0.0%	70.6%
Contains \boxed{}	99.9%	81.9%
Average compression ratio	0.375	0.291

Appendix

Table S37: Distillation data quality comparison.

Setting	AIME	AMC	MATH	Olympiad	Overall	$\Delta$ (percentage points)
Qwen-4B think	60.0%	84.3%	95.8%	68.4%	77.1%	N/A
Qwen-8B think	46.7%	80.7%	94.4%	68.3%	72.5%	N/A
Qwen-4B nothink	20.0%	57.8%	85.0%	49.0%	53.0%	N/A
S1 (4B + Phi4)	3.3%	28.9%	58.2%	25.0%	37.6%	$-39.5$
S2 (4B + Qwen)	3.3%	18.1%	42.0%	16.6%	26.2%	$-50.9$
S3 (8B + Phi4)	13.3%	24.1%	60.2%	26.4%	39.1%	$-33.4$

Appendix

Table S38: SFT distillation accuracy vs. baselines. All 4 SFT settings suffer 33 to 51 percentage points accuracy degradation.

\Delta

denotes the drop from the corresponding base models.

Setting	Accuracy	Avg. Tokens	Efficiency (acc%/ktok)
Qwen3-4B think (RL)	77.1%	8,728	8.83
Qwen3-8B think (RL)	72.5%	9,244	7.84
S1 (4B + Phi4)	37.6%	4,822	7.80
S2 (4B + Qwen)	26.2%	7,258	3.62
S3 (8B + Phi4)	39.1%	4,161	9.39
S4 (8B + Qwen)	31.8%	6,127	5.18

Appendix

Table S39: Token usage and efficiency comparison. Efficiency is measured as accuracy % / kilo-tokens. S3 is the only SFT setting exceeding the RL baseline’s efficiency, but at

-33.4

percentage points accuracy cost.

Model	Correct TokenProbe	Incorrect TokenProbe	$r(\text{TokenProbe},\text{correct})$	$r(\text{tokens},\text{TokenProbe})$
4B think (RL baseline)	0.784	0.762	$+0.307$	$-0.415$
8B think (RL baseline)	0.790	0.774	$+0.254$	$-0.367$
S1 (4B + Phi4)	0.810	0.891	$\mathbf{-0.404}$	$\mathbf{+0.858}$
S2 (4B + Qwen)	0.800	0.878	$\mathbf{-0.464}$	$\mathbf{+0.599}$
S3 (8B + Phi4)	0.789	0.883	$\mathbf{-0.525}$	$\mathbf{+0.866}$
S4 (8B + Qwen)	0.782	0.852	$\mathbf{-0.422}$	$\mathbf{+0.492}$

Appendix

Table S40: TokenProbe reversal in SFT models. The sign of both correlations flips, indicating that the logprob signal no longer carries the same semantic meaning after SFT distillation.

Setting	Category	lp_std	ZC/100tok	Max pos. run	2nd-half TokenProbe
S1	Correct ( $n$ =10)	0.398	22.8	124	0.854
S1	Incorrect ( $n$ =30)	0.240	12.0	7,522	0.920
S3	Correct ( $n$ =11)	0.440	24.0	74	0.824
S3	Incorrect ( $n$ =29)	0.276	12.6	7,811	0.915
S4	Correct ( $n$ =7)	0.454	24.6	545	0.823
S4	Incorrect ( $n$ =33)	0.353	15.0	7,096	0.911

Appendix

Table S41: Degeneration loop statistics. Incorrect samples exhibit flattened logprob distributions, rare zero-crossings, and extremely long positive runs, all of which are hallmarks of repetitive degeneration.

Setting	Tokens	TokenProbe	Behavior
S1 (4B+Phi4)	451	0.885	Compact think section. Factorization $\to$ sum $\to$ correct answer $\boxed{284}$
S2 (4B+Qwen)	8,192	0.894	Empty think $\to$ correct reasoning initially, then enters infinite loop. “284’s sum of proper divisors is 220. 220’s sum of proper divisors is 284.” repeated until truncation
S3 (8B+Phi4)	484	0.872	Think section present, correct approach, but arithmetic error. Sums 220’s divisors to 234 instead of 284 $\to$ $\boxed{234}$
S4 (8B+Qwen)	8,192	0.948	Empty think $\to$ “This is a complex or challenging question…” $\to$ brute-force enumeration checking every integer from 4 to 365 $\to$ truncated

Appendix

Table S42: MATH #12 across all 4 SFT settings. The same problem elicits four completely different failure modes.

Figure S7: Failure modes of length-controlled RL training. (A) Llama-3.2-8B-Instruct under vanilla GRPO. Response length oscillates wildly and locks at the 4096-token cap after $\sim$ 200 steps. (B) Qwen3-4B-Instruct under both GRPO (blue) and our KL variant (red). Both runs saturate at the cap with degenerate, repetitive outputs. (C, D) Qwen3-0.6B-Base and DeepScaleR-1.5B trained with our method. Training is stable until the running compression scale crosses $\approx 1.6\times$ (dashed line, at step 355 for 0.6B and step 455 for 1.5B), after which length explodes, accuracy collapses to zero, and outputs degenerate into pure repetition.

Domain	Base	w/ KL	w/ RS
Math	0.782	0.887	0.846
General	0.736	0.846	0.802

Appendix

Table S43: TokenProbe on fixed 40-sample transfer diagnostics. Each domain contains four datasets with 10 fixed samples per dataset. We report only TokenProbe.

Model	Average	AIME	MATH	AMC	Olympiad
Qwen-0.6B w/ KL	0.90	1.96	1.70	1.83	0.95
Qwen-4B w/ RS	0.20	5.10	1.01	3.17	1.85

Appendix

Table S44: Run-to-run standard deviation. Each entry is the sample standard deviation across three repeated runs. We report standard deviation of accuracy in percentage points.

Backbone	Hardware	Average training time
Qwen-0.6B	$8\times$ H100 80GB	12h 14m
DeepScaleR-1.5B	$8\times$ H100 80GB	9h 10m
Qwen-4B	$8\times$ H100 80GB	21h 56m
Phi-Reasoning-4B	$8\times$ H100 80GB	25h 13m
Qwen-8B	$8\times$ H100 80GB	38h 20m

Appendix

Table S45: Compute resources for main training runs. All runs use a single node with

8\times

NVIDIA H100 80GB GPUs.

Asset	Identifier / source	License / terms	Role
DeepScaleR-Preview-Dataset	agentica-org/DeepScaleR-Preview-Dataset	MIT	Training data
Qwen family	Qwen/Qwen3-* , Qwen/Qwen3.5-* , and Qwen/Qwen3-Next-*	Apache-2.0	Training and open baselines
DeepScaleR-1.5B	agentica-org/DeepScaleR-1.5B-Preview	MIT	Training backbone
Phi-4-mini-reasoning	microsoft/Phi-4-mini-reasoning	MIT	Training backbone
DeepSeek-R1-Distill-Qwen	deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B	MIT	Open baseline
GPT-OSS	openai/gpt-oss-120b	Apache-2.0	Open baseline

Appendix

Table S46: Licenses and terms for existing assets. License entries follow the corresponding public model or dataset cards when available. Closed-API models are governed by provider API terms rather than open model licenses.

Asset	Identifier / source	License / terms	Role
DeepScaleR-Preview-Dataset	agentica-org/DeepScaleR-Preview-Dataset	MIT	Training data
Qwen family	Qwen/Qwen3-* , Qwen/Qwen3.5-* , and Qwen/Qwen3-Next-*	Apache-2.0	Training and open baselines
DeepScaleR-1.5B	agentica-org/DeepScaleR-1.5B-Preview	MIT	Training backbone
Phi-4-mini-reasoning	microsoft/Phi-4-mini-reasoning	MIT	Training backbone
DeepSeek-R1-Distill-Qwen	deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B	MIT	Open baseline
GPT-OSS	openai/gpt-oss-120b	Apache-2.0	Open baseline

Appendix

Dataset	Token deletion	Attention-key masking
MATH	2.6% / 1.1% / 0.8%	2.4% / 0.8% / 0.3%
OlympiadBench	1.8% / 0.7% / 0.5%	1.5% / 0.6% / 0.4%

Appendix

Table S47: Correctness damage under token deletion and attention-key masking. Entries in each method column are ordered High-NormLP / Random / Low-NormLP.

Model Name	API Identifier	Version	Mode	Output price	Official link
OpenAI
GPT-4o	gpt-4o	4o	Default	$10.00	Announcement
OpenAI o1	o1	o1	Reasoning	$60.00	Announcement
OpenAI o3	o3	o3	Reasoning	$8.00	Announcement
GPT-5.2 Pro	gpt-5.2-pro	5.2	Pro	$168.00	Announcement
GPT-5.4 Mini	gpt-5.4-mini	5.4	Mini	$4.50	Announcement

Appendix

Table S48: Closed-source baseline models used in our experiments. Version denotes the model version number. Links point to official announcements or model pages. Output prices are standard public list prices in USD per million tokens, as of September 24, 2026.

Model	Average		AIME-2025		MATH-500		AMC		OlympiadBench
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Previously reported closed-API baselines
GPT-5.4-Mini	987	69.6	1,714	36.7	589	88.6	1,137	73.5	1,232	56.6
Claude-4.6-Sonnet	1,221	61.3	1,907	20.0	801	86.0	1,413	51.8	1,478	46.1
Grok-4.1-Fast	4,945	52.3	9,361	10.0	1,866	74.8	4,017	43.4	7,144	38.5
Gemini-3.1-Pro	1,586	54.4	2,020	10.0	1,274	79.2	1,798	37.3	1,771	40.1

Appendix

Table S49: Per-task results on in-domain reasoning benchmarks. #Tk. is the average number of generated completion tokens per response, including billable reasoning tokens where present. Acc. is accuracy in percent.

Model	Average		MMLU		MMLU-Pro		LSAT		GPQA
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Previously reported closed-API baselines
Grok-4.1-Fast	2,892	88.2	1,026	90.9	2,288	84.1	2,797	96.1	5,459	81.8
Claude-4.5-Haiku	1,090	72.0	706	88.6	883	79.8	1,521	54.8	1,250	64.7
Gemini-3.1-Pro	1,294	68.6	681	92.8	1,055	80.3	1,727	54.4	1,714	47.0
Claude-4.6-Sonnet	1,290	57.1	645	88.8	965	73.0	1,797	35.7	1,754	30.8

Appendix

Table S50: Per-task results on out-of-domain generalization. The four tasks are MMLU, MMLU-Pro, LSAT, and GPQA. #Tk. is the average number of generated completion tokens per response, including billable reasoning tokens where present. Acc. is accuracy in percent.

Model	Average		AIME		MATH		AMC		Olympid
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Closed-API Reasoning Models
Claude-4.6-Opus	1,130	70.0	1,852	30.0	689	93.2	1,375	65.1	1,395	55.1
Claude-4.5-Haiku	1,339	72.0	1,803	23.3	1,061	89.8	1,533	66.3	1,500	61.6
Gemini-3.1-Flash-Lite	778	80.4	1,036	36.7	614	94.6	893	81.9	875	71.6
Open Source Reasoning Models

Appendix

Table S51: Main results on four in-domain reasoning benchmarks. #Tk. is the average number of generated tokens per response (lower is better). Acc. is averaged accuracy in % (higher is better).

Model	Average		AIME		MATH		AMC		OlympiadBench
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Qwen-3.6-Plus	15,207	80.0	25,812	56.7	8812	96.2	18,520	90.4	19,065	67.7
Gemini-3-Flash	4844	91.4	8332	93.3	2830	98.6	5049	96.4	6156	85.3
Qwen3-Next-80B-3B	8472	88.0	16,183	83.3	4185	97.8	9193	92.8	11,216	80.4
GLM-Air-106B-12B	11,278	86.1	20,859	83.3	6517	97.8	13,257	92.8	14,136	76.7
Step-Flash-196B-11B	10,147	80.2	23,018	56.7	4124	97.0	10,868	88.0	13,949	67.9

Appendix

Table S52: Token budget gradient results at 32K.

Model	Average		AIME		MATH		AMC		OlympiadBench
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Qwen-3.6-Plus	10,678	55.9	15,472	10.0	7432	80.4	13,440	42.2	12,530	41.5
Gemini-3-Flash	4802	90.4	8332	93.3	2830	98.6	4989	95.2	6083	83.6
Qwen3-Next-80B-3B	7045	77.3	12,568	56.7	3988	95.2	8334	83.1	8905	64.3
GLM-Air-106B-12B	8831	71.7	14,191	40.0	5970	90.4	10,837	71.1	10,465	59.3
Step-Flash-196B-11B	6692	71.7	13,740	33.3	3338	92.0	7752	72.3	8734	58.2

Appendix

Table S53: Token budget gradient results at 16K.

Model	Average		AIME		MATH		AMC		OlympiadBench
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Qwen-3.6-Plus	6492	38.5	7919	0.0	5196	64.8	7500	19.3	7265	23.1
Gemini-3-Flash	4106	73.8	6515	53.3	2673	92.4	4286	78.3	5039	60.3
Qwen3-Next-80B-3B	4991	60.2	7722	16.7	3326	83.0	6111	56.6	5965	45.8
GLM-Air-106B-12B	6033	49.6	7988	10.0	4711	72.8	7038	33.7	6801	36.1
Step-Flash-196B-11B	4245	63.2	7484	16.7	2527	85.2	5022	57.8	5279	49.6

Appendix

Table S54: Token budget gradient results at 8K.

Model	Average		AIME		MATH		AMC		OlympiadBench
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Qwen-3.6-Plus	3727	21.0	3959	0.0	3361	43.2	3986	7.2	3957	7.3
Gemini-3-Flash	2860	58.6	3885	20.0	2217	82.2	3038	54.2	3269	43.4
Qwen3-Next-80B-3B	3101	42.8	4096	0.0	2391	69.2	3590	25.3	3522	27.3
GLM-Air-106B-12B	3606	28.3	4096	0.0	3179	49.0	3875	16.9	3867	15.7
Step-Flash-196B-11B	2619	53.4	3962	10.0	1784	76.2	3004	45.8	3131	39.4

Appendix

Table S55: Token budget gradient results at 4K.

Model	Average		AIME		MATH		AMC		OlympiadBench
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Qwen-3.6-Plus	1016	0.1	990	0.0	1021	0.2	1024	0.0	1012	0.0
Gemini-3-Flash	1008	6.9	1024	0.0	986	16.4	1022	1.2	1023	0.9
Qwen3-Next-80B-3B	1007	7.6	1024	0.0	987	16.0	1021	3.6	1020	2.2
GLM-Air-106B-12B	1020	2.4	1024	0.0	1015	5.8	1024	0.0	1024	0.3
Step-Flash-196B-11B	912	25.8	1024	0.0	794	48.6	982	13.3	985	11.6

Appendix

Table S56: Token budget gradient results at 1K.

Model	Average		MMLU		MMLU-Pro		LSAT		GPQA
Model	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.	#Tk.	Acc.
Closed-API Reasoning Models
Grok-4.1-Fast	2892	88.2	1026	90.9	2288	84.1	2797	96.1	5459	81.8
Claude-4.5-Haiku	1090	72.0	706	88.6	883	79.8	1521	54.8	1250	64.7
Gemini-3.1-Pro	1294	68.6	681	92.8	1055	80.3	1727	54.4	1714	47.0
Claude-4.6-Sonnet	1290	57.1	645	88.8	965	73.0	1797	35.7	1754	30.8

Appendix

Table S57: Per-task results on out-of-domain generalization. Models are evaluated on four general-domain tasks (MMLU, MMLU-Pro, LSAT, GPQA) under a 2,048-token rollout cap. #Tk. is the average number of generated tokens per response (lower is better). Acc. is accuracy in % (higher is better).

Explore similar work

Aug 31, 2026cs.CL

Every Token Leaves a Ripple in the Stream of Thought: Eliciting Model-Internal Token Saliency for Chain-of-Thought Compression

Chain-of-thought (CoT) reasoning improves multi-step problem solving, but long reasoning traces inflate inference cost. Token-level CoT compression reduces this cost by pruning full reasoning chains into shorter traces for model adaptation, making token selection the central challenge. Existing methods often rely on external scorers or heuristic signals only indirectly tied to the model's internal answer computation. We instead adopt a model-internal perspective: as the model forms an answer, each reasoning token induces a ripple in the residual stream whose effect on the answer reflects the token's contribution to the underlying computation. Building on this view, we propose \textsc{MIST} (Model-Internal Saliency for Token-level CoT compression), which defines token importance along two complementary axes: \emph{necessity}, the drop in answer likelihood when a token's internal contribution is removed, and \emph{sufficiency}, the gain in answer likelihood when that contribution alone is provided. Combining the two yields a unified importance score for pruning. Across four reasoning benchmarks and four models, \textsc{MIST} consistently outperforms baseline methods, suggesting that model-internal saliency provides an effective proxy for reasoning-token importance.

Tianyi Zhao, Yinhan He, Wendy Zheng +2

University of Virginia

Chain-of-Thought Reasoning Reasoning Chain Token-Level Uncertainty Saliency

Ask PDF

Aug 5, 2026cs.AI

Fewer Tokens, Smaller Cache: Reward-Coordinated Efficient Reasoning

Large Reasoning Models (LRMs) excel on complex tasks through long chain-of-thought (CoT) reasoning, but their lengthy intermediate steps cause severe overthinking that inflates inference cost. KV-cache compression is a common solution, yet existing reasoning-oriented methods apply a uniform policy across the trajectory and judge compression only by what it removes from the cache. Two observations point the other way. First, a reasoning state's tolerance to context loss varies along the trajectory, and process reward tracks it: deleting tokens at high-reward steps preserves accuracy far better than deleting the same budget at random. Second, compression is not free on the generation side, since a smaller cache leads the model to generate more tokens, partly canceling the saving. Together these motivate coordinating both sides under a single process reward. We propose ReCo (Reward-Coordinated Compression), a step-wise framework in which a lightweight process-reward estimator scores each completed step and drives three components: (1) reward-adaptive KV-cache compression that shrinks the retained cache harder at high-reward steps and less at low-reward ones, (2) a reward-banded penalty on reflection tokens that curbs redundant generation, and (3) confidence-based early stopping that triggers when the reasoning is reliable. Across three reasoning models and six benchmarks, ReCo reduces generated tokens by 37%-65% and end-to-end latency by 2.08x-2.35x over Full CoT, all while largely preserving accuracy.

Qiyuan Zhu, Dezhi Li, Pengyu Cheng +8

1HKUST · 2XJTU · 3TSE +1

Large Reasoning Models Key-Value Cache Compression Inference Cost Cache Redundancy

Ask PDF

Sep 28, 2026cs.AI

Efficient Reasoning via Constrained Optimization in Latent Space

Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they still suffer from overthinking, generating redundant reasoning steps which incur substantial token consumption. Existing methods, such as suppressing reflective keywords or forcing shorter reasoning lengths, attempt to mitigate this issue but inevitably truncate necessary steps and induce underthinking, thereby compromising performance. To address this dilemma, we investigate the latent representations and observe that efficient reasoning steps naturally cluster into a concentrated region in latent space, while those deviating from this region tend to produce verbose sequences. To leverage this, we keep reasoning focused within this region via a quadratic program which projects deviating hidden states back into the region. Then we propose a novel training-free framework to achieve efficient reasoning that reduces token generation costs without sacrificing performance. Extensive experiments conducted on four models ranging from 1.5B to 14B, and across six benchmarks in math reasoning, coding, and scientific QA, validate the effectiveness of our method, up to a 12.1% improvement in accuracy while reducing generated tokens by 11.8% to 52.8%. Codes are available at https://github.com/hzn18/Opt4Reasoning.

Zhinan Hou, XingChen Li, Keyou You

Tsinghua University

Large Reasoning Models Reasoning Traces Latent Space Constrained Optimization

Ask PDF