From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representations

Yixin Yang; Yue Wang; Fandong Meng; Zhifang Sui

cs.CLOct 11, 2026

From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representations

Authors: Yixin Yang, Yue Wang, Fandong Meng, Zhifang Sui

Organizations: State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · WeChat AI, Tencent Inc., China

Abstract

Large reasoning models generate reasoning trajectories token by token. At a given prefix, multiple next-token candidates may be plausible, yet shape subsequent reasoning and differ in their chances of leading to the correct answer. Whether these differences are reflected in internal representations remains unclear. To study this question, we define relative long-horizon candidate value to quantify within-prefix differences and construct the Reasoning Candidate-Token Value (RCTV) dataset, pairing mathematical reasoning prefixes with plausible next-token candidates and their empirical long-horizon candidate values. Using RCTV, we examine hidden states, multilayer perceptron outputs, and attention outputs at candidate positions with sparse linear probes. Experimental results provide evidence that relative long-horizon candidate value is linearly decodable from these representations at multiple layers. Candidate-value probes can also outperform probability- and embedding-based baselines. Furthermore, a controlled online intervention experiment shows that using the decoded signals to guide token selection during generation can improve the chances of eventual success. Our findings provide evidence that internal representations encode information about relative long-horizon candidate value, offering insights into how local candidate-level representations relate to long-horizon reasoning outcomes.

Figures & tables

Figure 1: Illustration of our candidate-level analysis of long-horizon reasoning outcomes. Three plausible candidates at a shared reasoning prefix are shown. pp denotes percentage points. In this example, the pairwise ordering produced by the probe matches the empirical ordering of candidates by relative long-horizon value.

Subset	Model	Entropy	# Problems	# Candidates	# Rollouts
RCTV-Core	Qwen3-1.7B	$[q_{.95},1]$	500	2,715	347,520
RCTV-Qwen8B	Qwen3-8B	$[q_{.95},1]$	500	3,152	403,456
RCTV-Llama8B	DS-Llama-3.1-8B	$[q_{.95},1]$	200	3,051	390,528
RCTV-InterEnt	Qwen3-1.7B	$[q_{.85},q_{.95})$	500	1,678	214,784

Table 1: Dataset statistics for RCTV. The four subsets cover three reasoning models and two entropy regimes.

Figure 2: Candidate-value readout across layers and selected dimensionalities. Each cell reports the mean pairwise AUC over ten problem-level fold seeds for hidden states, MLP outputs, and attention outputs. Within each representation family, $\star$ , $\diamond$ , $\triangle$ , and $\circ$ denote the first- through fourth-highest observed mean AUC, respectively.

Repr	$k$	Pairwise $\uparrow$	Tier $\uparrow$	Regret@1 $\downarrow$
Baselines
Uniform	-	50.0	50.0	0.0355
Log prob	1	50.9	51.6	0.0346
Prob rank	1	52.4	51.8	0.0346
Embedding	16	52.2 $\pm$ 2.0	52.2 $\pm$ 1.9	0.0340 $\pm$ 0.0014
Act norm ( $\mathrm{M}_{9}$ )	1	53.2 $\pm$ 0.9	54.5 $\pm$ 0.8	0.0338 $\pm$ 0.0000

Table 2: Candidate-value readout results and baselines. We report the mean and standard deviation over ten problem-level fold seeds. For each internal-representation family and the embedding baseline, we report the strongest observed configuration according to mean pairwise AUC.

Figure 3: Candidate-value readout from candidate-position MLP outputs across Qwen3-1.7B, Qwen3-8B, and DS-Llama-3.1-8B. Each cell reports mean pairwise AUC over ten problem-level fold seeds. $\star$ , $\diamond$ , $\triangle$ , and $\circ$ mark the first- through fourth-highest observed mean AUC within each model. Layer indices are normalized by model depth.

Configuration		Main readout	Controls		Alternative readouts
Layer	$k$	Top- $k$ ridge	Random dims	Permuted labels	Signed sum	Corr-weighted
$\mathrm{M}_{9}$	4	58.7 $\pm$ 2.6	50.1 $\pm$ 0.7	49.9 $\pm$ 0.7	58.7 $\pm$ 2.6	58.8 $\pm$ 2.5
$\mathrm{M}_{9}$	8	58.3 $\pm$ 2.3	50.4 $\pm$ 1.0	50.5 $\pm$ 0.9	58.3 $\pm$ 2.6	58.7 $\pm$ 2.4
$\mathrm{M}_{17}$	2	58.1 $\pm$ 2.5	50.1 $\pm$ 0.9	49.6 $\pm$ 0.5	58.1 $\pm$ 2.7	58.3 $\pm$ 2.6
$\mathrm{M}_{17}$	4	58.0 $\pm$ 2.7	49.7 $\pm$ 0.7	49.6 $\pm$ 0.6	57.8 $\pm$ 3.0	58.2 $\pm$ 2.8

Table 3: Robustness of four high-performing candidate-value MLP readouts. Values are mean pairwise AUC (%) and standard deviation over ten problem-level fold seeds. The table compares the primary Top-

k

ridge readout with null controls and alternative readout rules. Control repetitions are first averaged within each fold seed.

$\mathrm{M}_{9}$ ( $k=4$ )
Dim	Selected	Direction
432	45/50	$+$ 45/45
950	38/50	$-$ 38/38
256	18/50	$+$ 18/18
1274	17/50	$-$ 17/17

Table 4: Selection frequency and sign consistency of the most frequently selected MLP dimensions across 10 problem-level fold seeds (50 training folds). Direction denotes the sign of the training-fold correlation.

Figure 4: Mean continuation accuracy under different intervention budgets

B

. The dashed line denotes generation without intervention.

Policy	Avg@128	$\Delta$ vs. Standard (95% CI)
Standard	50.21	—
Uniform	51.84	+1.63 [-2.03, +5.30]
Top-prob	50.56	+0.35 [-3.47, +4.18]
Probe-guided	54.51	+4.30 [+0.60, +8.28]

Table 5: Online intervention results at

B=64

. Avg@128 is reported in percent; differences are in percentage points with 95% paired problem-bootstrap CIs.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Subset	Source C/W	Difficulty 2/3/4/5/6	Position median [ $p_{10}$ , $p_{90}$ ]	Candidates/prefix mean (median)
RCTV-Core	252/248	100/100/100/100/100	2,259 [420, 6,224]	5.43 (5.0)
RCTV-Qwen8B	236/264	100/100/100/100/100	2,960 [851, 6,641]	6.30 (6.0)
RCTV-Llama8B	97/103	40/40/40/40/40	3,232 [800, 6,860]	15.26 (15.0)
RCTV-InterEnt	252/248	100/100/100/100/100	1,894 [433, 5890]	3.36 (3.0)

Appendix

Table 6: Additional statistics for the RCTV variants. C/W denotes prefixes drawn from source trajectories with correct and incorrect final answers. Difficulty entries report the numbers of problems in strata 2–6, where each stratum is determined by the number of successful outcomes among eight sampled generations. Prefix positions are measured in tokens from the beginning of the generated reasoning trajectory.

Figure 5: Original Big-Math source composition of the selected problems in each RCTV variant. The four most common sources are shown separately. Other comprises AOPS Forum, Math, OmniMath, HARP, GSM8K, and OpenMath. Percentages are normalized within each variant.

Panel of Figure 6 — Figure 6: Additional analyses of within-prefix candidate-value variation. Left: Value-gap distributions grouped by whether the source trajectory produces a correct final answer. Right: Value gaps across prefix positions; the line shows binned means and the shaded region indicates 95% bootstrap confidence intervals.

Empirical gap	# Pairs	# Prefixes	Median conf.	Pairs $\geq.90$	HC prefixes
$>0$	5,515	460	0.794	26.2%	340
$\geq 0.04$	2,043	328	0.915	56.2%	285
$\geq 0.08$	729	199	0.971	100.0%	199

Appendix

Table 7: Pairwise uncertainty in rollout-estimated candidate values. Directional confidence is the posterior probability that the empirically higher-valued candidate has the higher underlying success probability, computed from all 128 rollouts using a Jeffreys prior. The final column, labeled high-confidence (HC) prefixes, reports prefixes containing at least one pair with directional confidence of at least

0.90

Model	Parameters	Number of layers	Hidden size
Qwen3-1.7B	1.7B	28	2,048
Qwen3-8B	8.2B	36	4,096
DeepSeek-R1-Distill-Llama-8B	8B	32	4,096

Appendix

Table 8: Architectural specifications of the reasoning models used in our experiments.

Figure 7: Distribution of value gaps in RCTV-Core.

Figure 8: Candidate-value probing of prefix–token interaction representations across layers and selected dimensionalities. Each cell reports the mean pairwise AUC over ten problem-level fold seeds for attention outputs, MLP outputs, and hidden states. Within each representation family, $\star$ , $\diamond$ , $\triangle$ , and $\circ$ mark the first- through fourth-highest observed mean pairwise AUC, respectively.

Repr	$k$	Pairwise $\uparrow$	Tier $\uparrow$	Regret@1 $\downarrow$
Attn ( $\mathrm{A}_{15}$ )	128	59.4 $\pm$ 3.2	57.5 $\pm$ 2.2	0.0328 $\pm$ 0.0012
MLP ( $\mathrm{M}_{20}$ )	256	59.2 $\pm$ 2.6	57.8 $\pm$ 2.5	0.0322 $\pm$ 0.0008
Hidden ( $\mathrm{H}_{28}$ )	256	56.4 $\pm$ 2.0	54.3 $\pm$ 2.0	0.0345 $\pm$ 0.0014

Appendix

Table 9: Candidate-value readout results for prefix-token interaction representations. We report the mean and standard deviation over ten problem-level fold seeds. For each representation family, we report the strongest observed configuration according to mean pairwise AUC.

Figure 9: Sensitivity of candidate-value readout performance to the value-gap margin

m

(left) and tier threshold

\epsilon

(right) on RCTV-Core. Points and error bars indicate the mean and standard deviation across ten problem-level fold seeds. Log Prob uses untrained next-token log probabilities and therefore does not vary across fold seeds. Dashed lines indicate chance-level AUC.

Figure 10: Standard deviation of pairwise AUC across ten problem-level fold seeds for candidate-value readouts. Each cell corresponds to a layer and selected dimensionality.

Figure 11: Fraction of the ten problem-level fold seeds with pairwise AUC above $0.5$ for candidate-value readouts.

Repr	$k$	Pairwise $\uparrow$	Tier $\uparrow$	Regret@1 $\downarrow$
Baselines
Uniform	-	50.0	50.0	0.0386
Log prob	1	48.2	48.5	0.0412
Prob rank	1	52.5	49.5	0.0412
Embedding	64	55.1 $\pm$ 1.8	53.0 $\pm$ 1.3	0.0393 $\pm$ 0.0010
Candidate-value readouts

Appendix

Table 11: Candidate-value readout results and baselines for Qwen3-8B. For fitted readouts, we report the mean and standard deviation over ten problem-level fold seeds; parameter-free baselines are reported as single values. For each internal-representation family and the embedding baseline, we report the strongest observed configuration according to mean pairwise AUC.

Repr	$k$	Pairwise $\uparrow$	Tier $\uparrow$	Regret@1 $\downarrow$
Baselines
Uniform	-	50.0	50.0	0.0490
Log prob	1	50.9	50.1	0.0509
Prob rank	1	50.7	50.5	0.0509
Embedding	128	52.6 $\pm$ 1.7	53.2 $\pm$ 1.5	0.0504 $\pm$ 0.0013
Candidate-value readouts

Appendix

Table 12: Candidate-value readout results and baselines for DS-Llama-3.1-8B. For fitted readouts, we report the mean and standard deviation over ten problem-level fold seeds; parameter-free baselines are reported as single values. For each internal-representation family and the embedding baseline, we report the strongest observed configuration according to mean pairwise AUC.

Figure 12: Candidate-value readout from hidden states, MLP outputs, and attention outputs in Qwen3-8B. Each cell reports mean pairwise AUC across ten problem-level fold seeds. Within each representation family, $\star$ , $\diamond$ , $\triangle$ , and $\circ$ mark the first- through fourth-highest observed mean pairwise AUC, respectively.

Figure 13: Candidate-value readout from hidden states, MLP outputs, and attention outputs in DS-Llama-3.1-8B. Each cell reports mean pairwise AUC across ten problem-level fold seeds. Within each representation family, $\star$ , $\diamond$ , $\triangle$ , and $\circ$ mark the first- through fourth-highest observed mean pairwise AUC, respectively.

Repr	Decoder	$k$	Pairwise $\uparrow$	Tier $\uparrow$	Regret@1 $\downarrow$
Baselines
Uniform	-	-	50.0	50.0	0.0275
Log prob	-	1	54.9	54.9	0.0281
Prob rank	-	1	55.2	53.3	0.0281
Fixed Core-selected dimensions at $\mathrm{M}_{17}$
MLP ( $\mathrm{M}_{17}$ )	Ridge	2	57.3 $\pm$ 2.9	56.1 $\pm$ 2.3	0.0241 $\pm$ 0.0006

Appendix

Table 13: Readout results with fixed Core-selected dimensions on RCTV-InterEnt. The dimension sets are

\{1834,312\}

and

\{1834,312,2021,822\}

\mathrm{M}_{17}

, and

\{432,950\}

and

\{432,950,256,1274\}

\mathrm{M}_{9}

. Decoder weights are estimated using each InterEnt training fold. We report mean and standard deviation over ten problem-level fold seeds; parameter-free baselines are reported as single values.

$B$	Strategy	Avg@128 (%)	$\Delta$ [95% CI]
0	Standard	54.93	—
8	Probe-guided	57.10	+2.16 [-1.76, +6.28]
16	Probe-guided	57.42	+2.49 [-2.04, +7.79]
32	Probe-guided	60.64	+5.71 [+1.20, +10.58]
64	Probe-guided	59.63	+4.70 [+0.65, +9.54]

Appendix

Table 14: Online intervention results on the 61 problems outside the offline pairwise-evaluation subset. Continuation accuracy is reported in percent, and differences relative to standard sampling are reported in percentage points with paired 95% bootstrap confidence intervals. Standard sampling involves no interventions (

B=0

$B$	Strategy	Trajectory 0	Trajectory 1
0	Standard	46.98	53.44
8	Probe-guided	47.49	54.12
16	Probe-guided	48.34	56.23
32	Probe-guided	50.87	56.87
64	Probe-guided	52.35	56.67

Appendix

Table 15: Continuation accuracy (%) reported separately for the two trajectory seeds, estimated using 128 continuations per prefix.

$B$	Strategy	Pass@4	Pass@8	Pass@16	Pass@32
0	Standard	57.14	59.54	61.35	62.82
8	Top-prob	59.37	62.16	64.31	65.91
	Uniform	58.11	60.86	62.92	64.51
	Probe-guided	57.78	60.64	62.61	63.89
16	Top-prob	57.01	59.72	61.70	63.06
	Uniform	58.77	61.01	62.62	63.93

Appendix

Table 16: Prefix-conditioned Pass@

k

(%) across intervention budgets. Standard sampling involves no interventions.

$\boldsymbol{B}$	Policy	Avg@128	$\boldsymbol{\Delta}$ vs. Standard [95% CI]	$\boldsymbol{\Delta}$ vs. Uniform [95% CI]	$\boldsymbol{\Delta}$ vs. Top-prob [95% CI]
0	Standard (no intervention)	50.21	—	—	—
8	Uniform	51.63	+1.42 [-2.36, +5.10]	—	-0.61 [-3.74, +2.43]
	Top-prob	52.23	+2.03 [-1.54, +5.70]	+0.61 [-2.43, +3.74]	—
	Probe-guided	50.81	+0.60 [-2.48, +3.81]	-0.82 [-3.67, +2.06]	-1.43 [-3.97, +0.88]
16	Uniform	52.21	+2.00 [-0.59, +4.80]	—	+1.56 [-2.14, +5.16]
	Top-prob	50.64	+0.44 [-3.43, +4.41]	-1.56 [-5.16, +2.14]	—

Appendix

Table 17: Mean continuation accuracy under online intervention. Differences are in percentage points; intervals are 95% paired problem-bootstrap CIs. Under interventions based on candidate-value readouts, continuation accuracy increases with the budget, with the largest gain at

B=64

LLM Reasoning

Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.

See all 926 papers

Every Token Leaves a Ripple in the Stream of Thought: Eliciting Model-Internal Token Saliency for Chain-of-Thought Compression
Aug 31, 2026Tianyi Zhao, Yinhan He, Wendy Zheng +2LLM Compression Efficient Language Model Reasoning
University of Virginia
On the Token Value Inequality in Efficient Reasoning
Sep 27, 2026Runjia Zeng, Hang Hua, Yiyang Liu +5RL for Language Model Reasoning Efficient Language Model Reasoning
Rochester Institute of Technology · Purdue University · MIT-IBM Watson AI Lab +3
Reasoning Can Be Restored by Correcting a Few Decision Tokens
May 16, 2026Changshuo Shen, Leheng Sheng, Yuxin Chen +2Efficient Language Model Reasoning Inference-Time Compute Allocation
University of Science and Technology of China, Hefei, China · National University of Singapore, Singapore

Aug 31, 2026J/K move · Enter open · S save

Every Token Leaves a Ripple in the Stream of Thought: Eliciting Model-Internal Token Saliency for Chain-of-Thought Compression

Tianyi Zhao, Yinhan He, Wendy Zheng, Jundong Li+1

University of Virginia

LLM Compression Efficient Language Model Reasoning CoT Compression Large Reasoning Models

Open paper AskPDF arXiv

Chain-of-thought (CoT) reasoning improves multi-step problem solving, but long reasoning traces inflate inference cost. Token-level CoT compression reduces this cost by pruning full reasoning chains into shorter traces for model adaptation, making token selection the central challenge. Existing methods often rely on external scorers or heuristic signals only indirectly tied to the model's internal answer computation. We instead adopt a model-internal perspective: as the model forms an answer, each reasoning token induces a ripple in the residual stream whose effect on the answer reflects the token's contribution to the underlying computation. Building on this view, we propose MIST (Model-Internal Saliency for Token-level CoT compression), which defines token importance along two complementary axes: necessity, the drop in answer likelihood when a token's internal contribution is removed, and sufficiency, the gain in answer likelihood when that contribution alone is provided. Combining the two yields a unified importance score for pruning. Across four reasoning benchmarks and four models, MIST consistently outperforms baseline methods, suggesting that model-internal saliency provides an effective proxy for reasoning-token importance.

Abstract

LLM Reasoning

Explore similar work

Every Token Leaves a Ripple in the Stream of Thought: Eliciting Model-Internal Token Saliency for Chain-of-Thought Compression

On the Token Value Inequality in Efficient Reasoning

Reasoning Can Be Restored by Correcting a Few Decision Tokens