Understanding how individual token choices affect reasoning length is important for characterizing reasoning models. Building on prior findings that changing the first reasoning token (hereafter, the first token) can substantially alter reasoning length, we focus on this initial choice to investigate how its influence operates during subsequent generation. One possibility is that the first token changes the early text that follows, which in turn affects reasoning length. However, replacing only the first token of an existing reasoning prefix and resuming generation substantially changes the remaining reasoning length, even though all other prefix tokens are unchanged. To trace this persistent influence, we compare next-token distributions while varying only the first token and find that sensitivity is concentrated at a small set of generation positions. Blocking attention from these positions to the first token reduces the length gap between first-token conditions. Using next-token distributions from a different first-token condition at these positions also substantially changes subsequent reasoning length. Together, these findings suggest that attention to the first token at a few sensitive positions helps sustain its length effect, illustrating how a local token choice persistently influences a sequence-level outcome.
Figures & tables
Figure 1: First-token effects on GSM8K: (a) first token frequencies under natural sampling; (b) mean reasoning length conditioned on the natural first token; (c) mean reasoning length under forced first tokens. Heatmap cells report percentages, bar colors identify first tokens. Other includes all remaining first tokens. Q and L denote Qwen and Llama. Error bars indicate 95% bootstrap confidence intervals.
Figure 2: First-token replacement on GSM8K with subsequent prefix tokens held fixed. (a) Long-to-short replacements. (b) Short-to-long replacements. Values are ratios of mean remaining reasoning lengths to matched unchanged-prefix references, averaging the observed source suffix and two new continuations. Error bars indicate 95% bootstrap confidence intervals.
Model
Opening
Rem. tokens
Acc. (%)
Long-to-short
Qwen-1.5B
Source
1,461
88.6
To
419
86.3
First
525
86.9
Qwen-7B
Source
1,019
94.8
To
483
93.7
Table 1: Remaining reasoning length and answer accuracy on GSM8K at K=200 .
Figure 3: Concentrated first-token sensitivity. (a) Aggregate JS concentration on GSM8K and MATH for all three models. Positions are ranked within each trajectory; curves show mean cumulative JS shares. Markers indicate the highest-JS 10% of positions; diagonals denote uniform contributions. (b) Unsmoothed JS along one Qwen-7B GSM8K trajectory, illustrating intermittent peaks.
Figure 4: First-token attention versus JS on GSM8K (top) and MATH (bottom), under First (left) and Okay (right) first tokens. Curves show mean attention percentiles by within-trajectory JS decile. Insets report mean Spearman correlations before ( ρ ) and after ( ρpos ) adjusting for token position.
Figure 5: Reasoning length under first-token attention blocking on GSM8K (top) and MATH (bottom). Base denotes no intervention; Every 3 masks every third eligible position. The remaining groups mask positions with JS above the indicated threshold. Error bars indicate 95% bootstrap confidence intervals.
Model
Opening
Every 3
0.2
0.1
0.05
GSM8K
Qwen-1.5B
First
33.3
6.1
10.9
16.9
Okay
33.4
5.2
9.7
15.7
Qwen-7B
First
33.4
6.2
10.4
16.1
Okay
33.4
6.3
10.9
16.6
Llama-8B
First
33.3
9.5
16.2
23.2
Table 2: Attention intervention rates (%). Values are mean per-trajectory percentages of eligible positions. Every 3 denotes periodic masking; numeric headings denote JS thresholds.
Figure 6: Reasoning length under distribution replacement on GSM8K (top) and MATH (bottom). Base denotes no intervention; Every 3 replaces distributions at every third eligible position; numeric headings denote JS thresholds. Error bars show 95% question-bootstrap intervals.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: First-token effects on MATH-500, HumanEval, and FOLIO (rows). Columns show (a) natural opening frequencies in percent, (b) mean reasoning length by natural opening, and (c) mean reasoning length under forced openings. Length panels share a vertical scale within each dataset. Error bars show 95% question-bootstrap intervals; dashes indicate openings with no observed natural samples. Other includes all remaining openings. Q and L denote Qwen and Llama.
Figure 8: Prefix-preserving first-token replacements on MATH-500. Top: long-to-short replacements across three models. Bottom: short-to-long replacements across the same models. Values are ratios of mean remaining reasoning lengths to matched unchanged-prefix references; error bars show 95% question-bootstrap intervals.
Figure 9: Prefix-preserving first-token replacement on HumanEval. Ratios compare mean edited continuation lengths with the three-continuation unchanged-prefix references. Error bars show 95% question-bootstrap intervals; dashed lines mark a ratio of one.
Figure 10: Prefix-preserving first-token replacement on FOLIO, using the same comparison as Figure 9 .
Long-to-short (Okay / Alright → To / First)
Model
K
Questions / sources
Reference
To
First
Length
Acc.
Length
Acc.
Length
Acc.
Qwen-1.5B
50
500 / 1000
1,608.6
88.5
274.5
86.3
343.2
87.6
100
500 / 1000
1,565.4
89.2
358.5
86.7
432.5
87.8
150
500 / 1000
1,472.6
89.9
404.5
87.1
489.8
87.8
200
500 / 1000
1,461.2
88.6
419.5
86.3
524.7
86.9
Appendix
Table 3: Prefix-preserving interventions on GSM8K. Length is mean remaining reasoning tokens; Acc. is answer accuracy (%). Reference averages the observed source outcome and two unchanged-prefix continuations; each replacement averages three continuations. All quantities use matched eligible sources and equal question weights.
Long-to-short (Okay / Alright → To / First)
Model
K
Questions / sources
Reference
To
First
Length
Acc.
Length
Acc.
Length
Acc.
Qwen-1.5B
50
500 / 1000
3,503.2
78.0
1,367.5
73.5
1,872.0
76.0
100
500 / 1000
3,452.4
77.8
1,810.6
75.4
2,174.3
77.0
150
500 / 1000
3,406.4
78.7
1,872.7
75.1
2,248.6
77.1
200
500 / 1000
3,342.6
78.5
1,968.3
76.4
2,234.3
77.3
Appendix
Table 4: Prefix-preserving interventions on MATH-500. Length is mean remaining reasoning tokens; Acc. is answer accuracy (%). Reference averages the observed source outcome and two unchanged-prefix continuations; each replacement averages three continuations. All quantities use matched eligible sources and equal question weights.
Long-to-short (Okay / Alright → To / First)
Model
K
Questions / sources
Reference
To
First
Length
Acc.
Length
Acc.
Length
Acc.
Qwen-1.5B
50
164 / 328
3,554.1
64.7
3,140.9
64.9
3,092.6
63.9
100
164 / 328
3,589.9
63.7
3,263.8
63.2
3,165.6
64.2
150
164 / 328
3,441.8
64.6
3,116.0
66.5
3,106.7
66.6
200
164 / 328
3,426.0
65.0
3,186.4
64.7
3,133.2
62.7
Appendix
Table 5: Prefix-preserving interventions on HumanEval. Length is mean remaining reasoning tokens; Acc. is answer accuracy (%). Reference averages the observed source outcome and two unchanged-prefix continuations; each replacement averages three continuations. All quantities use matched eligible sources and equal question weights.
Long-to-short (Okay / Alright → To / First)
Model
K
Questions / sources
Reference
To
First
Length
Acc.
Length
Acc.
Length
Acc.
Qwen-1.5B
50
499 / 998
1,580.1
44.6
884.7
42.1
796.9
41.5
100
499 / 998
1,531.2
44.0
881.3
42.4
810.5
41.3
150
499 / 998
1,515.9
43.4
856.0
44.1
804.4
42.2
200
499 / 998
1,402.6
43.6
869.9
42.2
816.5
42.6
Appendix
Table 6: Prefix-preserving interventions on FOLIO. Length is mean remaining reasoning tokens; Acc. is answer accuracy (%). Reference averages the observed source outcome and two unchanged-prefix continuations; each replacement averages three continuations. All quantities use matched eligible sources and equal question weights.
Figure 11: Long-to-short prefix-preserving intervention. Tokens r2,…,r200 are identical in the two conditions. Bracketed ellipses mark omitted text; lengths count the entire reasoning continuation after position 200, excluding the final answer. The source revisits the calculation several times, whereas the edited continuation reaches the same answer with less repeated checking.
Figure 13: HumanEval long-to-short prefix replacement. Bracketed ellipses indicate omissions; lengths count the full reasoning suffix after position 200. Both continuations develop a modulo-three solution and check examples, with different amounts of elaboration.
Figure 15: FOLIO long-to-short prefix replacement. Bracketed ellipses indicate omissions; lengths count the full reasoning suffix after position 200. The edited continuation reaches the same label with less discussion of the geographically irrelevant premise.
Figure 17: JS concentration for the other three long–short pairs on GSM8K and MATH. Columns identify the compared first tokens; colors identify models. For each dataset, the upper row uses all scanned positions except the first; the lower row excludes the first 50 positions and reranks and renormalizes the remainder within each trajectory. Curves average repetitions and source conditions within questions, then questions. Tail curves retain questions with available tails in all conditions and repetitions (248–250 questions per setting). Markers indicate the highest-JS 10%; diagonals denote uniform contributions. Bands show 95% question-bootstrap intervals from 2,000 resamples.
Figure 18: First-token attention versus JS for the other three long–short pairs on GSM8K. Rows identify pairs; columns show attention under the short-associated (left) and long-associated (right) first-token conditions. Curves show mean within-trajectory attention percentiles by JS decile; bands show 95% question-bootstrap intervals. Insets report mean Spearman correlations before ( ρ ) and after ( ρpos ) adjusting for token position.
Figure 19: First-token attention versus JS for the other three long–short pairs on MATH, using the same layout and statistics as Figure 18 .
Figure 20: Unsmoothed First–Okay JS for GSM8K QID 2396 (left) and MATH-500 QID 84 (right), using natural-sampling replicate 0. QIDs are zero-based row indices in the respective dataset files; L is reasoning length. Shading marks the first 50 positions. Panels show the complete scanned trajectories except the first position; horizontal scales differ with trajectory length.
Figure 21: First-token attention blocking on HumanEval (top) and FOLIO (bottom). Base denotes no intervention; Every 3 uses periodic masking; numeric headings denote JS thresholds. Error bars show 95% question-bootstrap intervals.
Figure 22: Attention-blocking case from Qwen-7B on GSM8K (question ID 958). Columns compare First and Okay openings; rows show generation without and with attention blocking at Jt>0.05 . The unmodified Okay trajectory is excerpted, with […] marking omissions; the other reasoning traces are shown in full. All four trajectories yield the correct answer, 28.
Model
Direction
Every 3
0.2
0.1
0.05
GSM8K
Qwen-1.5B
F→O
33.3
3.4
5.4
8.5
O→F
33.3
10.6
16.9
23.7
Qwen-7B
F→O
33.3
3.8
6.0
9.4
O→F
33.4
7.0
11.2
16.4
Llama-8B
F→O
33.3
2.5
3.6
5.4
Appendix
Table 8: Distribution-replacement rates (%). F and O denote First and Okay. Values are mean per-trajectory percentages of eligible positions; numeric headings denote JS thresholds.
Figure 23: Distribution replacement on HumanEval (top) and FOLIO (bottom). Colors identify the replacement direction; Base uses the source-opening condition, Every 3 denotes periodic replacement, and numeric headings denote JS thresholds. Error bars show 95% question-bootstrap intervals.
We consider the question: when a large language reasoning model makes a choice, did it think first and then decide to, or decide first and then think? In this paper, we present evidence that detectable, early-encoded decisions shape chain-of-thought in reasoning models. Specifically, we show that a simple linear probe successfully decodes tool-calling decisions from pre-generation activations with very high confidence, and in some cases, even before a single reasoning token is produced. Activation steering supports this causally: perturbing the decision direction leads to inflated deliberation, and flips behavior in many examples (between 7 - 79% depending on model and benchmark). We also show through behavioral analysis that, when steering changes the decision, the chain-of-thought process often rationalizes the flip rather than resisting it. Together, these results suggest that reasoning models can encode action choices before they begin to deliberate in text.
Esakkivel Esakkiraja, Sai Rajeswar, Denis Akhiyarov +1
Khoury College of Computer Sciences Northeastern University · Mila, ServiceNow Research · ServiceNow
Large language models increasingly rely on long chains of thought to improve accuracy, yet such gains come with substantial inference-time costs. We revisit token-efficient post-training and argue that existing sequence-level reward-shaping methods offer limited control over how reasoning effort is allocated across tokens. To bridge the gap, we propose IAPO, an information-theoretic post-training framework that assigns token-wise advantages based on each token's conditional mutual information (MI) with the final answer. This yields an explicit, principled mechanism for identifying informative reasoning steps and suppressing low-utility exploration. We provide a theoretical analysis showing that our IAPO can induce monotonic reductions in reasoning verbosity without harming correctness. Empirically, IAPO consistently improves reasoning accuracy while reducing reasoning length by up to 36%, outperforming existing token-efficient RL methods across various reasoning datasets. Extensive empirical evaluations demonstrate that information-aware advantage shaping is a powerful and general direction for token-efficient post-training. The code is available at https://github.com/YinhanHe123/IAPO.
Yinhan He, Yaochen Zhu, Mingjia Shi +5
University of Virginia, Charlottesville, VA, USA · LinkedIn Inc., Sunnyvale, CA, USA
Large language models (LLMs) have demonstrated impressive reasoning capabilities by scaling test-time compute via long Chain-of-Thought (CoT). However, recent findings suggest that raw token counts are unreliable proxies for reasoning quality: increased generation length does not consistently correlate with accuracy and may instead signal "overthinking," leading to performance degradation. In this work, we quantify inference-time effort by identifying deep-thinking tokens -- tokens where internal predictions undergo significant revisions in deeper model layers prior to convergence. Across four challenging mathematical and scientific benchmarks (AIME 24/25, HMMT 25, and GPQA-diamond) and a diverse set of reasoning-focused models (GPT-OSS, DeepSeek-R1, and Qwen3), we show that deep-thinking ratio (the proportion of deep-thinking tokens in a generated sequence) exhibits a robust and consistently positive correlation with accuracy, substantially outperforming both length-based and confidence-based baselines. Leveraging this insight, we introduce Think@n, a test-time scaling strategy that prioritizes samples with high deep-thinking ratios. We demonstrate that Think@n matches or exceeds standard self-consistency performance while significantly reducing inference costs by enabling the early rejection of unpromising generations based on short prefixes.