Beyond the Opening: Tracing the First Token's Persistent Influence on Reasoning Length

Ruochen Liu; Hao Yan; Ziluowen Luo; Senzhang Wang; Sihong Xie

cs.CLOct 25, 2026

Beyond the Opening: Tracing the First Token's Persistent Influence on Reasoning Length

Authors: Ruochen Liu, Hao Yan, Ziluowen Luo, Senzhang Wang, Sihong Xie

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Central South University, China

Abstract

Understanding how individual token choices affect reasoning length is important for characterizing reasoning models. Building on prior findings that changing the first reasoning token (hereafter, the first token) can substantially alter reasoning length, we focus on this initial choice to investigate how its influence operates during subsequent generation. One possibility is that the first token changes the early text that follows, which in turn affects reasoning length. However, replacing only the first token of an existing reasoning prefix and resuming generation substantially changes the remaining reasoning length, even though all other prefix tokens are unchanged. To trace this persistent influence, we compare next-token distributions while varying only the first token and find that sensitivity is concentrated at a small set of generation positions. Blocking attention from these positions to the first token reduces the length gap between first-token conditions. Using next-token distributions from a different first-token condition at these positions also substantially changes subsequent reasoning length. Together, these findings suggest that attention to the first token at a few sensitive positions helps sustain its length effect, illustrating how a local token choice persistently influences a sequence-level outcome.

Figures & tables

Figure 1: First-token effects on GSM8K: (a) first token frequencies under natural sampling; (b) mean reasoning length conditioned on the natural first token; (c) mean reasoning length under forced first tokens. Heatmap cells report percentages, bar colors identify first tokens. Other includes all remaining first tokens. Q and L denote Qwen and Llama. Error bars indicate 95% bootstrap confidence intervals.

Figure 2: First-token replacement on GSM8K with subsequent prefix tokens held fixed. (a) Long-to-short replacements. (b) Short-to-long replacements. Values are ratios of mean remaining reasoning lengths to matched unchanged-prefix references, averaging the observed source suffix and two new continuations. Error bars indicate 95% bootstrap confidence intervals.

Model	Opening	Rem. tokens	Acc. (%)
Long-to-short
Qwen-1.5B	Source	1,461	88.6
	To	419	86.3
	First	525	86.9
Qwen-7B	Source	1,019	94.8
	To	483	93.7

Table 1: Remaining reasoning length and answer accuracy on GSM8K at

K=200

Figure 3: Concentrated first-token sensitivity. (a) Aggregate JS concentration on GSM8K and MATH for all three models. Positions are ranked within each trajectory; curves show mean cumulative JS shares. Markers indicate the highest-JS 10% of positions; diagonals denote uniform contributions. (b) Unsmoothed JS along one Qwen-7B GSM8K trajectory, illustrating intermittent peaks.

Figure 4: First-token attention versus JS on GSM8K (top) and MATH (bottom), under First (left) and Okay (right) first tokens. Curves show mean attention percentiles by within-trajectory JS decile. Insets report mean Spearman correlations before (

\rho

) and after (

\rho_{\mathrm{pos}}

) adjusting for token position.

Figure 5: Reasoning length under first-token attention blocking on GSM8K (top) and MATH (bottom). Base denotes no intervention; Every 3 masks every third eligible position. The remaining groups mask positions with JS above the indicated threshold. Error bars indicate 95% bootstrap confidence intervals.

Model	Opening	Every 3	$0.2$	$0.1$	$0.05$
GSM8K
Qwen-1.5B	First	33.3	6.1	10.9	16.9
	Okay	33.4	5.2	9.7	15.7
Qwen-7B	First	33.4	6.2	10.4	16.1
	Okay	33.4	6.3	10.9	16.6
Llama-8B	First	33.3	9.5	16.2	23.2

Table 2: Attention intervention rates (%). Values are mean per-trajectory percentages of eligible positions. Every 3 denotes periodic masking; numeric headings denote JS thresholds.

Figure 6: Reasoning length under distribution replacement on GSM8K (top) and MATH (bottom). Base denotes no intervention; Every 3 replaces distributions at every third eligible position; numeric headings denote JS thresholds. Error bars show 95% question-bootstrap intervals.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: First-token effects on MATH-500, HumanEval, and FOLIO (rows). Columns show (a) natural opening frequencies in percent, (b) mean reasoning length by natural opening, and (c) mean reasoning length under forced openings. Length panels share a vertical scale within each dataset. Error bars show 95% question-bootstrap intervals; dashes indicate openings with no observed natural samples. Other includes all remaining openings. Q and L denote Qwen and Llama.

Figure 8: Prefix-preserving first-token replacements on MATH-500. Top: long-to-short replacements across three models. Bottom: short-to-long replacements across the same models. Values are ratios of mean remaining reasoning lengths to matched unchanged-prefix references; error bars show 95% question-bootstrap intervals.

Figure 9: Prefix-preserving first-token replacement on HumanEval. Ratios compare mean edited continuation lengths with the three-continuation unchanged-prefix references. Error bars show 95% question-bootstrap intervals; dashed lines mark a ratio of one.

Figure 10: Prefix-preserving first-token replacement on FOLIO, using the same comparison as Figure 9 .

Long-to-short (Okay / Alright $\rightarrow$ To / First)
Model	$K$	Questions / sources	Reference		To		First
			Length	Acc.	Length	Acc.	Length	Acc.
Qwen-1.5B	50	500 / 1000	1,608.6	88.5	274.5	86.3	343.2	87.6
	100	500 / 1000	1,565.4	89.2	358.5	86.7	432.5	87.8
	150	500 / 1000	1,472.6	89.9	404.5	87.1	489.8	87.8
	200	500 / 1000	1,461.2	88.6	419.5	86.3	524.7	86.9

Appendix

Table 3: Prefix-preserving interventions on GSM8K. Length is mean remaining reasoning tokens; Acc. is answer accuracy (%). Reference averages the observed source outcome and two unchanged-prefix continuations; each replacement averages three continuations. All quantities use matched eligible sources and equal question weights.

Long-to-short (Okay / Alright $\rightarrow$ To / First)
Model	$K$	Questions / sources	Reference		To		First
			Length	Acc.	Length	Acc.	Length	Acc.
Qwen-1.5B	50	500 / 1000	3,503.2	78.0	1,367.5	73.5	1,872.0	76.0
	100	500 / 1000	3,452.4	77.8	1,810.6	75.4	2,174.3	77.0
	150	500 / 1000	3,406.4	78.7	1,872.7	75.1	2,248.6	77.1
	200	500 / 1000	3,342.6	78.5	1,968.3	76.4	2,234.3	77.3

Appendix

Table 4: Prefix-preserving interventions on MATH-500. Length is mean remaining reasoning tokens; Acc. is answer accuracy (%). Reference averages the observed source outcome and two unchanged-prefix continuations; each replacement averages three continuations. All quantities use matched eligible sources and equal question weights.

Long-to-short (Okay / Alright $\rightarrow$ To / First)
Model	$K$	Questions / sources	Reference		To		First
			Length	Acc.	Length	Acc.	Length	Acc.
Qwen-1.5B	50	164 / 328	3,554.1	64.7	3,140.9	64.9	3,092.6	63.9
	100	164 / 328	3,589.9	63.7	3,263.8	63.2	3,165.6	64.2
	150	164 / 328	3,441.8	64.6	3,116.0	66.5	3,106.7	66.6
	200	164 / 328	3,426.0	65.0	3,186.4	64.7	3,133.2	62.7

Appendix

Table 5: Prefix-preserving interventions on HumanEval. Length is mean remaining reasoning tokens; Acc. is answer accuracy (%). Reference averages the observed source outcome and two unchanged-prefix continuations; each replacement averages three continuations. All quantities use matched eligible sources and equal question weights.

Long-to-short (Okay / Alright $\rightarrow$ To / First)
Model	$K$	Questions / sources	Reference		To		First
			Length	Acc.	Length	Acc.	Length	Acc.
Qwen-1.5B	50	499 / 998	1,580.1	44.6	884.7	42.1	796.9	41.5
	100	499 / 998	1,531.2	44.0	881.3	42.4	810.5	41.3
	150	499 / 998	1,515.9	43.4	856.0	44.1	804.4	42.2
	200	499 / 998	1,402.6	43.6	869.9	42.2	816.5	42.6

Appendix

Table 6: Prefix-preserving interventions on FOLIO. Length is mean remaining reasoning tokens; Acc. is answer accuracy (%). Reference averages the observed source outcome and two unchanged-prefix continuations; each replacement averages three continuations. All quantities use matched eligible sources and equal question weights.

Panel of Figure 11 — Figure 11: Long-to-short prefix-preserving intervention. Tokens $r_{2},\ldots,r_{200}$ are identical in the two conditions. Bracketed ellipses mark omitted text; lengths count the entire reasoning continuation after position 200, excluding the final answer. The source revisits the calculation several times, whereas the edited continuation reaches the same answer with less repeated checking.

Panel of Figure 13 — Figure 13: HumanEval long-to-short prefix replacement. Bracketed ellipses indicate omissions; lengths count the full reasoning suffix after position 200. Both continuations develop a modulo-three solution and check examples, with different amounts of elaboration.

Panel of Figure 15 — Figure 15: FOLIO long-to-short prefix replacement. Bracketed ellipses indicate omissions; lengths count the full reasoning suffix after position 200. The edited continuation reaches the same label with less discussion of the geographically irrelevant premise.

Figure 17: JS concentration for the other three long–short pairs on GSM8K and MATH. Columns identify the compared first tokens; colors identify models. For each dataset, the upper row uses all scanned positions except the first; the lower row excludes the first 50 positions and reranks and renormalizes the remainder within each trajectory. Curves average repetitions and source conditions within questions, then questions. Tail curves retain questions with available tails in all conditions and repetitions (248–250 questions per setting). Markers indicate the highest-JS 10%; diagonals denote uniform contributions. Bands show 95% question-bootstrap intervals from 2,000 resamples.

Figure 18: First-token attention versus JS for the other three long–short pairs on GSM8K. Rows identify pairs; columns show attention under the short-associated (left) and long-associated (right) first-token conditions. Curves show mean within-trajectory attention percentiles by JS decile; bands show 95% question-bootstrap intervals. Insets report mean Spearman correlations before (

\rho

) and after (

\rho_{\mathrm{pos}}

) adjusting for token position.

Figure 19: First-token attention versus JS for the other three long–short pairs on MATH, using the same layout and statistics as Figure 18 .

Figure 20: Unsmoothed First–Okay JS for GSM8K QID 2396 (left) and MATH-500 QID 84 (right), using natural-sampling replicate 0. QIDs are zero-based row indices in the respective dataset files;

L

is reasoning length. Shading marks the first 50 positions. Panels show the complete scanned trajectories except the first position; horizontal scales differ with trajectory length.

Figure 21: First-token attention blocking on HumanEval (top) and FOLIO (bottom). Base denotes no intervention; Every 3 uses periodic masking; numeric headings denote JS thresholds. Error bars show 95% question-bootstrap intervals.

Panel of Figure 22 — Figure 22: Attention-blocking case from Qwen-7B on GSM8K (question ID 958). Columns compare First and Okay openings; rows show generation without and with attention blocking at $J_{t}>0.05$ . The unmodified Okay trajectory is excerpted, with $[\ldots]$ marking omissions; the other reasoning traces are shown in full. All four trajectories yield the correct answer, 28.

GSM8K
Model	Direction	Every 3	$0.2$	$0.1$	$0.05$
Qwen-1.5B	$F\to O$	33.3	3.4	5.4	8.5
	$O\to F$	33.3	10.6	16.9	23.7
Qwen-7B	$F\to O$	33.3	3.8	6.0	9.4
	$O\to F$	33.4	7.0	11.2	16.4
Llama-8B	$F\to O$	33.3	2.5	3.6	5.4

Appendix

Table 8: Distribution-replacement rates (%).

F

and

O

denote First and Okay. Values are mean per-trajectory percentages of eligible positions; numeric headings denote JS thresholds.

Figure 23: Distribution replacement on HumanEval (top) and FOLIO (bottom). Colors identify the replacement direction; Base uses the source-opening condition, Every 3 denotes periodic replacement, and numeric headings denote JS thresholds. Error bars show 95% question-bootstrap intervals.

Mechanistic Interpretability

Mechanistic interpretability studies how neural networks implement behaviors through internal features, circuits, and causal information flow.

See all 388 papers

Therefore I am. I Think
Apr 1, 2026Esakkivel Esakkiraja, Sai Rajeswar, Denis Akhiyarov +1LLM Decision-Making Implicit Reasoning in Language Models
Khoury College of Computer Sciences Northeastern University · Mila, ServiceNow Research · ServiceNow
IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning
Feb 22, 2026Yinhan He, Yaochen Zhu, Mingjia Shi +5Efficient Language Model Reasoning Language Model Post-Training
University of Virginia, Charlottesville, VA, USA · LinkedIn Inc., Sunnyvale, CA, USA
Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens
Feb 13, 2026Wei-Lin Chen, Liqian Peng, Tian Tan +5LLM Inference Efficiency Inference-Time Scaling
University of Virginia · Google

Apr 1, 2026J/K move · Enter open · S save

Therefore I am. I Think

Esakkivel Esakkiraja, Sai Rajeswar, Denis Akhiyarov, Rajagopal Venkatesaramani

Khoury College of Computer Sciences Northeastern University · Mila, ServiceNow Research · ServiceNow

LLM Decision-Making Implicit Reasoning in Language Models LLM Interpretability

Open paper AskPDF arXiv

We consider the question: when a large language reasoning model makes a choice, did it think first and then decide to, or decide first and then think? In this paper, we present evidence that detectable, early-encoded decisions shape chain-of-thought in reasoning models. Specifically, we show that a simple linear probe successfully decodes tool-calling decisions from pre-generation activations with very high confidence, and in some cases, even before a single reasoning token is produced. Activation steering supports this causally: perturbing the decision direction leads to inflated deliberation, and flips behavior in many examples (between 7 - 79% depending on model and benchmark). We also show through behavioral analysis that, when steering changes the decision, the chain-of-thought process often rationalizes the flip rather than resisting it. Together, these results suggest that reasoning models can encode action choices before they begin to deliberate in text.

Abstract

Mechanistic Interpretability

Explore similar work

Therefore I am. I Think

IAPO: Information-Aware Policy Optimization for Token-Efficient Reasoning

Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens