Mechanistic Interpretability of Chain-of-Thought Reasoning via Sequential Activation Patching

Murat Dura; Serkan Öztürk; Selma Tekir

cs.CLAug 23, 2026

Mechanistic Interpretability of Chain-of-Thought Reasoning via Sequential Activation Patching

Authors: Murat Dura, Serkan Öztürk, Selma Tekir

Organizations: Department of Computer Engineering Izmir Institute of Technology Izmir, Türkiye

Abstract

Large Language Models (LLMs) demonstrate remarkable problem-solving capabilities when guided by Chain-of-Thought (CoT) prompting, yet the internal mechanisms underlying these improvements remain poorly understood. In this work, we investigate where CoT-related causal effects emerge across the generated reasoning trajectory and which attention heads carry signals that contribute to final-answer computation. Because CoT reasoning unfolds over multiple generated tokens, standard activation patching at a single static token position is insufficient to characterize these temporally distributed effects. To address this limitation, we introduce a sequential activation patching framework that traces CoT-conditioned attention-head activations across token positions and aggregates their effects using Part-of-Speech-guided analysis. We further introduce Sequential Multi-Head Patching to evaluate the joint contribution of distributed head sets, together with cross-question and random activation controls. Targeted zero-ablation experiments show that the identified heads are functionally important for successful answer generation and affect several overlapping mechanisms, including reasoning-trajectory maintenance, answer anchoring, exemplar-target separation, and numerical generation. Overall, our results provide evidence for distributed reasoning-support sub-circuits associated with CoT-conditioned computation.

Figures & tables

Figure 1: The figure illustrates our sequential activation patching setup (Appendix A , Algorithm 1 ). First, the model generates a Chain-of-Thought (CoT) token sequence, where each token $y_{t}$ corresponds to a generated reasoning token such as “Katie,” “friends,” or “is.” For each token, we extract the attention-head activations from the clean CoT run and patch them into the corrupted No-CoT run at the final answer generation position. The effect of each patched token is then measured using JSD and MRR. Since individual token-level effects can be noisy, the results are grouped by Part-of-Speech (POS) categories such as [VB], [NNP], [CD], [NNS], and [VBZ]. For each POS category, token-level patching results are aggregated into a layer-by-head heatmap, showing which attention heads consistently carry CoT-related information for that linguistic role.

Figure 2: Comparison of average JSD across different patching setups. The figure displays the baseline JSD ( $0.3865$ ) alongside the final JSD scores for Sequential Multi-Head Patching (Table 2 ), Sequential Multi-Head Cross Patching (Table 3 ), and Random-Activation Sequential Multi-Head Patching (Table 6 ). Lower JSD indicates greater distributional similarity to the clean CoT run.

Figure 3: Comparison of average MRR across Sequential Multi-Head Patching (Table 4 ), Sequential Multi-Head Cross Patching (Table 5 ), and Random-Activation Sequential Multi-Head Patching (Table 7 ). Higher values indicate better recovery of the CoT–No-CoT decision-margin difference.

Experiment	Metric	Equation Prompt	No-CoT Prompt	CoT Prompt
		Accuracy	Accuracy	Accuracy
Normal Run	N/A	34.38%	31.25%	65.63%
Sequential Multi-Head Patching	JSD	9.38%	12.50%	0.00%
Sequential Multi-Head Patching	MRR	9.38%	9.38%	6.25%
Sequential Multi-Head Cross Patching	JSD	6.25%	6.25%	9.38%
Sequential Multi-Head Cross Patching	MRR	18.75%	12.50%	6.25%

Table 1: Zero-ablation accuracy results across prompting settings. The Normal Run row reports the original accuracy without ablation or patching. All other rows report accuracy after zero-ablating the corresponding selected head sets.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Overview of the Sequential Multi-Head Patching and deduplication procedure. First, the top- $k$ most influential attention heads are selected for each individual Part-of-Speech (POS) category based on their isolated patching performance. These category-specific heads are then merged into a single global multi-head set. To resolve intervention conflicts, a deduplication mechanism is applied such that any head appearing in multiple POS categories is assigned exclusively to its best-performing functional role. Finally, the activations associated with the resulting global set are jointly patched into the corrupted No-CoT execution at the final-answer position, and the resulting recovery is quantified using JSD and MRR.

ID	Type	Eq	Stat	Logit (No $\rightarrow$ Pat.)		Prob (No $\rightarrow$ Pat.)		JSD
chal-961	Addition	1	$\times\times$	23.61 $\rightarrow$	28.68	36.2% $\rightarrow$	97.8%	0.0072
Heads: L16H13[NNP], L16H10[CD], L13H12[:], L13H0[NNP], L14H5[VBZ], L14H10[JJ], L12H13[VBZ], L14H13[.], L16H8[.], L14H1[NN], L13H4[NN], L12H0[JJ], L15H1[NNS], L13H7[PRP$]
chal-569	Addition	1	$\checkmark\times$	21.87 $\rightarrow$	22.13	27.7% $\rightarrow$	21.9%	0.4047
Heads: L10H7[:], L13H12[.], L8H7[:], L14H5[IN], L10H6[NN], L14H8[NN], L8H3[VBD], L10H12[JJ], L16H10[TO], L14H4[JJ], L9H2[.], L14H2[CD], L10H3[CD], L13H10[NNS], L9H1[IN], L10H9[IN], L13H13[CC], L13H4[DT], L13H1[DT], L12H4[VBZ], L14H10[VBZ], L11H5[VBZ]
chal-290	Addition	1	$\times\times$	22.23 $\rightarrow$	22.82	28.2% $\rightarrow$	40.6%	0.2662
Heads: L13H12[VBD], L20H10[IN], L15H13[:], L14H10[RB], L14H7[CD], L20H12[.], L14H13[.], L14H5[NNS], L13H13[DT], L12H12[TO], L12H6[,], L17H8[VBZ]

Appendix

Table 2: Detailed results of Sequential Multi-Head Patching using JSD across 32 SVAMP instances. The Stat column indicates correct (✓) or incorrect (

\times

) answers for CoT (left) and No-CoT (right) prompting. The Eq column indicates how many operations the question includes. Logit and probability shifts represent values before and after patching attention head outputs into the No-CoT execution. Lower JSD values denote a closer distributional alignment with the clean CoT run. The Heads sub-row lists the aggregated sets of the most influential attention heads, selected based on their specific POS tags for each respective problem. The notation format for these elements indicates the layer number, head number, and the corresponding POS category (e.g., L15H3[VB] refers to Layer 15, Head 3, and POS category VB). The average final JSD across all multi-head patching instances is 0.2415.

T-ID	S-ID	T-Stat	S-Stat	Logit (No $\rightarrow$ Pat.)		Prob (No $\rightarrow$ Pat.)		JSD
chal-961	chal-674	$\times\times$	$\checkmark$	23.61 $\rightarrow$	22.07	36.2% $\rightarrow$	27.1%	0.3626
Cross-Heads: L12H0[CD], L14H5[.], L15H1[:], L14H10[MD], L14H8[DT], L12H12[VBP], L14H13[.], L14H1[NNS], L13H1[,], L8H3[VBZ]
chal-569	chal-514	$\checkmark\times$	$\times$	21.87 $\rightarrow$	23.26	27.7% $\rightarrow$	55.2%	0.1864
Cross-Heads: L14H5[.], L10H7[:], L13H12[CD], L11H3[NN], L14H8[NN], L14H10[VBZ], L13H4[DT], L13H6[CD], L15H1[NNS], L8H3[IN], L13H1[DT]
chal-290	chal-33	$\times\times$	$\checkmark$	22.23 $\rightarrow$	22.73	28.2% $\rightarrow$	52.3%	0.1993
Cross-Heads: L13H12[.], L15H8[:], L20H10[NNS], L14H10[.], L14H13[.], L14H1[CD], L12H0[CD], L15H10[NNS], L20H12[IN], L13H1[IN], L14H7[DT], L22H7[VBZ]

Appendix

Table 3: Detailed results of Sequential Multi-Head Cross Patching using JSD across 32 SVAMP instances. T-ID and S-ID represent the Target and Source problem IDs, respectively. The T-Stat column indicates correct (✓) or incorrect (

\times

) answers for the target problem under CoT (left) and No-CoT (right) prompting, while the S-Stat column shows the correctness of the source CoT generation. Logit and probability shifts represent values before and after patching the source attention-head outputs into the target No-CoT execution. Lower JSD values denote a closer distributional alignment with the clean target CoT run. The Cross-Heads sub-row lists the aggregated sets of the most influential attention heads extracted from the source problem, selected based on their specific POS tags. The notation indicates the layer number, head number, and corresponding POS category (e.g., L15H3[VB] denotes Layer 15, Head 3, with POS category VB). The average final JSD across all multi-head cross-patching instances is 0.2594.

ID	Type	Eq	Stat	Logit (No $\rightarrow$ Pat.)		Prob (No $\rightarrow$ Pat.)		MRR
chal-961	Addition	1	$\times\times$	23.61 $\rightarrow$	19.38	36.2% $\rightarrow$	0.3%	-0.6122
Heads: L16H13[NNP], L16H10[CD], L13H12[:], L13H1[NNP], L13H0[NNP], L14H5[VBZ], L14H10[CD], L12H13[VBZ], L16H8[.], L14H13[.], L13H4[NN], L14H1[NN], L12H0[JJ], L15H1[NNS], L13H7[PRP$]
chal-569	Addition	1	$\checkmark\times$	21.87 $\rightarrow$	21.07	27.7% $\rightarrow$	6.7%	-0.3062
Heads: L10H7[:], L13H12[NN], L16H10[NNS], L14H5[CC], L14H8[NN], L8H3[VBD], L10H12[JJ], L8H5[JJ], L9H2[.], L10H3[CD], L9H1[CD], L10H9[IN], L13H13[CC], L13H4[DT], L13H10[DT], L14H10[VBZ]
chal-290	Addition	1	$\times\times$	22.23 $\rightarrow$	24.18	28.2% $\rightarrow$	52.3%	-0.0281
Heads: L14H10[NN], L13H8[:], L22H7[:], L13H12[VBD], L8H3[EX], L12H12[TO], L20H12[.], L14H7[CD], L15H10[NNS], L15H3[NNP], L13H4[RB], L12H6[,], L14H12[VBZ], L14H13[VBZ], L20H10[VBZ]

Appendix

Table 4: Detailed results of Sequential Multi-Head Patching using MRR (Margin Recovery Ratio) across 32 SVAMP instances. The Stat column indicates correct (✓) or incorrect (

\times

) answers for CoT (left) and No-CoT (right) prompting. Logit and probability shifts represent values before and after patching attention head outputs into the No-CoT execution. Higher MRR values indicate greater recovery of the CoT–No-CoT margin difference. The Heads sub-row lists the aggregated sets of the most influential attention heads, selected based on their specific POS tags for each respective problem. The notation indicates the layer number, head number, and corresponding POS category (e.g., L15H3[VB] denotes Layer 15, Head 3, with POS category VB). The average MRR across all multi-head patching instances is 0.1780.

T-ID	S-ID	T-Stat	S-Stat	Logit (No $\rightarrow$ Pat.)		Prob (No $\rightarrow$ Pat.)		MRR
chal-961	chal-674	$\times\times$	$\checkmark$	23.61 $\rightarrow$	21.54	36.2% $\rightarrow$	14.6%	-0.0750
Cross-Heads: L12H0[CD], L14H5[.], L12H12[VBP], L14H10[MD], L14H8[IN], L14H13[.], L14H1[NNS], L13H1[,], L8H3[VBZ]
chal-569	chal-290	$\checkmark\times$	$\times$	21.87 $\rightarrow$	22.71	27.7% $\rightarrow$	4.6%	-0.3640
Cross-Heads: L10H7[:], L13H12[IN], L14H8[TO], L13H10[NN], L15H1[NNP], L9H1[NN], L13H4[.], L16H8[CD], L14H10[CD], L13H7[NNP], L14H5[NNS], L10H12[JJ], L10H9[TO], L10H13[VBZ]
chal-290	chal-622	$\times\times$	$\checkmark$	22.23 $\rightarrow$	23.30	28.2% $\rightarrow$	35.5%	-0.0634
Cross-Heads: L14H10[:], L13H12[.], L15H10[CD], L15H3[CD], L20H12[VBD], L20H10[.], L13H11[VBD], L15H1[NN], L14H7[NNS], L12H0[VBN], L15H7[,], L12H6[PRP], L14H12[VBZ]

Appendix

Table 5: Detailed results of Sequential Multi-Head Cross Patching using MRR (Margin Recovery Ratio) across 32 SVAMP instances. T-ID and S-ID represent the Target and Source problem IDs, respectively. The T-Stat column indicates correct (✓) or incorrect (

\times

) answers for the target problem under CoT (left) and No-CoT (right) prompting, while the S-Stat column shows the correctness of the source CoT generation. Logit and probability shifts represent values before and after patching the source attention-head outputs into the target No-CoT execution. Higher MRR values indicate greater recovery of the target CoT–No-CoT margin difference using the source activations. The Cross-Heads sub-row lists the aggregated sets of the most influential attention heads extracted from the source problem, selected based on their specific POS tags. The notation indicates the layer number, head number, and corresponding POS category (e.g., L15H3[VB] denotes Layer 15, Head 3, with POS category VB). The average MRR across all multi-head cross-patching instances is 0.0962.

ID	Type	Eq	Stat	Logit (No $\rightarrow$ Pat.)		Prob (No $\rightarrow$ Pat.)		JSD
chal-961	Addition	1	$\times\times$	23.61 $\rightarrow$	21.52	36.2% $\rightarrow$	27.1%	0.3626
Random Heads: L12H5[RAND], L14H5[RAND], L13H4[RAND], L15H10[RAND], L12H9[RAND], L3H4[RAND], L13H1[RAND], L10H9[RAND], L2H9[RAND], L12H4[RAND], L12H13[RAND], L11H11[RAND], L10H6[RAND], L9H2[RAND]
chal-569	Addition	1	$\checkmark\times$	21.87 $\rightarrow$	22.13	27.7% $\rightarrow$	18.8%	0.4319
Random Heads: L14H1[RAND], L10H9[RAND], L15H6[RAND], L10H0[RAND], L10H5[RAND], L8H1[RAND], L4H12[RAND], L9H8[RAND], L1H11[RAND], L10H11[RAND], L8H12[RAND], L10H13[RAND], L15H1[RAND], L4H13[RAND], L14H8[RAND], L9H0[RAND], L10H10[RAND], L13H1[RAND], L3H10[RAND], L10H7[RAND], L10H12[RAND], L9H3[RAND]
chal-290	Addition	1	$\times\times$	22.23 $\rightarrow$	29.10	28.2% $\rightarrow$	38.2%	0.2812
Random Heads: L23H1[RAND], L12H13[RAND], L12H4[RAND], L15H1[RAND], L18H2[RAND], L13H13[RAND], L20H9[RAND], L14H8[RAND], L13H1[RAND], L19H2[RAND], L12H7[RAND], L15H8[RAND]

Appendix

Table 6: Detailed results of Random-Activation Sequential Multi-Head Patching using JSD across 32 SVAMP instances. The Stat column indicates correct (✓) or incorrect (

\times

) answers for CoT (left) and No-CoT (right) prompting. Logit and probability shifts represent values before and after patching random activations into the corresponding No-CoT heads. Lower JSD values denote a closer distributional alignment with the clean CoT run. Random-Activation Sequential Multi-Head Patching serves as a control condition for comparison with POS-guided head selection. The average final JSD across all Random-Activation Sequential Multi-Head Patching instances is 0.4018.

ID	Type	Eq	Stat	Logit (No $\rightarrow$ Pat.)		Prob (No $\rightarrow$ Pat.)		MRR
chal-961	Addition	1	$\times\times$	23.61 $\rightarrow$	21.43	36.2% $\rightarrow$	3.3%	-0.3336
Random Heads: L14H5[RAND], L16H8[RAND], L14H2[RAND], L11H0[RAND], L4H9[RAND], L10H5[RAND], L10H9[RAND], L16H10[RAND], L14H12[RAND], L10H4[RAND], L8H3[RAND], L8H6[RAND], L13H8[RAND], L14H1[RAND], L16H4[RAND]
chal-569	Addition	1	$\checkmark\times$	21.87 $\rightarrow$	21.63	27.7% $\rightarrow$	25.9%	-0.0214
Random Heads: L13H12[RAND], L14H5[RAND], L10H5[RAND], L9H10[RAND], L7H8[RAND], L9H6[RAND], L15H7[RAND], L10H8[RAND], L7H13[RAND], L16H10[RAND], L10H12[RAND], L11H10[RAND], L9H2[RAND], L7H2[RAND], L7H3[RAND], L14H12[RAND]
chal-290	Addition	1	$\times\times$	22.23 $\rightarrow$	25.89	28.2% $\rightarrow$	47.7%	0.0461
Random Heads: L15H10[RAND], L15H6[RAND], L20H9[RAND], L14H3[RAND], L14H10[RAND], L14H2[RAND], L12H13[RAND], L13H13[RAND], L12H0[RAND], L12H12[RAND], L1H5[RAND], L13H12[RAND], L17H3[RAND], L12H1[RAND], L12H6[RAND]

Appendix

Table 7: Detailed results of Random-Activation Sequential Multi-Head Patching using MRR (Margin Recovery Ratio) across 32 SVAMP instances. The Stat column indicates correct (✓) or incorrect (

\times

) answers for CoT (left) and No-CoT (right) prompting. Logit and probability shifts represent values before and after injecting the selected random activations into the No-CoT execution. Higher MRR values indicate greater recovery of the CoT–No-CoT margin difference. The Random Heads sub-row lists the head sets selected under the random-activation control, providing a comparison against POS-guided head selection. The average MRR across all Random-Activation Sequential Multi-Head Patching instances is

-0.0032

, indicating essentially no average recovery of the CoT–No-CoT margin difference.

Selection A	Selection B	Head Jaccard (%)	Head Min-Overlap (%)	Mean Common Heads	Layer Jaccard (%)	Layer Min-Overlap (%)
Seq. Multi-Head (JSD)	Seq. Multi-Head (MRR)	56.4	78.7	8.75	69.9	91.0
Cross Multi-Head (MRR)	Seq. Multi-Head (MRR)	44.2	68.0	7.16	66.9	89.1
Cross Multi-Head (JSD)	Cross Multi-Head (MRR)	41.4	64.6	6.22	64.4	87.4
Cross Multi-Head (JSD)	Seq. Multi-Head (JSD)	41.1	66.2	6.62	62.8	85.1
Cross Multi-Head (JSD)	Seq. Multi-Head (MRR)	37.4	63.1	6.28	59.5	86.1
Cross Multi-Head (MRR)	Seq. Multi-Head (JSD)	34.6	56.8	5.91	60.7	83.9

Appendix

Table 8: Pairwise overlap between head-selection families, averaged over 32 questions. Head Jaccard measures exact

(\ell,h)

agreement, while layer Jaccard measures agreement after collapsing selected heads to their corresponding layers. Head and layer minimum-overlap coverage measure how much of the smaller selected set is covered by the larger selected set. Mean Common Heads reports the average number of exact heads shared by each pair of selection families.

Selection family	Mean $\|H\|$	Median $\|H\|$	Range $\|H\|$	Unique heads	Unique layers	Most frequent heads	Most frequent layers
Seq. Multi-Head (JSD)	12.16	12.0	7–22	94	23	L14H10 (21/32), L22H7 (16/32), L10H6 (15/32)	L14 (104), L16 (55), L13 (54), L12 (34), L15 (31)
Seq. Multi-Head (MRR)	12.44	12.5	7–20	99	22	L14H10 (23/32), L16H7 (16/32), L14H13 (16/32)	L14 (100), L16 (61), L13 (54), L15 (35), L12 (34)
Cross Multi-Head (JSD)	11.16	11.0	5–19	90	21	L14H10 (26/32), L13H3 (18/32), L10H6 (15/32)	L14 (94), L13 (58), L16 (45), L15 (41), L12 (30)
Cross Multi-Head (MRR)	11.41	11.0	6–20	89	19	L14H10 (24/32), L13H3 (20/32), L12H0 (16/32)	L14 (110), L16 (53), L13 (51), L12 (36), L15 (36)
Random-Activation Multi-Head (JSD)	12.16	12.0	7–22	175	23	L12H5 (8/32), L13H1 (8/32), L13H3 (7/32)	L14 (56), L12 (48), L13 (47), L10 (32), L15 (29)
Random-Activation Multi-Head (MRR)	12.44	12.5	7–20	186	23	L13H3 (7/32), L14H12 (7/32), L15H7 (7/32)	L14 (55), L13 (42), L15 (33), L12 (31), L16 (30)

Appendix

Table 9: Descriptive statistics of selected head sets across 32 questions. Unique heads refer to the number of distinct layer–head pairs

(\ell,h)

selected at least once, while Unique layers refer to the number of distinct layers containing selected heads. The most frequent heads and layers columns report the top five entries for each selection family. Head frequencies are shown as the number of questions in which the head appears, whereas layer frequencies count total selected-head occurrences across all questions.

Selection family	Early L0–L7	Middle L8–L15	Late L16–L23	Dominant band	Most frequent layers
Seq. Multi-Head (JSD)	18 (4.6%)	274 (70.4%)	97 (24.9%)	Middle	L14 (104), L16 (55), L13 (54), L12 (34), L15 (31), L10 (25)
Seq. Multi-Head (MRR)	15 (3.8%)	281 (70.6%)	102 (25.6%)	Middle	L14 (100), L16 (61), L13 (54), L15 (35), L12 (34), L10 (28)
Cross Multi-Head (JSD)	12 (3.4%)	268 (75.1%)	77 (21.6%)	Middle	L14 (94), L13 (58), L16 (45), L15 (41), L12 (30), L10 (23)
Cross Multi-Head (MRR)	8 (2.2%)	277 (75.9%)	80 (21.9%)	Middle	L14 (110), L16 (53), L13 (51), L12 (36), L15 (36), L10 (22)
Random-Activation Multi-Head (JSD)	87 (22.4%)	250 (64.3%)	52 (13.4%)	Middle	L14 (56), L12 (48), L13 (47), L10 (32), L15 (29), L16 (26)
Random-Activation Multi-Head (MRR)	100 (25.1%)	238 (59.8%)	60 (15.1%)	Middle	L14 (55), L13 (42), L15 (33), L12 (31), L16 (30), L10 (26)

Appendix

Table 10: Layer-band distribution of selected attention heads across 32 questions. Since the model contains 24 layers indexed from L0 to L23, we divide layers into early (L0–L7), middle (L8–L15), and late (L16–L23) bands. Counts denote selected-head occurrences across all questions, with percentages computed within each selection family.

Finding	Example	Question / Prompt	Original Run	Selected-Head Ablation (JSD)	Random-Head Ablation
CoT final-anchor / termination failure	chal-906	Exemplar: Q: John has 24 cookies. He wants to divide them equally among 6 friends. How many cookies will each friend get? A: John has 24 cookies and divides them equally among 6 friends. 24 / 6 = 4. The answer is 4. Target: Q: Jesse’s room is 2 feet long and 12 feet wide. If each tile is of size 4 square feet, how many tiles does she need to cover the whole floor? A:	Jesse’s room is 2 feet long and 12 feet wide. The total area of the room is 2 * 12 = 24 square feet. Each tile is of size 4 square feet. The number of tiles needed to cover the whole floor is 24 / 4 = 6 tiles. The answer is 6.	Jesse’s room is 2 feet long and 12 feet wide. 2 * 12 = 12 feet. 2 feet * 12 feet = 24 feet. 2 feet * 12 feet = 24 feet. 2 feet * 12 feet = 24 feet. …	Jesse’s room is 2 feet long and 12 feet wide. The total area of the room is 2 * 12 = 24 square feet. Each tile is of size 4 square feet. The number of tiles needed to cover the whole floor is 24 / 4 = 6 tiles. The answer is 6.
Few-shot exemplar contamination	chal-829	Exemplar: Q: John has 24 cookies. He wants to divide them equally among 6 friends. How many cookies will each friend get? A: John has 24 cookies and divides them equally among 6 friends. 24 / 6 = 4. The answer is 4. Target: Q: For the walls of the house he would use 11 nails in all to secure large planks of wood. If each plank needs 3 pieces of nails to be secured and an additional 8 nails were used, how many planks does John need for the house wall? A:	John needs 11 - 8 = 3 nails for the house wall. He needs 3 / 3 = 1 plank. The answer is 1.	John has 24 planks of wood. He wants to divide them equally among 6 friends. If each plank needs 3 pieces of nails to be secured and an additional 8 nails were used, how many planks does John need for the house wall? A: John has 24 planks of wood. He wants to divide them equally among 6 friends. How many planks does John need for the house wall? A: …	John needs 11 nails to secure the large planks of wood. Each plank needs 3 nails to be secured and an additional 8 nails were used. John needs 4 planks for the house wall. The answer is 4.
Reasoning-state / arithmetic trajectory corruption	chal-674	Exemplar: Q: Emily reads 15 pages of a book every day. How many pages will she read in 8 days? A: Emily reads 15 pages every day. In 8 days, 15 * 8 = 120. The answer is 120. Target: Q: The Ferris wheel in paradise park has 18 seats. Each seat can hold 15 people. If 10 seats are broken, how many people can ride the Ferris wheel at the same time? A:	There are 18 - 10 = 8 seats that are not broken. So, 8 x 15 = 120 people can ride the Ferris wheel at the same time. The answer is 120.	The Ferris wheel has 18 - 10 = 5 broken seats. There are 18 - 5 = 15 regular seats. 18 / 15 = 2 / 1. The answer is 2.	There are 18 - 10 = 8 seats that are still open. So, 8 x 15 = 120 people can ride the Ferris wheel at the same time. The answer is 120.

Appendix

Table 11: Representative qualitative examples from CoT zero-ablation experiments. The examples illustrate structural failures observed under selected-head ablation, including final-answer anchor loss, exemplar–target contamination, and reasoning-state corruption.

Finding	Example	Question / Prompt	Original Run	Selected-Head Ablation (JSD)	Random-Head Ablation
No-CoT exemplar-answer copying	chal-569	Exemplar: Q: Every day Ryan spends 5 hours on learning Chinese and some more hours on learning English. If he spends 2 hours more on learning English than on learning Chinese, how many hours does he spend on learning English? A: The answer is 7.0. Target: Q: Rachel had to complete 9 pages of math homework, 11 pages of reading homework and 29 more pages of biology homework. How many pages of math and reading homework did she have to complete? A: The answer is	20.0	7.0	20.0
No-CoT salient operand copying	chal-821	Exemplar: Q: Last week Fred had 33 dollars and Jason had 95 dollars. Over the weekend Fred delivered newspapers earning 16 dollars and washed cars earning 74 dollars. How much money did Fred earn over the weekend? A: The answer is 90.0. Target: Q: Paul had some books. After selling 137 in a garage sale he had 105 left. How many books did he have at the start? A: The answer is	242.0	105.0	10.0
Direct-equation operand copying	chal-163	Equation-only prompt: 28.0 - 15.0 =	28.0 - 15.0 = 13.0	28.0 - 15.0 = 28.0	28.0 - 15.0 = 13.0
Direct-equation numeric-generation corruption	chal-196	Equation-only prompt: 4.0 * 7.0 =	4.0 * 7.0 = 28.0	4.0 * 7.0 = 1980.00000000000000000000	4.0 * 7.0 = 28.0

Appendix

Table 12: Representative qualitative examples from No-CoT and direct-equation zero-ablation experiments. The examples illustrate answer-slot failures under No-CoT prompting and numerical-generation failures under direct-equation prompting after selected-head ablation.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
Jan 12, 2026Zhenghao He, Guangzhi Xiong, Bohan Liu +2LLM Interpretability CoT Reasoning
University of Virginia, USA
When Chain-of-Thought Fails, the Solution Hides in the Hidden States
Apr 25, 2026Houman Mehrafarin, Amit Parekh, Ioannis KonstasTransformer Interpretability CoT Reasoning
Heriot-Watt University, Edinburgh, United Kingdom · University of Edinburgh, Edinburgh, United Kingdom
CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness
Jul 21, 2026Ziming Wang, Yinghua Yao, Changwu Huang +2CoT Faithfulness Faithfulness of Language Model Explanations
Department of Computer Science and Engineering of Southern University of Science and Technology, Shenzhen, China · Center for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore · School of AI and Liberal Arts, Beijing Normal-Hong Kong Baptist University, Zhuhai, China +1

Jan 12, 2026J/K move · Enter open · S save

Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models

Zhenghao He, Guangzhi Xiong, Bohan Liu, Sanchit Sinha+1

University of Virginia, USA

LLM Interpretability CoT Reasoning Causal Interventions in Language Models LLM Reasoning Latent CoT Reasoning

Open paper AskPDF arXiv

Chain-of-Thought (CoT) prompting often improves the reasoning performance of large language models (LLMs), but the internal signal that triggers this behavior remains poorly understood. Leveraging the sparse features captured by Sparse Autoencoders (SAEs), we propose a systematic framework to analyze and intervene on the internal representations of LLMs, identifying a small set of latent features that are linked to reasoning behavior and can be causally tested through targeted intervention. Across multiple model families and reasoning benchmarks, we show that steering one or a small number of reasoning-related latent features can substantially induce reasoning behavior without explicit CoT prompting, achieving accuracy comparable to CoT. We further show that the identified features are not tied to particular wording patterns or verbosity, and confirm their causal role in reasoning through suppression experiments that impair performance even under CoT prompting. These results suggest that CoT prompting activates specific latent features to trigger reasoning, and that targeted intervention on these features offers an alternative pathway to elicit efficient reasoning behavior without explicit CoT prompting. Code is available at https://github.com/Zhenghao-He/LatentCoT.

Abstract

LLM Interpretability

Explore similar work

Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models

When Chain-of-Thought Fails, the Solution Hides in the Hidden States

CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness