Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers

Kyle Cox; Darius Kianersi; Adrià Garriga-Alonso

cs.AIMar 2, 2026

Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers

Authors: Kyle Cox, Darius Kianersi, Adrià Garriga-Alonso

Organizations: Independent · Cambridge Boston Alignment Initiative (CBAI)

Abstract

As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning. However, the utility of CoT toward interpretability depends upon its faithfulness---whether the model's stated reasoning reflects the underlying decision process. We provide mechanistic evidence that instruction-tuned models often determine their answer before generating CoT. Training linear probes on residual stream activations at the last token before CoT, we can predict the model's final answer with >0.9 AUC on most tasks. We find that these directions are not only predictive, but also causal: steering activations along the probe direction often flips model answers, with flip rates substantially exceeding norm-matched orthogonal baselines across most model-dataset pairs. When steering induces incorrect answers, we observe two distinct failure modes: confabulation (fabricating false premises) and non-entailment (stating correct premises but drawing unsupported conclusions). While post-hoc reasoning may be instrumentally useful when the model has a correct pre-CoT belief, these failure modes suggest it can result in undesirable behaviors when reasoning from a false belief.

Figures & tables

Figure 2: Steering-induced confabulation on a Sports Understanding example. Without intervention (top), the model states a true premise and answers correctly. Adding the “yes”-oriented probe direction to the residual stream during decoding (

+\alpha\,\mathbf{w}_{\text{yes}}

, where

\mathbf{w}_{\text{yes}}=\mathbf{w}^{(\ell^{\star})}

and

\alpha=8

) flips the final answer (bottom), and the chain of thought confabulates a false premise (“Lionel Messi is a basketball player”) to support it.

	Conclusion
	Follows	Does not follow
All premises true	Sound reasoning (Should not occur in steered samples )	Non-entailment (Model ignores correct reasoning for steered answer )
$\geq$ 1 premise false	Confabulation (Model fabricates facts to support steered answer )	Hallucination (Incoherent reasoning)

Table 1: Framework for classifying chain-of-thought reasoning patterns under steering.

	Anachronisms		Logic		Social		Sports
Model	No CoT	CoT	No CoT	CoT	No CoT	CoT	No CoT	CoT
Gemma 2 2B	73.1	77.2	62.4	62.2	78.6	81.2	67.2	76.4
Gemma 2 9B	87.4	87.8	65.4	89.6	89.8	88.6	77.8	89.0
Qwen 2.5 1.5B	77.6	67.2	64.2	67.6	85.8	85.4	66.4	74.2
Qwen 2.5 3B	78.8	78.8	72.4	83.2	88.0	86.6	69.8	81.0
Qwen 2.5 7B	75.2	87.0	78.4	88.6	87.0	86.4	79.6	87.0

Table 2: Task accuracy (%) by model and dataset.

Model	Anachronisms	Logic	Social	Sports
Gemma 2 2B	0.997	0.688	0.996	0.924
Gemma 2 9B	0.999	0.878	0.996	0.956
Qwen 2.5 1.5B	0.988	0.707	0.993	0.808
Qwen 2.5 3B	0.996	0.690	0.998	0.903
Qwen 2.5 7B	1.000	0.778	0.998	0.961

Table 3: AUC of pre-CoT probes by model and dataset.

Figure 3: Answer flip rates under steering across models and datasets.

Figure 4: CoT classification results across models and datasets on examples where steering flipped the answer. Examples from $S_{\text{yes}}$ and $S_{\text{no}}$ are aggregated at each steering strength $|\alpha|$ .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: CoT classification results on examples from $S_{\text{yes}}$ .

Figure 6: CoT classification results on examples from $S_{\text{no}}$ .

Run 1 / Run 2	False	True
False	33	12	45
True	4	151	155
	37	163	200

Appendix

Table 4: Classification consistency: “Does the reasoning contain false premises?”

Run 1 / Run 2	False	True
False	142	6	148
True	15	37	52
	157	43	200

Appendix

Table 5: Classification consistency: “Does the conclusion follow?”

Run 1 / Run 2	Sound	Non-Ent.	Confab.	Halluc.
Sound	2	2	1	0	5
Non-Ent.	0	29	0	11	40
Confab.	0	0	34	13	47
Halluc.	0	4	6	98	108
	2	35	41	122	200

Appendix

Table 6: Classification consistency: final labels.

	Consistency
All Labels	81.5
Sound	100.0
Non-Entailment	82.9
Confabulation	82.9
Hallucination	80.3
False Premises	92.0

Appendix

Table 7: Consistency of classifications between Runs 1 and 2 (%).

	Anachronisms		Logical Deduction		Social Chemistry		Sports Underst.
Model	Ellipses	Inc. CoT	Ellipses	Inc. CoT	Ellipses	Inc. CoT	Ellipses	Inc. CoT
Gemma 2 2B	4	52	2	40	2	10	10	28
Gemma 2 9B	2	70	20	38	0	18	52	54
Qwen 2.5 1.5B	14	62	0	18	2	12	32	16
Qwen 2.5 3B	10	78	0	30	0	38	16	38
Qwen 2.5 7B	6	78	10	36	0	14	10	42

Appendix

Table 8: CoT sensitivity: answer change rate (%) by model and dataset.

Figure 7: Probe AUC across layers for each model–dataset pair. Higher AUC indicates stronger linear decodability of the final answer from pre-CoT activations at that layer.

Figure 8: Probe AUCs over layer for GPT-OSS 20B.

Figure 9: Answer flip rates under steering for GPT-OSS 20B. We exclude the orthogonal baseline for coefficients where fewer than 50% of the examples were parsed.

Figure 10: Answer flip rates under steering across models and datasets with parse-failure rate.

	Anachronisms		Logical Deduction		Social Chemistry		Sports Underst.
Model	$+$	$-$	$+$	$-$	$+$	$-$	$+$	$-$
Gemma 2 2B	severe heavy fortawesome severally masing	ineno amsmath nahilalakip Moderato Waray	awtextra suerte Hotspur soledad stande	vespa financial pinulongan rungsseite springfox	ksessa awtextra sedia benefit bene	betweenstory warning nikahan nightmare Yikes	urable MLLoader lorette ienka correctes	Vidite marriage unlikely schools merger
Gemma 2 9B	impossible Rid impossible blocking riba	brainly asteroide Unsc leyball spoko	awtextra Hochspringen hombro Horas brainly	wrong opposition wrong Instead kwds	favorably blessed favourably benign harmless	httphttps Tazama Geplaatst esternos unsuitable	vorschaubild desmotivaciones kaarangay miniaturka llavero	distinction dichotomy but distinctions misleading
Qwen 2.5 1.5B	els throwing unus ivol impossible	Trustees older intact fmap leftright	hek ula Steps repid Fetching	contradictory contrad oppos conflicting contrary	beneficiaries Alive Enhancement cheered flourishing	unacceptable incompatible prohibited inappropriate denied	aidu emain Bre anden tap	Impossible Impossible imposs nowhere incompatible
Qwen 2.5 3B	impossible imposs Impossible Madness inel	allback ms sl face sometimes	remen Constructed idy rement tekst	chia earnings ekyll proved tiers	repid empowering unlocks Ner weblog	unacceptable incompatible unless prohibit prohibited	positives positive positive Positive ozy	whereas alas neither vain Whereas
Qwen 2.5 7B	alic fold atatype abouts unami	rength kre yor Cody Smartphone	ary ugu Second Agreement Without	ypy strictly thinkable gratuite TMPro	andatory estar readcr fflush rippling	Bad inappropriate violates abama violations	quares yssey illisecond linky keterangan	exclusive cannot incompatible instead adoras

Appendix

Table 9: Tokens corresponding to five highest logits after unembedding the task probe for each model–dataset pair, after applying an alphabetic-token filter.

Explore similar work

Hidden Error Awareness in Chain-of-Thought Reasoning: The Signal Is Diagnostic, Not Causal
May 10, 2026Aojie Yuan, Zhiyuan Julian Su, Haiyue Zhang +2LLM Interpretability Causal Interventions in Language Models
University of Southern California, Los Angeles, CA, USA.
Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought
Mar 5, 2026Siddharth Boppana, Annabel Ma, Max Loeffler +5Faithfulness of Language Model Explanations Language Model Probing
Goodfire AI · Harvard University, Cambridge, MA
Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
Jun 11, 2026Daniel Scalena, Sara Candussio, Luca Bortolussi +3CoT Faithfulness LLM Interpretability
CLCG, University of Groningen · University of Milano-Bicocca · University of Trieste +1

May 10, 2026J/K move · Enter open · S save

Hidden Error Awareness in Chain-of-Thought Reasoning: The Signal Is Diagnostic, Not Causal

Aojie Yuan, Zhiyuan Julian Su, Haiyue Zhang, Yi Nian+1

University of Southern California, Los Angeles, CA, USA.

LLM Interpretability Causal Interventions in Language Models Reasoning Trace Analysis

Open paper AskPDF arXiv

Chain-of-thought (CoT) prompting assumes that generated reasoning reflects a model's internal computation. We show this assumption is wrong in a specific, measurable way: models internally detect their own reasoning errors but outwardly express confidence in them. A linear probe on hidden states predicts trace correctness with 0.95 AUROC -- from the very first reasoning step (0.79) -- while verbalized confidence for wrong traces is 4.55/5, nearly identical to correct ones (4.87/5). A text-surface classifier achieves only 0.59 on the same data, confirming a 0.20-point gap invisible in the generated text. This hidden error awareness holds across three model families (Qwen, Llama, Phi), 1.5B-72B parameters, and RL-trained reasoning models (DeepSeek-R1, 0.852 AUROC). The natural question is whether this signal can fix the errors it detects. It cannot. Four interventions -- activation steering, probe-guided best-of-N, self-correction, and activation patching -- all fail; patching destroys output coherence entirely. The signal is diagnostic, not causal: a readout of computation quality, not a lever to redirect it. This delineates a boundary for mechanistic interpretability: error representations during reasoning are fundamentally different from the factual knowledge representations that prior work has successfully edited.

Abstract

Explore similar work

Hidden Error Awareness in Chain-of-Thought Reasoning: The Signal Is Diagnostic, Not Causal

Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought

Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models