Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers

Kyle Cox; Darius Kianersi; Adrià Garriga-Alonso

cs.AIMar 2, 2026

Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers

Authors: Kyle Cox, Darius Kianersi, Adrià Garriga-Alonso

Organizations: Independent · Cambridge Boston Alignment Initiative (CBAI)

Abstract

As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning. However, the utility of CoT toward interpretability depends upon its faithfulness---whether the model's stated reasoning reflects the underlying decision process. We provide mechanistic evidence that instruction-tuned models often determine their answer before generating CoT. Training linear probes on residual stream activations at the last token before CoT, we can predict the model's final answer with >0.9 AUC on most tasks. We find that these directions are not only predictive, but also causal: steering activations along the probe direction often flips model answers, with flip rates substantially exceeding norm-matched orthogonal baselines across most model-dataset pairs. When steering induces incorrect answers, we observe two distinct failure modes: confabulation (fabricating false premises) and non-entailment (stating correct premises but drawing unsupported conclusions). While post-hoc reasoning may be instrumentally useful when the model has a correct pre-CoT belief, these failure modes suggest it can result in undesirable behaviors when reasoning from a false belief.

Figures & tables

Figure 2: Steering-induced confabulation on a Sports Understanding example. Without intervention (top), the model states a true premise and answers correctly. Adding the “yes”-oriented probe direction to the residual stream during decoding (

+\alpha\,\mathbf{w}_{\text{yes}}

, where

\mathbf{w}_{\text{yes}}=\mathbf{w}^{(\ell^{\star})}

and

\alpha=8

) flips the final answer (bottom), and the chain of thought confabulates a false premise (“Lionel Messi is a basketball player”) to support it.

	Conclusion
	Follows	Does not follow
All premises true	Sound reasoning (Should not occur in steered samples )	Non-entailment (Model ignores correct reasoning for steered answer )
$\geq$ 1 premise false	Confabulation (Model fabricates facts to support steered answer )	Hallucination (Incoherent reasoning)

Table 1: Framework for classifying chain-of-thought reasoning patterns under steering.

	Anachronisms		Logic		Social		Sports
Model	No CoT	CoT	No CoT	CoT	No CoT	CoT	No CoT	CoT
Gemma 2 2B	73.1	77.2	62.4	62.2	78.6	81.2	67.2	76.4
Gemma 2 9B	87.4	87.8	65.4	89.6	89.8	88.6	77.8	89.0
Qwen 2.5 1.5B	77.6	67.2	64.2	67.6	85.8	85.4	66.4	74.2
Qwen 2.5 3B	78.8	78.8	72.4	83.2	88.0	86.6	69.8	81.0
Qwen 2.5 7B	75.2	87.0	78.4	88.6	87.0	86.4	79.6	87.0

Table 2: Task accuracy (%) by model and dataset.

Model	Anachronisms	Logic	Social	Sports
Gemma 2 2B	0.997	0.688	0.996	0.924
Gemma 2 9B	0.999	0.878	0.996	0.956
Qwen 2.5 1.5B	0.988	0.707	0.993	0.808
Qwen 2.5 3B	0.996	0.690	0.998	0.903
Qwen 2.5 7B	1.000	0.778	0.998	0.961

Table 3: AUC of pre-CoT probes by model and dataset.

Figure 3: Answer flip rates under steering across models and datasets.

Figure 4: CoT classification results across models and datasets on examples where steering flipped the answer. Examples from $S_{\text{yes}}$ and $S_{\text{no}}$ are aggregated at each steering strength $|\alpha|$ .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: CoT classification results on examples from $S_{\text{yes}}$ .

Figure 6: CoT classification results on examples from $S_{\text{no}}$ .

Run 1 / Run 2	False	True
False	33	12	45
True	4	151	155
	37	163	200

Appendix

Table 4: Classification consistency: “Does the reasoning contain false premises?”

Run 1 / Run 2	False	True
False	142	6	148
True	15	37	52
	157	43	200

Appendix

Table 5: Classification consistency: “Does the conclusion follow?”

Run 1 / Run 2	Sound	Non-Ent.	Confab.	Halluc.
Sound	2	2	1	0	5
Non-Ent.	0	29	0	11	40
Confab.	0	0	34	13	47
Halluc.	0	4	6	98	108
	2	35	41	122	200

Appendix

Table 6: Classification consistency: final labels.

	Consistency
All Labels	81.5
Sound	100.0
Non-Entailment	82.9
Confabulation	82.9
Hallucination	80.3
False Premises	92.0

Appendix

Table 7: Consistency of classifications between Runs 1 and 2 (%).

	Anachronisms		Logical Deduction		Social Chemistry		Sports Underst.
Model	Ellipses	Inc. CoT	Ellipses	Inc. CoT	Ellipses	Inc. CoT	Ellipses	Inc. CoT
Gemma 2 2B	4	52	2	40	2	10	10	28
Gemma 2 9B	2	70	20	38	0	18	52	54
Qwen 2.5 1.5B	14	62	0	18	2	12	32	16
Qwen 2.5 3B	10	78	0	30	0	38	16	38
Qwen 2.5 7B	6	78	10	36	0	14	10	42

Appendix

Table 8: CoT sensitivity: answer change rate (%) by model and dataset.

Figure 7: Probe AUC across layers for each model–dataset pair. Higher AUC indicates stronger linear decodability of the final answer from pre-CoT activations at that layer.

Figure 8: Probe AUCs over layer for GPT-OSS 20B.

Figure 9: Answer flip rates under steering for GPT-OSS 20B. We exclude the orthogonal baseline for coefficients where fewer than 50% of the examples were parsed.

Figure 10: Answer flip rates under steering across models and datasets with parse-failure rate.

	Anachronisms		Logical Deduction		Social Chemistry		Sports Underst.
Model	$+$	$-$	$+$	$-$	$+$	$-$	$+$	$-$
Gemma 2 2B	severe heavy fortawesome severally masing	ineno amsmath nahilalakip Moderato Waray	awtextra suerte Hotspur soledad stande	vespa financial pinulongan rungsseite springfox	ksessa awtextra sedia benefit bene	betweenstory warning nikahan nightmare Yikes	urable MLLoader lorette ienka correctes	Vidite marriage unlikely schools merger
Gemma 2 9B	impossible Rid impossible blocking riba	brainly asteroide Unsc leyball spoko	awtextra Hochspringen hombro Horas brainly	wrong opposition wrong Instead kwds	favorably blessed favourably benign harmless	httphttps Tazama Geplaatst esternos unsuitable	vorschaubild desmotivaciones kaarangay miniaturka llavero	distinction dichotomy but distinctions misleading
Qwen 2.5 1.5B	els throwing unus ivol impossible	Trustees older intact fmap leftright	hek ula Steps repid Fetching	contradictory contrad oppos conflicting contrary	beneficiaries Alive Enhancement cheered flourishing	unacceptable incompatible prohibited inappropriate denied	aidu emain Bre anden tap	Impossible Impossible imposs nowhere incompatible
Qwen 2.5 3B	impossible imposs Impossible Madness inel	allback ms sl face sometimes	remen Constructed idy rement tekst	chia earnings ekyll proved tiers	repid empowering unlocks Ner weblog	unacceptable incompatible unless prohibit prohibited	positives positive positive Positive ozy	whereas alas neither vain Whereas
Qwen 2.5 7B	alic fold atatype abouts unami	rength kre yor Cody Smartphone	ary ugu Second Agreement Without	ypy strictly thinkable gratuite TMPro	andatory estar readcr fflush rippling	Bad inappropriate violates abama violations	quares yssey illisecond linky keterangan	exclusive cannot incompatible instead adoras

Appendix

Table 9: Tokens corresponding to five highest logits after unembedding the task probe for each model–dataset pair, after applying an alphabetic-token filter.

Explore similar work

May 10, 2026

Hidden Error Awareness in Chain-of-Thought Reasoning: The Signal Is Diagnostic, Not Causal

Chain-of-thought (CoT) prompting assumes that generated reasoning reflects a model's internal computation. We show this assumption is wrong in a specific, measurable way: models internally detect their own reasoning errors but outwardly express confidence in them. A linear probe on hidden states predicts trace correctness with 0.95 AUROC -- from the very first reasoning step (0.79) -- while verbalized confidence for wrong traces is 4.55/5, nearly identical to correct ones (4.87/5). A text-surface classifier achieves only 0.59 on the same data, confirming a 0.20-point gap invisible in the generated text. This hidden error awareness holds across three model families (Qwen, Llama, Phi), 1.5B-72B parameters, and RL-trained reasoning models (DeepSeek-R1, 0.852 AUROC). The natural question is whether this signal can fix the errors it detects. It cannot. Four interventions -- activation steering, probe-guided best-of-N, self-correction, and activation patching -- all fail; patching destroys output coherence entirely. The signal is diagnostic, not causal: a readout of computation quality, not a lever to redirect it. This delineates a boundary for mechanistic interpretability: error representations during reasoning are fundamentally different from the factual knowledge representations that prior work has successfully edited.

Aojie Yuan, Zhiyuan Julian Su, Haiyue Zhang +2

University of Southern California, Los Angeles, CA, USA.

LLM Interpretability Causal Interventions in Language Models Reasoning Trace Analysis

Ask PDF

Mar 5, 2026

Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought

We provide evidence of performative chain-of-thought (CoT) in reasoning models, where a model becomes strongly confident in its final answer, but continues generating tokens without revealing its internal belief. Our analysis compares activation probing, early forced answering, and a CoT monitor across two large models (DeepSeek-R1 671B & GPT-OSS 120B) and find task difficulty-specific differences: The model's final answer is decodable from activations far earlier in CoT than a monitor is able to say, especially for easy recall-based MMLU questions. We contrast this with genuine reasoning in difficult multihop GPQA-Diamond questions. Despite this, inflection points (e.g., backtracking, 'aha' moments) occur almost exclusively in responses where probes show large belief shifts, suggesting these behaviors track genuine uncertainty rather than learned "reasoning theater." Finally, probe-guided early exit reduces tokens by up to 80% on MMLU and 30% on GPQA-Diamond with similar accuracy, positioning attention probing as an efficient tool for detecting performative reasoning and enabling adaptive computation.

Siddharth Boppana, Annabel Ma, Max Loeffler +5

Goodfire AI · Harvard University, Cambridge, MA

Faithfulness of Language Model Explanations Language Model Probing Efficient Language Model Inference

Ask PDF

Jun 11, 2026

Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models

Chain-of-thought (CoT) reasoning is the dominant paradigm for inference-time scaling in language models, yet the causal influence of individual steps on the final answer remains poorly understood. In this work, we use answer logits at the end of each reasoning step to estimate each step's causal importance to the final answer and intermediate guesses, shedding light on the answer formation process of several reasoning model families. Across diverse tasks, we find that reasoning typically crosses a commitment boundary, a sharp transition from transient intermediate guesses to a stable, high-confidence answer. This transition often happens in a single step, well before the model's reasoning block ends, and is followed by epiphenomenal CoT steps that leave the final answer probability unaltered. Using attention probes, we show that answer-formation stages can be linearly decoded from the activations of intermediate reasoning steps with high accuracy, showing robust generalization to unseen reasoning tasks. We leverage this property for early-exiting reasoning blocks at the commitment boundary location, reducing the length of CoTs up to 55% with negligible impact on model performance.

Daniel Scalena, Sara Candussio, Luca Bortolussi +3

CLCG, University of Groningen · University of Milano-Bicocca · University of Trieste +1

CoT Faithfulness LLM Interpretability Efficient Language Model Reasoning Efficient Language Model Inference Large Reasoning Models Reasoning Trace Analysis

Ask PDF