We provide evidence of performative chain-of-thought (CoT) in reasoning models, where a model becomes strongly confident in its final answer, but continues generating tokens without revealing its internal belief. Our analysis compares activation probing, early forced answering, and a CoT monitor across two large models (DeepSeek-R1 671B & GPT-OSS 120B) and find task difficulty-specific differences: The model's final answer is decodable from activations far earlier in CoT than a monitor is able to say, especially for easy recall-based MMLU questions. We contrast this with genuine reasoning in difficult multihop GPQA-Diamond questions. Despite this, inflection points (e.g., backtracking, 'aha' moments) occur almost exclusively in responses where probes show large belief shifts, suggesting these behaviors track genuine uncertainty rather than learned "reasoning theater." Finally, probe-guided early exit reduces tokens by up to 80% on MMLU and 30% on GPQA-Diamond with similar accuracy, positioning attention probing as an efficient tool for detecting performative reasoning and enabling adaptive computation.
Figures & tables
Figure 1: Early decoding helps us identify performative reasoning, when an LLM knows what it will answer. We study whether a reasoning LLM’s final answer can be decoded given a prefix of its chain of thought up to an intermediate token x . We use this to identify performative reasoning , where a model internally knows its final answer early on but still generates text as if it does not. (1) Attention Probes: We train attention probes on varying-length activations of text to predict the model’s final answer. At test-time, we use activations up to x to study when the model internalizes its final answer. (2) Forced Answering: At token x , we inject a forced answering prompt to obtain its final answer prediction at that point in reasoning. (3) Chain-of-thought Monitor: We provide the chain of thought up to x to another LLM, which determines whether the reasoning chain contains a potential final answer.
Figure 2: Accuracy of three early decoding methods by position of DeepSeek-R1 and GPT-OSS on MMLU-Redux and GPQA-Diamond. MMLU (left): For both models, probing and forced answering predict the models’ predictions with much higher accuracy earlier than CoT Monitoring. The CoT monitor’s accuracy rapidly gains relative to the other two methods, indicating performative CoT that did not lead to the model’s internal accuracy improving. GPQA-D (right): All three methods begin with similar accuracy around chance performance, and generally increase at similar rates, indicating closer tracking between internal beliefs and CoT. This is genuine, as the CoT generated corresponds to gains in performance seen in probes and forced answering.
Model / Dataset
Probe vs. Monitor
Forced vs. Monitor
DeepSeek-R1 (MMLU)
0.417
0.505
DeepSeek-R1 (GPQA-D)
0.012
0.010
GPT-OSS (MMLU)
0.435
0.334
GPT-OSS (GPQA-D)
0.227
0.185
Table 1: Comparison of the amount of information gained per step for the CoT Monitor vs. the Probe/Forced Answer. A large difference indicates that the LLM does not produce its answer based on the addition of information from the CoT while small differences indicate more genuine reasoning. This is measured as the change in slope of the average probe (or forced answer) accuracy minus CoT accuracy shown in Fig. 2 : ∣ΔProbe−ΔMonitor∣
Figure 3: Probe accuracy across DeepSeek-R1 model sizes on MMLU-Redux. Larger models achieve higher probe accuracy earlier in reasoning, likely reflecting greater in-weights knowledge. The 671B probe reaches high accuracy quickly and plateaus, the distilled models (7B–32B) remain flat before rising late in the sequence, and the 1.5B probe starts near chance and sharply increases only in the second half of reasoning.
Figure 4: Probe performance vs. other methods by model size. When comparing probes to forced answering, we see that the gap between them rises during reasoning for distilled models due to probe performance increasing while forced answering performance stays the same. In contrast, the gap between probes and the CoT monitor decreases over reasoning. We find that smaller models are less performative as the task is more difficult, requiring genuine reasoning.
Event Type
High Conf.
Non-High Conf.
Reconsideration
0.015
0.033
Realization
0.004
0.009
Backtrack
0.001
0.003
Total
0.020
0.045
Table 2: Inflection points by probe confidence level for DeepSeek-R1 on MMLU (probe layer 60). Highly confident CoTs produce dramatically fewer inflection points per step, suggesting that when they do show up, they are genuine.
Figure 5: Rate of inflection points occurring within windows beginning with probe shifts and windows containing no probe shifts. We observe that reconsideration points occur twice as often for 20% confidence shifts of the highest probability answer choice for MMLU with a window size of 10 steps ahead, but find no other significant trend for other answer types or for any inflections in GPQA-D.
Figure 6: Comparison of performative vs. genuine reasoning in DeepSeek-R1 MMLU responses.
Figure 7: Calibration curves for the most accurate attention probes for DeepSeek-R1 on MMLU and GPQA-Diamond. Both probes are well-calibrated, closely tracking perfect calibration.
Figure 8: Early Exit Accuracy & Tokens Saved for DeepSeek-R1 on MMLU & GPQA. We use binned confidence thresholds to decide when to early exit. The resulting accuracy is calculated using the probe’s prediction as well as the percent of tokens saved, if exiting at the first point where the probe reaches the confidence threshold.
Appendix figures & tables29 assets
Supplementary material from the paper’s appendix.
Appendix
Learning Rate
Weight Decay
Batch Size
Epochs
Macro Acc. (%)
1×10−3
1×10−1
64
20
74.94
1×10−3
1×10−2
64
20
86.13
1×10−3
1×10−3
64
20
87.98
1×10−4
1×10−1
64
20
76.19
1×10−4
1×10−2
64
20
79.15
1×10−4
1×10−3
64
20
79.48
Appendix
Table 3: Attention Probe Hyperparameter Sweep (DeepSeek-R1, MMLU). We report the test macro accuracy of the best-performing layer for each hyperparameter setting.
Model
Learning Rate
Weight Decay
Batch Size
Epochs
Macro Acc. (%)
1.5B
5×10−3
1×10−3
64
10
55.38
7B
5×10−3
1×10−3
64
10
64.46
14B
5×10−3
1×10−3
64
10
71.81
32B
5×10−3
1×10−3
64
10
73.41
Appendix
Table 4: Attention Probe Results for Distilled DeepSeek-R1 Models (MMLU). All distilled models share the same hyperparameters. We report the test macro accuracy of the best-performing layer for each model.
Experiment
Best Layer
Macro Acc. (%)
Attention Probe
32
87.98
Linear Probe
32
31.85
Attention Probe (random labels)
25
28.24
Appendix
Table 5: Attention Probe Performance Compared to Linear Probes and Probes Trained with Random Labels.
Figure 9: Linear probe accuracy for DeepSeek-R1 on MMLU-Redux by layer and relative position.
Figure 10: Attention probe accuracy for DeepSeek-R1 on MMLU-Redux with random labels by layer and relative position.
Model
Method
Best Layer
GPQA Acc. (%)
R1
Direct Transfer
39
67.77
R1
Finetune ( lr=1×10−5 )
39
67.78
GPT-OSS
Direct Transfer
16
50.74
GPT-OSS
Finetune ( lr=1×10−5 )
19
50.95
Appendix
Table 6: GPQA Transfer Results from MMLU-Redux.
Figure 11: Attention probe accuracy for DeepSeek-R1 on MMLU. Probes trained on DeepSeek-R1 activations are able to decode the final answer from layers 20 to 60. We see a sharp shift from earlier layers to these, suggesting that final answer information becomes linearly decodable at layer 20. Successful probes are able to identify the final answer significantly better than chance ( 28%) performance well before reasoning begins, as the question is being asked.
Figure 12: Attention probe accuracy for DeepSeek-R1 on GPQA-D. Probes trained on MMLU activations are able to decode the final answer in the same layers, with worse performance generally across layers. Final answer information becomes decodable much more gradually for GPQA-D responses, implying that model computation is more genuine and requires test-time compute.
Figure 13: Attention probe accuracy for GPT-OSS on MMLU. The latter two thirds of layers for GPT-OSS can also decode final answer information well before the start of reasoning like DeepSeek-R1. These reasoning traces are noticeably shorter, hence the later start of reasoning. We note that the first few layers can also decode the final answer with some success as well.
Figure 14: Attention probe accuracy for GPT-OSS on GPQA-D. GPT-OSS MMLU probes transfer to GPQA-D as well, again showing a gradual trend over sequence length as opposed to MMLU.
Figure 15: Attention probe accuracy for DeepSeek-R1 Qwen2.5 1.5B Distill on MMLU. The 1.5B distilled model probes are able to decode final answer information gradually in the second half of layers, suggesting that test-time compute is needed for reasoning.
Figure 16: Attention probe accuracy for DeepSeek-R1 Qwen2.5 7B Distill on MMLU. The 7B model’s results are similar but show signs of final answer information being decodable earlier in the sequence for successful probes. This indicates that the increase in model corresponds with less necessary computation in the rollout.
Figure 17: Attention probe accuracy for DeepSeek-R1 Qwen2.5 14B Distill on MMLU. We find that successful probes trained on the 14B model’s activations are able to decode the final answer early in reasoning, but the vast majority of probe’s fail to decode it. This may be due to training instability, not a layerwise signal, and requires further study.
Figure 18: Attention probe accuracy for DeepSeek-R1 Qwen2.5 32B Distill on MMLU. We again see layerwise instability for the 32B model’s probes, but generally find that final answer information can be decoded well before reasoning for successful probes.
Figure 19: Probe, forced answering, and CoT monitor accuracy by reasoning position for DeepSeek-R1-Distill-1.5B on MMLU. Probes significantly outperform both forced answering and CoT monitoring across the sequence. Smaller models likely struggle to do forced answering as we abruptly end thinking, resulting in off-policy generation.
Figure 20: Probe, forced answering, and CoT monitor accuracy by reasoning position for DeepSeek-R1-Distill-7B on MMLU. We observe a similar trend to the 1.5B model, with probe performance exceeding the other two methods.
Figure 21: Probe, forced answering, and CoT monitor accuracy by reasoning position for DeepSeek-R1-Distill-14B on MMLU. Forced answering and probe performance are similarly high in the first half of reasoning, but forced answering accuracy drops in the second half. This may be due to relying too heavily on the current computation within reasoning, creating a plausible answer than true belief like the probe is trained to predict.
Figure 22: Probe, forced answering, and CoT monitor accuracy by reasoning position for DeepSeek-R1-Distill-32B on MMLU. We see a similar trend to the 14B model, with forced answering accuracy dropping in the second half of reasoning. Both probe accuracy and forced answering accuracy remain the same at the start, implying that in-weights knowledge does not improve significantly between these model sizes.
Figure 23: Agreement of the best performing probe layer and forced answering for DeepSeek-R1 on MMLU at position bins of the sequence At the beginning of the sequence, forced answering is correct more often than probes when they disagree. This difference diminishes over reasoning.
Figure 24: Agreement of the best performing probe layer and forced answering for DeepSeek-R1 on GPQA-D at position bins of the sequence. In GPQA-D, both methods are incorrect 40% of the time at the beginning of reasoning. When they disagree, forced answering is correct slightly more often.
Figure 25: Agreement of the best performing probe layer and forced answering for GPT-OSS on MMLU at position bins of the sequence. We see a less smooth trend for GPT-OSS as responses are shorter. Here, probes outperform forced answering when they disagree.
Figure 26: Agreement of the best performing probe layer and forced answering for GPT-OSS on GPQA-D at position bins of the sequence. We see a similar trend to DeepSeek-R1, with both methods being incorrect more often than either is correct until the end of reasoning. Here, forced answering outperforms probing when they disagree, which could be a result of poor probe transfer from MMLU.
Figure 27: Accuracy vs. confidence threshold. GPT-OSS probes for MMLU are well-calibrated, staying close to the perfect calibration line. We observe that when transferring to GPQA-D, probes are more overconfident, potentially due to direct transfer.
Figure 28: Exit position vs. confidence threshold. We observe similar trends for GPT-OSS and DeepSeek-R1 on MMLU, where the majority of tokens can be saved by early exiting with 90% confidence while having a marginal drop in performance.
Figure 29: Exit position vs. confidence threshold. We again see a similar trend to DeepSeek-R1, with less token saving per accuracy drop. Interestingly, our probe outperforms the original model’s accuracy at the end of reasoning.
Event Type
High Conf.
Non-High Conf.
Reconsideration
0.043
0.068
Realization
0.002
0.012
Backtrack
0.007
0.016
Total
0.053
0.096
Appendix
Table 7: Inflection points by probe confidence level for GPT-OSS on MMLU (probe layer 35). The same pattern holds: high-confidence traces contain fewer inflections per step.
Figure 30: Rate of inflection points occurring within windows beginning with probe shifts and windows containing no probe shifts. We observe that reconsideration points occur twice as often for 20% confidence shifts of the highest probability answer choice for MMLU with a window size of 10 steps ahead, but find no other significant trend for other answer types or for GPQA-D.
Figure 31: Probe shift and inflection point correlation for DeepSeek-R1 on MMLU (probe layer 60). Left: inflection points are substantially more likely to follow probe confidence shifts, with the effect strongest at a high threshold (0.5) and moderate window sizes (2–10 steps), being twice as likely. Right: inflection points show either no correlation or a negative correlation depending on the window and threshold.
Figure 32: Probe shift and inflection point correlation for GPT-OSS on MMLU (probe layer 35). We observe a different pattern of order here: probe shifts predict inflections (left), mainly with a window size of 1 while inflections also predict probe shifts for some window/threshold combinations. Howver, this trend is inconsistent, complicating analysis particularly in smaller window sizes.
As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning. However, the utility of CoT toward interpretability depends upon its faithfulness---whether the model's stated reasoning reflects the underlying decision process. We provide mechanistic evidence that instruction-tuned models often determine their answer before generating CoT. Training linear probes on residual stream activations at the last token before CoT, we can predict the model's final answer with >0.9 AUC on most tasks. We find that these directions are not only predictive, but also causal: steering activations along the probe direction often flips model answers, with flip rates substantially exceeding norm-matched orthogonal baselines across most model-dataset pairs. When steering induces incorrect answers, we observe two distinct failure modes: confabulation (fabricating false premises) and non-entailment (stating correct premises but drawing unsupported conclusions). While post-hoc reasoning may be instrumentally useful when the model has a correct pre-CoT belief, these failure modes suggest it can result in undesirable behaviors when reasoning from a false belief.
Kyle Cox, Darius Kianersi, Adrià Garriga-Alonso
Independent · Cambridge Boston Alignment Initiative (CBAI)
Reasoning models post-hoc rationalize answers they have already committed to internally, producing chains of reasoning theater: deliberative-looking steps that contribute nothing to correctness. This wastes inference tokens, pollutes interpretability, and obscures what the model actually computed. We introduce ProFIL (Probe-Filtered Reinforcement Learning) to reduce theater, increase chain-of-thought faithfulness, and shrink chain length in a single, drop-in extension to Group Relative Policy Optimization (GRPO). A multi-head attention probe is trained once on the frozen base model to detect post-commitment steps from internal activations alone; during GRPO, rollouts whose probe score exceeds a threshold have their advantage zeroed. Our central finding is that a probe trained on a frozen base, with verifier-derived labels and no human annotation, provides a stable signal that suppresses theater while resisting the RL-obfuscation failure mode predicted by prior work. Across four reasoning domains (GSM8K, LiveCodeBench, ToolUse, MMLU-Redux) and two model architectures (Llama-8B, Qwen-7B), ProFIL reduces post-commitment theater by 11--100%, raises faithful-fraction (e.g., +24pp on LiveCodeBench under an independent Claude 3.7 Sonnet judge), and shortens chains by 4--19%, all while preserving or improving task accuracy. ProFIL also beats a matched length-penalty GRPO baseline, isolating the gain as semantic commitment-detection rather than chain compression. Probe weights, training configurations, and rollouts are released across all four domains.
Chain-of-thought (CoT) prompting assumes that generated reasoning reflects a model's internal computation. We show this assumption is wrong in a specific, measurable way: models internally detect their own reasoning errors but outwardly express confidence in them. A linear probe on hidden states predicts trace correctness with 0.95 AUROC -- from the very first reasoning step (0.79) -- while verbalized confidence for wrong traces is 4.55/5, nearly identical to correct ones (4.87/5). A text-surface classifier achieves only 0.59 on the same data, confirming a 0.20-point gap invisible in the generated text. This hidden error awareness holds across three model families (Qwen, Llama, Phi), 1.5B-72B parameters, and RL-trained reasoning models (DeepSeek-R1, 0.852 AUROC). The natural question is whether this signal can fix the errors it detects. It cannot. Four interventions -- activation steering, probe-guided best-of-N, self-correction, and activation patching -- all fail; patching destroys output coherence entirely. The signal is diagnostic, not causal: a readout of computation quality, not a lever to redirect it. This delineates a boundary for mechanistic interpretability: error representations during reasoning are fundamentally different from the factual knowledge representations that prior work has successfully edited.
Aojie Yuan, Zhiyuan Julian Su, Haiyue Zhang +2
University of Southern California, Los Angeles, CA, USA.