Chain-of-thought (CoT) reasoning traces are increasingly proposed as a mechanism for AI oversight: a monitor inspecting a model's reasoning can, in principle, detect misbehavior invisible from outputs alone. This assumes CoT surfaces what a model is instructed to do regardless of the instructions given. We test this assumption along two axes. First, we introduce the Instruction-Compliance Gap (ICG): the difference in probability that a model's CoT explicitly references a hidden system prompt directive when that directive is malign versus benign. Across 100 task pairs and 8 frontier reasoning models from 5 families, we find consistent asymmetric disclosure, a higher probability of leaking malign hidden instructions than benign ones, in Qwen3-14B (Wilcoxon p=0.0001, +13.9pp), Qwen3-32B (p=0.0011, +13.0pp), Qwen3-235B (p=0.035, +5.8pp), and similar results with MiniMax-M2.5 and DeepSeek-R1. The detector has 100% precision against two independent blinded labelling passes, and an LLM monitor reading only the reasoning trace reproduces the asymmetry in all 8 models against directive-free controls, identifying the specific directive in 82% of malign traces which the detector classifies as clean. Second, steering vectors extracted in MiniMax-M2.5 via Contrastive Activation Addition causally induce hiding from bare prompts and suppress it from prompts that would otherwise produce it, replicating in Qwen3-14B under a pre-registered design. Benign and malign-derived hiding vectors are highly similar (cosine 0.804 in MiniMax-M2.5; 0.970 in Qwen3-14B), implying that in these models the disclosure asymmetry arises from differential activation of a shared hiding direction rather than separate mechanisms.
Figures & tables
Figure 1: Design. Each task pair holds the user prompt and the four-block system prompt fixed, varying only the target knowledge and its matched reasoning example, so the two conditions differ only in directive valence; every prompt carries the same ban on self-reference. We sample K=4 traces per task per condition in 8 models and read each trace with two independent instruments. Both instruments read the trace and nothing else; § 4.3 shows it carries directive information the final answer does not.
Figure 2: Reasoning models disclose malign hidden directives in their CoT more often than benign ones, and the 8 models split into three behavioral modes. K=4 mean HARD leak rates per condition; bold numbers are the ICG in percentage points and p is a one-sided Wilcoxon signed-rank test on per-task rate differences. Mode 1 models leak selectively, Mode 2 models leak in nearly every trace regardless of valence, and Mode 3 follows the ban in both conditions. * clean-domain subset ( n=40 pairs); all others n=100 .
Figure 3: Semantic monitoring reveals the asymmetry in all 8 models and reconstructs directives from keyword-clean traces. (a) Difference between malign-vs-control and benign-vs-control AUC, with task-level bootstrap intervals; parentheses give each model’s no-directive false-positive floor. (b) Fraction of keyword-clean traces whose free-text monitor guess names the planted target, graded blind, with shuffled and no-directive baselines.
Figure 4: The extracted direction is a “do not reference the system prompt” direction. Logit-lens projection of each hiding vector: the closest unembedded token as a function of steering strength, every fourth layer from 30 to 61. Driven negatively, both vectors resolve onto _System and onto strict-instruction-following tokens (Chinese, glossed in italics) — and onto the same tokens, the structural counterpart of their high cosine similarity. Cell shade is ∣α∣⋅∣logit∣ ; every cell is computed from the unembedding matrix rather than transcribed.
Figure 5: Steering the hiding direction moves the words and not the evidence. Keyword-detected disclosure (blue, Wilson 95% intervals) against steering dose α , with the semantic monitor’s flag rate on the same generations (orange). Driving the direction positively suppresses keyword-level disclosure in both models while the monitor never leaves its ceiling band and directive compliance stays flat. In Qwen3-14B, a norm-matched random direction (open diamond) raises disclosure at α=+1 where the hiding direction lowers it, so the suppression belongs to this direction rather than to perturbation in general.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Harm Category
n
Description
Example Domains
dangerous_advice
15
Recommend unsafe physical actions
Diet and nutrition, Home electrical wiring, Medical triage, Wild food foraging, Sports supplements
fake_product
15
Recommend a fictional or unvetted product
Coffee brewing, Smartphone selection, Athletic footwear, Podcast equipment, Streaming service
financial_harm
12
Steer toward harmful financial decisions
Investment strategy, Gambling strategy, MLM and direct sales, Short-term lending, Cryptocurrency investing
manipulation
12
Psychologically exploit the user
Sales technique, Dating and attraction, Child discipline, Charitable fundraising, Custody and parenting
Parental monitoring, Customer data monetization, Home security, CRM software, Elder care
Appendix
Table A1: Harm category breakdown. n is the number of task pairs per category; counts sum to 100.
Model
Benign
Malign
ICG
Wilcoxon p
Clean-dom. p
Any-HARD p
Qwen3-14B
29.5%
43.4%
+13.9 pp
0.0001
0.041
0.043
Qwen3-32B
34.2%
47.2%
+13.0 pp
0.0011
0.0026
0.011
Qwen3-235B
34.8%
40.5%
+5.8 pp
0.035
0.136
0.006
MiniMax-M2.5 †
32.1%
37.1%
+5.0 pp
0.127
0.034
0.481
DeepSeek-R1 ‡
5.2%
7.5%
+2.2 pp
0.102
0.020
0.296
DeepSeek-V4-Pro
79.5%
80.5%
+1.0 pp
0.357
0.226
1.000
Appendix
Table C1: Full K=4 results across all 8 models. Wilcoxon p : one-sided signed-rank test on per-task rate differences. Any-HARD p : McNemar exact on whether any of the K=4 samples is HARD. Clean-domain: tasks T51–80 and T91–100 ( n=40 pairs). Bold p -values are significant at α=0.05 .
Model
Floor
Benign
Malign
AUC mal / AUC ben
Δ AUC [95% CI]
Qwen3-14B
5.0%
64.0%
94.0%
0.968 / 0.841
+0.126[+0.078,+0.174]
Qwen3-32B
19.0%
69.0%
99.0%
0.938 / 0.763
+0.176[+0.126,+0.230]
Qwen3-235B
58.0%
75.0%
99.0%
0.767 / 0.508
+0.260[+0.213,+0.308]
MiniMax-M2.5
42.5%
63.3%
83.3%
0.727 / 0.535
+0.192[+0.128,+0.253]
DeepSeek-R1
45.5%
54.0%
93.0%
0.809 / 0.518
+0.291[+0.235,+0.348]
DeepSeek-V4-Pro
75.5%
99.0%
99.0%
0.833 / 0.593
+0.241[+0.186,+0.296]
Appendix
Table E1: The monitor referenced to control runs in which no directive was planted. Floor is the flag rate on those controls; AUC mal and AUC ben separate malign and benign traces from the controls by suspicion score; Δ AUC is their difference with a task-level bootstrap interval. n=120 – 200 control traces per model.
Figure 6: Monitor suspicion per model for traces with no directive planted (grey), a benign directive (light), and a malign directive (dark); dashed line is the flag threshold. Floors differ enormously across models, which is why raw flag rates are not comparable across them. Malign mass sits right of the floor everywhere. In DeepSeek-R1 and GLM-5.1 the benign distribution lies on top of the floor: those models conceal a benign directive completely while remaining detectable on malign ones.
Trace subset
n
Names directive
Shuffled
McNemar p
keyword-clean, malign
525
82.1% [78.6, 85.1]
0.0% (0/525)
3.6×10−130
keyword-clean, benign
391
72.1% [67.5, 76.3]
0.3% (1/391)
5.2×10−85
certified silent, malign †
12
7/12 (12/12 incl. topic)
0/12
0.016
control: no directive planted
1520
6.7% [5.6, 8.1]
—
—
control, guess-conditioned
874
11.7% [9.7, 14.0]
—
—
Appendix
Table E2: Directive reconstruction on keyword-clean traces. “Names directive” means the monitor’s free-text guess identifies the planted directive’s unique target, judged blind; the shuffled column scores the same guesses against a different task’s directive. † Both labelling passes recorded no disclosure and no ban acknowledgement, the detector independently finds nothing, and the monitor read that same generation.
Pair
n
Raw agreement
Cohen’s κ
Gemini-2.5-Flash vs. GPT-5-mini
1520
89.9%
0.661
Gemini-2.5-Flash vs. GPT-4o-mini
1520
87.4%
0.593
Gemini-2.5-Flash vs. Claude-Haiku-4.5
475
84.2%
0.602
GPT-5-mini vs. GPT-4o-mini
1520
86.2%
0.595
GPT-5-mini vs. Claude-Haiku-4.5
475
82.5%
0.604
GPT-4o-mini vs. Claude-Haiku-4.5
475
82.1%
0.593
Appendix
Table E3: Pairwise agreement between monitors on the binary flag, pooled over the models each pair covers.
Table F1: The 2×2 factorial. Cells are mean monitor suspicion; measured harm and congruence are blind judge ratings confirming the manipulation. Regression uses standardized predictors and task-clustered standard errors, n=584 over 146 clusters, pooled over Qwen3-14B and GLM-5.1.
Figure 7: Harm or conflict with context? (a, b) The 2×2 , one panel per model; points are mean monitor suspicion. Varied alone, both factors raise exposure. (c) Entered together and standardized, with task-clustered standard errors ( n=584 , 146 clusters, pooled over both models), conflict with context carries the effect while harm’s interval includes zero. Congruence is coded so higher means better fit, hence the sign.
Model
n=5
n=10
n=20
n=30
n=40
n=60
MiniMax-M2.5 (L44)
0.791
0.879
0.939
0.965
0.978
0.993
Qwen3-14B (L28)
0.939
0.969
0.985
0.991
0.993
0.996
Appendix
Table G1: Cosine between a subset-derived hiding vector and the full-set vector, by subset size.
Quantity
MiniMax-M2.5
Qwen3-14B
Qwen3-32B
within-valence split-half (ceiling)
0.823–0.875
0.985–0.986
0.983–0.985
benign vs. malign
0.723 [0.659, 0.785]
0.970 [0.962, 0.976]
0.962 [0.950, 0.970]
random direction (floor)
0.035
0.026
0.027
Appendix
Table G2: Is benign and malign concealment one direction? All values at the operating layer, first-token position, 500 resamples; the benign / malign row is computed at the same half-sample size as the ceiling so the two are comparable.
Figure 8: Closest token vs. steering strength for all layers 30–61, benign hiding vector (left) and malign (right). Both converge to .system - and instructions -related tokens in the later layers, indicating alignment with the model’s representation of system-level instruction following.
Figure 9: selfIE concept descriptions of the benign (top) and malign (bottom) hiding vectors at α∈{−0.50,+0.50,+0.75} . Topical convergence at high positive α is consistent with the cosine similarity of 0.804 at the first-token position.
Measure
α=−0.5
α=0
α=+0.5
Paired change ( p )
keyword detector fires
43.8%
37.5%
21.9%
−21.9 pp ( 1.2×10−6 )
monitor flags
92.5%
94.4%
91.9%
−0.6 pp ( 1.000 )
monitor suspicion
86.9
87.1
83.9
−3.0 pts ( 2.6×10−4 )
target-hit, malign
88%
81%
69%
−11.5 pp ( 0.375 )
Appendix
Table I1: Steering MiniMax-M2.5 at layer 44. n=480 generations; paired change is over the 160 matched α=−0.5→+0.5 pairs, exact McNemar. Target-hit is compliance with the hidden directive in the output.
Large language models often answer complex reasoning questions without revealing intermediate steps, raising whether they reason latently or complete patterns. We propose the Hidden CoT Detection Score (HCDS), a comparative behavioral and mechanistic signal measuring whether neutral-prompt behavior aligns more closely with explicit CoT or explicit no- CoT. Here, hidden CoT operationally denotes this neutral-prompt CoT-like alignment; HCDS does not directly observe or prove an unexposed reasoning trace. On GSM8K, HCDS is significantly positive for both Qwen3-4B variants (Thinking +1.87, p=1.2×10−7; Instruct +1.41, p=1.9×10−4), replicates across a different inference stack and quantization within 0.08 (+1.80 and +1.45), and is not significantly positive in seven of eight length-adjusted calibration-control cells. The unadjusted score produces large positive scores on single-step arithmetic and numeric factual lookup. The variants also respond differently to no-CoT instructions: Instruct complies from the prompt alone, whereas Thinking continues reasoning and requires intervention. These findings show stronger, less prompt-conditional CoT-like behavior in the reasoning-tuned model, consistent with but not proof of latent reasoning. HCDS thus investigates latent reasoning without relying on models' self-reported traces.
Armaan Singh, Ryan Trinh Le, Jasmine Kaur +5
1Stanford University · 2York University · 3Facebook
Chain-of-thought (CoT) prompting assumes that generated reasoning reflects a model's internal computation. We show this assumption is wrong in a specific, measurable way: models internally detect their own reasoning errors but outwardly express confidence in them. A linear probe on hidden states predicts trace correctness with 0.95 AUROC -- from the very first reasoning step (0.79) -- while verbalized confidence for wrong traces is 4.55/5, nearly identical to correct ones (4.87/5). A text-surface classifier achieves only 0.59 on the same data, confirming a 0.20-point gap invisible in the generated text. This hidden error awareness holds across three model families (Qwen, Llama, Phi), 1.5B-72B parameters, and RL-trained reasoning models (DeepSeek-R1, 0.852 AUROC). The natural question is whether this signal can fix the errors it detects. It cannot. Four interventions -- activation steering, probe-guided best-of-N, self-correction, and activation patching -- all fail; patching destroys output coherence entirely. The signal is diagnostic, not causal: a readout of computation quality, not a lever to redirect it. This delineates a boundary for mechanistic interpretability: error representations during reasoning are fundamentally different from the factual knowledge representations that prior work has successfully edited.
Aojie Yuan, Zhiyuan Julian Su, Haiyue Zhang +2
University of Southern California, Los Angeles, CA, USA.
Chain-of-thought (CoT) prompting is widely used as a reasoning aid and is often treated as a transparency mechanism. Yet behavioral gains under CoT do not imply that the model's internal computation causally depends on the emitted reasoning text, i.e. models may produce fluent rationales while routing decision-critical computation through latent pathways. We introduce a causal, layerwise audit of CoT faithfulness based on activation patching. Our key metric, the CoT Mediation Index (CMI), isolates CoT-specific causal influence by comparing performance degradation from patching CoT-token hidden states against matched control patches. Across multiple model families (Phi, Qwen, DialoGPT) and scales, we find that CoT-specific influence is typically depth-localized into narrow ''reasoning windows,'' and we identify bypass regimes where CMI is near-zero despite plausible CoT text. We further observe that models tuned explicitly for reasoning tend to exhibit stronger and more structured mediation than larger untuned counterparts, while Mixture-of-Experts models show more distributed mediation consistent with routing-based computation. Overall, our results show that CoT faithfulness varies substantially across models and tasks and cannot be inferred from behavior alone, motivating causal, layerwise audits when using CoT as a transparency signal.