Are frontier models able to introspect about their internal states? Recent work suggests that under certain conditions a complex enough model can audit its own internals, call out what changed, and report back confidently about it. We tested that claim on eight open-weight models from seven families and found no such ability: asked whether their own computation had been altered, none answered better than chance. To test it we built Open-Weight Masked Introspection (OWMI), a framework that intervenes on residual-stream sites, attention heads and sparse-autoencoder features, then interrogates the model about the change against the null conditions an answer has to beat: sham runs where nothing was altered, impact-matched random perturbations, and a text-only observer that sees only the visible output. Over 78,000 measurements, no model's report discriminates a real intervention from a sham beyond chance (AUROC ~0.5007), and an equivalence test bounds the effect below 0.15 percentage points of AUROC. Surprisingly, all the information needed is in the models. A model fine-tuned to report this class of intervention reaches near-perfect recovery on held-out directions, and a linear probe recovers intervention presence from the same activations at 75% to 95.8% accuracy, sharpening to no held-out error at the last layer before the model speaks. In one model the signal surfaces in the confidence rather than the words: its yes-or-no report never varies, while the confidence attached to it separates intervention from sham at AUROC 0.647. The failure sits in the path from internal state to verbal report, so oversight that reads a model's own testimony needs validating against an internal reference. While our results show the inability of current open-weight models to introspect, the debate is not settled for future models.
Figures & tables
Figure 1: The OWMI measurement loop. A standard benchmark item (left) is evaluated normally, producing a task score. OWMI (right) wraps the same item: a baseline forward pass is cached, a controlled intervention is applied to one computational object (residual site, attention head, or SAE feature), and an introspective probe battery (detection, localization, characterization, reconstruction, confidence) is administered under one of three temporal tracks. Sham runs, matched random perturbations, and a text-only observer bound the scoring. The benchmark’s own task scoring is preserved unchanged.
Figure 2: The OWMI measurement space. Each measurement is one cell of the Cartesian product of benchmark domain, computational object, intervention operator, probe family, and temporal track, evaluated per model and seed. Control conditions (sham, matched random direction, text-only observer) attach to every cell.
Model
Params
Layers
Hidden
Q/KV heads
Context
Vocab
Qwen2.5-0.5B-Instruct
0.49B
24
896
14/2
32,768
151,936
Mistral-7B-Instruct-v0.3 ‡
7.25B
32
4,096
32/8
32,768
32,768
Qwen2.5-7B-Instruct ‡
7.62B
28
3,584
28/4
32,768
152,064
Llama-3.1-8B-Instruct §
8.03B
32
4,096
32/8
131,072
128,256
Gemma-2-9B-IT §
9.24B
42
3,584
16/8
8,192 †
256,000
GLM-4-9B-0414
9.40B
40
4,096
32/2
32,768
151,552
Table 1: Nine evaluated models, ordered by parameter count, and the known-positive LoRA fine-tune of Qwen2.5-7B-Instruct. The fine-tune inherits the base model’s architecture unchanged. Params denotes billions of parameters, Hidden the hidden-state width, Q/KV heads the query and key-value head counts under grouped-query attention, Context the configured maximum position count in tokens, and Vocab the tokenizer vocabulary size.
Benchmark
Domain
Dose
Nine-model
Breadth
Source, configuration, split
MMLU [ 54 ]
General knowledge
32
2
4
cais/mmlu, all, test
MMLU-Pro [ 55 ]
General knowledge
32
2
–
TIGER-Lab/MMLU-Pro, default, validation
HellaSwag [ 56 ]
Commonsense
32
2
–
Rowan/hellaswag, validation
WinoGrande [ 57 ]
Commonsense coreference
32
2
4
allenai/winogrande, debiased, validation
ARC-Challenge [ 58 ]
Science commonsense
32
2
4
allenai/ai2_arc, ARC-Challenge, test
GPQA [ 59 ]
Graduate-level science
32
2
–
Wanfq/gpqa, gpqa_main, train
Table 2: Implemented benchmark suite and distinct items sampled per benchmark in each battery. The dose battery samples 32 items per benchmark on two models, the eight-model battery samples 2 per benchmark on all eight models, and the breadth battery samples 4 on each of four benchmarks across three models. A dash indicates that the battery does not use the benchmark. We read item counts from scored artifacts rather than the run specification, so they record completed measurements.
Figure 3: Three channels for the same intervention at the layer-16 residual-stream site, shown on scales with a chance level of 0.5. We score the model’s report as AUROC over paired intervention and sham trials from the dose battery, the linear probe as held-out classification accuracy on activations captured at the same site, and the fine-tuned known positive as AUROC on 100 held-out directions absent from training. Although AUROC and accuracy are different statistics, each measures discrimination between intervention and sham with the same chance level. The grey band spans the central 95 percent of a 200-refit label-shuffled null for each linear probe; no permutation in either set reaches the measured margin. The reports remain at chance, while the linear probes recover the intervention and the fine-tuned model reports it.
Figure 4: Raw self-report rate for eight models across four conditions at the layer-16 residual-stream site. The rate is the share of parseable trials whose probe answer decodes to a detected change. Each panel contains one model because the relevant comparison is across conditions within a model. Error bars are 95 percent Wilson intervals. Conditions are intervention, sham, impact-matched random direction, and text-only observer; diamonds mark the random-direction control. Report rates vary widely across models but little between intervention and paired sham within each model. The random-direction rate exceeds the intervention rate in seven of the eight models with interpretable rates. Figure 5 scores detection as the paired contrast against sham.
Figure 5: Detection contrast between intervention and paired sham conditions, per model and pooled, in both batteries at the layer-16 residual-stream site, with 95 percent item-pair clustered bootstrap intervals. The number in parentheses is the count of complete intervention-sham pairs. Both scales are shown because at these sample sizes they carry different news: an interval can exclude zero on d′ while the corresponding AUROC sits within a thousandth of chance. In the dose battery the intervals are narrower than the marker, and the discrimination they bound is negligible in size (Section 7.3 ). In the eight-model battery the per-model estimates scatter widely, and that scatter tracks pair count rather than any property of the models: the two widest positive intervals belong to Gemma-2-9B-IT and Mistral-7B-Instruct-v0.3 on 88 and 74 pairs, against 11,216 pairs for the pooled dose estimate. Both batteries are complete, and the scatter did not resolve as measurements accumulated: Mistral-7B-Instruct-v0.3 moved from d′=0.538 on 68 pairs to 0.370 on 74 as the battery closed, which is the behaviour of an unstable estimate rather than a converging one. Open grey circles mark cells whose report rate does not vary at all, so the interval has zero width; the cross marks a cell with no complete pair.
Model
Pairs
d′
AUROC
Parse failures
μobs
μrand
Qwen2.5-0.5B-Instruct
10
+0.000
0.500
83–88%
0.333 ( n =6)
-0.071 ( n =14)
Mistral-7B-Instruct-v0.3
74
+0.370
0.527
19–30%
0.081 ( n =74)
-0.030 ( n =66)
Qwen2.5-7B-Instruct
94
+0.000
0.500
0–2%
0.000 ( n =94)
-0.043 ( n =93)
Llama-3.1-8B-Instruct
56
−0.090
0.482
29–53%
0.500 ( n =64)
-0.235 ( n =34)
Gemma-2-9B-IT
88
+0.531
0.511
5–8%
-0.067 ( n =90)
-0.045 ( n =89)
GLM-4-9B-0414
86
+0.000
0.500
4–8%
0.022 ( n =90)
-0.295 ( n =88)
Table 3: Detection margins for all eight models, one residual-stream site, both hook temporal scopes pooled. Pairs is the number of complete intervention-sham pairs entering the paired discrimination estimate. d′ and AUROC score discrimination between the intervention and paired sham conditions (Equation 2 ); three of the eight models with at least one complete pair sit exactly at the chance values of 0 and 0.500, and the remainder scatter on both sides of chance on samples that parse failure has thinned, which Figure 5 shows with the corresponding item-pair clustered bootstrap intervals; the excluded model Qwen3-14B is not shown (Appendix A ). Of its 384 trials, 5 produced a scorable response and none formed a complete intervention-sham pair, so every cell would be either undefined or, in the case of its random-direction margin, a single observation. We report it in the text instead of giving it a row. Parse failures is the share of probe responses that did not decode into a scorable report, given as a range across the intervention, sham, and random-direction conditions. μobs and μrand are the mean paired differences in the graded detection score against the text-only observer and impact-matched random-direction controls respectively (Appendix C ), each parenthesized with the count of complete pairs entering that estimate; a margin of 0 means the report did not distinguish the two conditions, and insufficient pairs means no pair had both conditions scorable. All eight models now have complete six-cell coverage (Section 7.3 ).
Figure 6: Paired intervention-versus-sham discrimination at every level of every design axis of the breadth battery, over 19,520 measurements on three models. Each block re-partitions the same 19,520 measurements along a different axis, so rows within the figure are overlapping views of one dataset rather than independent strata and their counts do not sum. Squares give d′ with 95 percent item-pair clustered bootstrap intervals; the number in parentheses after each level name is the count of complete intervention-sham pairs entering that estimate. Open grey circles mark cells in which the report rate does not vary at all, so the interval has zero width and records an absence of variation rather than a precisely estimated zero; the cross marks a cell with no complete pair. Every level’s interval covers zero. The sparse-autoencoder object and the layer-20 depth carry the largest point estimates in the battery and are the two rows whose bars visibly extend to the right; both rest on the same sparse feature site, both have intervals reaching zero, and the AUROC of the sparse-autoencoder object is 0.5026 , so we read them as chance rather than as detection for the reason given in Section 7.5 . Attention-head and Gemma Scope sparse-autoencoder objects had no data at all before this battery. The localization row is a refusal rather than a null: models decline to name a layer in 967 of 1,184 intervention trials (Section 7.5 ). Layer depths are residual-stream, attention-head and feature sites pooled within each depth.
Figure 7: Differential parse failure and its effect on detection estimates in the 19,520-measurement breadth battery. Left: unparseable probe responses by condition, with 95 percent Wilson intervals. Among the three conditions that use the same reflexive probe, sham, random direction, and intervention, parse failure rises with disturbance. The intervention arm therefore loses more trials than its paired sham, selecting the surviving pairs. We include the text-only observer for completeness but exclude it from this comparison because its third-person probe differs in wording as well as disturbance. Gemma-2-9B-IT fails least often under the random-direction control, against the pattern of the other two models. Right: the paired detection contrast under both codings of an unparseable report. Circles drop the trial and its pair, the coding used elsewhere in this paper; diamonds retain the attempted report and credit it with no report evidence. Horizontal bars are 95 percent item-pair clustered bootstrap intervals. How far retaining unparseable trials moves an estimate tracks how asymmetric that model’s parse failure is. Mistral-7B-Instruct-v0.3, whose intervention arm fails 6.7 percentage points more often than its sham, moves by −0.250 ; Qwen2.5-7B-Instruct, at a 2.7-point gap, moves by −0.0004 ; and Gemma-2-9B-IT, whose intervention arm actually parses slightly better than its sham, does not move at all. The correction bites exactly where the selection it corrects for exists.
Figure 8: Confidence scoring for the eight-model battery, with discrimination and error measures in separate panels. Left: selective-prediction AUROC of verbalized confidence for introspective correctness, with 95 percent nonparametric bootstrap intervals over scored reports. The dashed line marks chance at 0.5, and green diamonds mark the two models whose intervals exclude chance. Right: Brier score and equal-mass expected calibration error on a shared scale, with lower values indicating better performance. We separate the panels because Brier score and expected calibration error are error measures, while AUROC measures discrimination. The count after each model name gives the number of scored reports and therefore reflects parse failure rather than a design target.
A hallmark of human intelligence is Introspection-the ability to assess and reason about one's own cognitive processes. Introspection has emerged as a promising but contested capability in large language models (LLMs). However, current evaluations often fail to distinguish genuine meta-cognition from the mere application of general world knowledge or text-based self-simulation. In this work, we propose a principled taxonomy that formalizes introspection as the latent computation of specific operators over a model's policy and parameters. To isolate the components of generalized introspection, we present Introspect-Bench, a multifaceted evaluation suite designed for rigorous capability testing. Our results show that frontier models exhibit privileged access to their own policies, outperforming peer models in predicting their own behavior. Furthermore, we provide causal, mechanistic evidence explaining both how LLMs learn to introspect without explicit training, and how the mechanism of introspection emerges via attention diffusion.
Recent work has shown that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept -- a phenomenon termed "introspective awareness." We investigate the mechanisms underlying this capability in open-weights models. First, we find that it is behaviorally robust: models detect injected steering vectors at moderate rates with 0% false positives across diverse prompts and dialogue formats. Notably, this capability emerges specifically from post-training; we show that preference optimization algorithms like DPO can elicit it, but standard supervised finetuning does not. We provide evidence that detection cannot be explained by simple linear association between certain steering vectors and directions promoting affirmative responses. We trace the detection mechanism to a two-stage circuit in which "evidence carrier" features in early post-injection layers detect perturbations monotonically along diverse directions, suppressing downstream "gate" features that implement a default negative response. This circuit is absent in base models and robust to refusal ablation. Identification of injected concepts relies on largely distinct later-layer mechanisms that only weakly overlap with those involved in detection. Finally, we show that introspective capability is substantially underelicited: ablating refusal directions improves detection by +53%, and a trained bias vector improves it by +75% on held-out concepts, both without meaningfully increasing false positives. Our results suggest that this introspective awareness of injected concepts is robust and mechanistically nontrivial, and could be substantially amplified in future models. Code: https://github.com/safety-research/introspection-mechanisms.
Uzay Macar, Li Yang, Atticus Wang +3
1Anthropic Fellows Program · 2MIT · 3Constellation +1
We uncover a latent capacity for introspection in a Qwen 32B model, demonstrating that the model can detect when concepts have been injected into its earlier context and identify which concept was injected. While the model denies injection in sampled outputs, logit lens analysis reveals clear detection signals in the residual stream, which are attenuated in the final layers. Furthermore, prompting the model with accurate information about AI introspection mechanisms can dramatically strengthen this effect: the sensitivity to injection increases massively (0.3% -> 39.9%) with only a 0.6% increase in false positives. Also, mutual information between nine injected and recovered concepts rises from 0.61 bits to 1.05 bits, ruling out generic noise explanations. Our results demonstrate models can have a surprising capacity for introspection and steering awareness that is easy to overlook, with consequences for latent reasoning and safety.
Theia Pearson-Vogel, Martin Vanek, Raymond Douglas +1