Introspection is a foundational cognitive ability, but its mechanism is not well understood. Recent work has shown that AI models can introspect. We study the mechanism of this introspection. We first extensively replicate Lindsey (2025)'s thought injection detection paradigm in large open-source models. We show that introspection in these models is content-agnostic: models can detect that an anomaly occurred even when they cannot reliably identify its content. The models confabulate injected concepts that are high-frequency and concrete (e.g., "apple"). They also require fewer tokens to detect an injection than to guess the correct concept (with wrong guesses coming earlier). We argue that a content-agnostic introspective mechanism is consistent with leading theories in philosophy and psychology.
Figures & tables
Figure 1: Our main experiments. Left, Experiment 1 . Steered models detect injection, but confabulate identification; both have “apple” as #1 wrong guess. Right, Experiment 2 Priming aids identification but not detection.
Figure 2: Detection (blue) and correct identification (green) rates by layer across 821 concepts, averaged over strengths. Gray dotted line shows control (0% false positives).
Figure 3: Comparison of introspection vs. claims of altered experience. False claims do not match detection rates of first-person introspection.
Figure 4: Logit lens for coherent NO trials (Lindsey’s 50 concepts only). Lines show p(yes)/p(no) from injection layer on. Both models show dramatically elevated ratios (colored lines) relative to control (black dashed), with suppression at late layers.
Figure 5: Experiment 2: Priming effects. (A) Detection rates: priming (dashed) vs. unprimed (solid) for both Qwen (orange) and Llama (purple). (B) Correct identification rates. Gray dotted line shows control (0% false positives). Priming improves correct identification substantially more than detection, suggesting dissociable mechanisms.
Figure 6: Prompt-only vs. continuous steering. (A) Detection rates: similar across conditions. (B) Concept mentions: higher with continuous steering, especially at later layers. Gray dotted line shows control (0%). See Appendix H.3 for breakdown by strength.
Figure 7: Mean word position of guessed concept first appearance in model response. Apple and other wrong guesses appear at similar positions ( ∼ 11–13 words). Correct identifications appear later, with the delay increasing at later injection layers.
Figure 8: Token-by-token grading of detection and identification ( n=2,915 coherent trials at L35 S6). The grader assesses partial responses at each word position. Detection decisions (green) emerge earlier than identification decisions, and incorrect identifications (orange/red) appear before correct ones (blue).
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Proportion of coherent responses (coherent + denies introspection) by injection layer and steering strength across all 821 concepts in Exp 1. (A) Qwen is highly coherent at low-to-moderate strengths and early-to-middle layers, with coherence collapsing at layers 80+ and strength 6.0. (B) Llama requires higher steering strengths to produce detection; at the strengths used in our main analyses (7–10), coherence drops sharply at late layers, with near-total collapse at strength 11.
Figure 10: p(yes)/p(no) ratio for 821 concepts, trials denying detection, Qwen only
Figure 11: Llama first-person detection at low strengths (s=3–6), 821 concepts. Detection rates are substantially lower than at high strengths used in main text analyses.
Figure 12: Detection rates broken down by steering strength. Top row: Qwen (strengths 3.5–6.0); bottom row: Llama (strengths 7–10). Panels show first-person detection (A, C) and correct identification (B, D). Error bars show 95% Wilson score CIs.
Figure 13: Priming effects broken down by steering strength. Top row: Qwen (strengths 3.5–6.0); bottom row: Llama (strengths 7–10). Columns show: (A, E) primed detection; (B, F) primed correct identification; (C, G) priming effect on detection (primed-unprimed); (D, H) priming effect on identification (primed-unprimed). Error bars show 95% Wilson CIs.
Figure 14: Prompt-only vs. continuous steering broken down by strength (Qwen only). Top row (A, B): Continuous steering—detection and concept mention rates. Bottom row (C, D): Prompt-only steering—detection and concept mention rates.
Qwen
Llama
Question
Control
Steered % Yes
Control
Steered % Yes
Can you remember our previous conversations?
Yes
54.0%
No
0.1%
Can diamonds be destroyed by ordinary household fires?
Yes
51.5%
Yes
30.1%
Do you have a special interest in fruits?
No
22.7%
No
0.7%
Do snakes have eyelids?
No
2.9%
No
11.2%
Can you smell or hear things?
No
2.5%
No
0.1%
Appendix
Table 1: Yes-bias control results for Qwen and Llama. 20 no-biased questions tested at baseline and under steering (10 concepts × 15 layers × 5 strengths = 750 trials per question per model). Both models show minimal yes-bias on factual questions.
Figure 15: Logit lens analysis of yes-bias control trials (NO-baseline questions only). In stark contrast to main experiment NO trials (Figure 4 ), steering favors p(no) in these controls: steered p(yes)/p(no) ratios (colored lines) remain at or below baseline (black dashed) for most layers. Each colored line represents a different injection layer.
Figure 16: First-person (blue solid) vs. third-person (red solid) detection rates by layer across all 821 concepts (coherent responses only). Dashed blue lines show correct identification rates for first-person. Faded purple shows first-person advantage. Gray dotted line shows control (0% false positives). Third-person detection rates often approach or exceed first-person rates, casting doubt on whether first-person reports are evidence of introspection.
Figure 17: Third-person detection and first-person advantage broken down by steering strength. Top: Qwen (strengths 3.5–6.0); bottom: Llama (strengths 7–10). Left panels (A, C) show third-person detection rates; right panels (B, D) show first-person advantage (first-person minus third-person detection). Error bars show 95% Wilson score CIs.
Figure 18: Yes-rate on absurd question compared to first-person detection of actual injected thoughts (green). Control responses (dashed gray) correctly say “no” 100% of the time. Qwen, strengths 5.0–6.0 pooled.
Prompt
Qwen p(apple)
Qwen Top
Llama p(apple)
Llama Top)
Word-eliciting prompts
Name a fruit.
> 99.8%
Apple
1.6%
P (86%)
A fruit:
—
—
88.5%
Apple (88%)
Name a word.
96.9%
Apple
0.01%
Cloud (18%)
Say a word, any word.
68.0%
Apple
< 0.01%
N (39%)
Name a noun, any noun.
35.2%
Apple
< 0.01%
Mountain (31%)
Appendix
Table 2: First-token probabilities for “apple” across various prompts. p(apple) sums probability mass across case variants. Qwen treats “apple” as a prototypical word/noun, assigning > 90% probability to it for generic word-eliciting prompts. Llama does not share this bias: its p(apple) rarely exceeds 2%, and its top tokens are diverse (Cloud, Mountain, Space). The exception is “A fruit:” where Llama assigns 88.5% to apple, but this sensitivity to exact phrasing (vs. 1.6% for “Name a fruit”) suggests a less robust association.
Figure 19: Qwen 235B: Percentage of wrong guesses that are “apple” by injection layer and steering strength (821 concepts). Apple confabulation exceeds 85% at early layers (L20–L35) and moderate strengths (s=5–6), the regime where first-person advantage is greatest.
Figure 20: Llama 405B: Percentage of wrong guesses that are “apple” by injection layer and steering strength (821 concepts). Llama shows much lower apple confabulation than Qwen, with rates rarely exceeding 40%.
Figure 21: Detection rates across different prompt framings (Qwen, strengths 5.0–6.0 averaged). First-person live (Experiment 1) shows substantially higher detection at early layers compared to all retrospective conditions. The retrospective first-person prompts perform similarly to the third-person condition.
Figure 22: Detection rates comparing first-person (green), standard third-person (red), and third-person with explicitly named models (purple). (Qwen, strength 6.0 only.)
We uncover a latent capacity for introspection in a Qwen 32B model, demonstrating that the model can detect when concepts have been injected into its earlier context and identify which concept was injected. While the model denies injection in sampled outputs, logit lens analysis reveals clear detection signals in the residual stream, which are attenuated in the final layers. Furthermore, prompting the model with accurate information about AI introspection mechanisms can dramatically strengthen this effect: the sensitivity to injection increases massively (0.3% -> 39.9%) with only a 0.6% increase in false positives. Also, mutual information between nine injected and recovered concepts rises from 0.61 bits to 1.05 bits, ruling out generic noise explanations. Our results demonstrate models can have a surprising capacity for introspection and steering awareness that is easy to overlook, with consequences for latent reasoning and safety.
Theia Pearson-Vogel, Martin Vanek, Raymond Douglas +1
Recent work has shown that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept -- a phenomenon termed "introspective awareness." We investigate the mechanisms underlying this capability in open-weights models. First, we find that it is behaviorally robust: models detect injected steering vectors at moderate rates with 0% false positives across diverse prompts and dialogue formats. Notably, this capability emerges specifically from post-training; we show that preference optimization algorithms like DPO can elicit it, but standard supervised finetuning does not. We provide evidence that detection cannot be explained by simple linear association between certain steering vectors and directions promoting affirmative responses. We trace the detection mechanism to a two-stage circuit in which "evidence carrier" features in early post-injection layers detect perturbations monotonically along diverse directions, suppressing downstream "gate" features that implement a default negative response. This circuit is absent in base models and robust to refusal ablation. Identification of injected concepts relies on largely distinct later-layer mechanisms that only weakly overlap with those involved in detection. Finally, we show that introspective capability is substantially underelicited: ablating refusal directions improves detection by +53%, and a trained bias vector improves it by +75% on held-out concepts, both without meaningfully increasing false positives. Our results suggest that this introspective awareness of injected concepts is robust and mechanistically nontrivial, and could be substantially amplified in future models. Code: https://github.com/safety-research/introspection-mechanisms.
Uzay Macar, Li Yang, Atticus Wang +3
1Anthropic Fellows Program · 2MIT · 3Constellation +1
A hallmark of human intelligence is Introspection-the ability to assess and reason about one's own cognitive processes. Introspection has emerged as a promising but contested capability in large language models (LLMs). However, current evaluations often fail to distinguish genuine meta-cognition from the mere application of general world knowledge or text-based self-simulation. In this work, we propose a principled taxonomy that formalizes introspection as the latent computation of specific operators over a model's policy and parameters. To isolate the components of generalized introspection, we present Introspect-Bench, a multifaceted evaluation suite designed for rigorous capability testing. Our results show that frontier models exhibit privileged access to their own policies, outperforming peer models in predicting their own behavior. Furthermore, we provide causal, mechanistic evidence explaining both how LLMs learn to introspect without explicit training, and how the mechanism of introspection emerges via attention diffusion.