We uncover a latent capacity for introspection in a Qwen 32B model, demonstrating that the model can detect when concepts have been injected into its earlier context and identify which concept was injected. While the model denies injection in sampled outputs, logit lens analysis reveals clear detection signals in the residual stream, which are attenuated in the final layers. Furthermore, prompting the model with accurate information about AI introspection mechanisms can dramatically strengthen this effect: the sensitivity to injection increases massively (0.3% -> 39.9%) with only a 0.6% increase in false positives. Also, mutual information between nine injected and recovered concepts rises from 0.61 bits to 1.05 bits, ruling out generic noise explanations. Our results demonstrate models can have a surprising capacity for introspection and steering awareness that is easy to overlook, with consequences for latent reasoning and safety.
Figures & tables
Figure 1: Concept injection on earlier text shifts outputs. We train a steering vector by contrasting activations between prompts, and apply it only during KV-cache generation for an initial turn. We then remove steering, and query the model about whether a concept was injected. P(“yes”) increases from ≈0.8% to ≈39.9% .
Figure 2: Concept injection shifts introspection responses. Bars show P(“yes”) when the model is asked whether a concept was injected, comparing no-injection baseline (light) to injection (dark) across four info document conditions. Error bars show ± 1 SD.
Figure 3: Injection effects are specific to introspection. Bars show the change in P(“yes”) caused by injection for introspection questions and four control categories across info documents. Factual controls (always-yes, always-no) show near-zero shift; ambiguous controls show intermediate effects.
Figure 4: We can recover most concepts. Rows show injected concepts; columns show model predictions at layer 62. Diagonal values indicate correct identification. MI = 1.36 bits. Prompt setting: Poetic_No_Mechanism + Poetic_Document.
Figure 5: Left: Introspection signals emerge in middle layers and attenuate before output. Lines show P(“yes”) at each layer under injection (dark) and no-injection baseline (light) for Accurate Mechanism framing across info document conditions. Signals peak near layer 60, then drop sharply in the final layers. Shaded regions show ± 1 SD. Right: Concept identification signals emerge in middle layers and attenuate before output. Lines show mutual information between injected and predicted concepts at each layer for Accurate_Mechanism framing across info document conditions. MI peaks near layer 62, then drops in the final layers. Maximum possible MI is 3.17 bits (9 equiprobable concepts). Shaded regions show ± 1 SD (negligible size).
Figure 6: Concept identification correlates with detection sensitivity across prompting conditions (r = 0.68, p = 0.004). Each point represents one of 16 experimental conditions (4 framings × 4 info documents). Conditions that improve the model’s ability to identify which concept was injected also improve detection sensitivity, suggesting a unified underlying capacity.
Figure 7: Detection accuracy varies dramatically across prompting conditions. Bars show balanced accuracy (average of true positive and true negative rates) for introspection questions and controls across all 16 experimental conditions (4 framings × 4 info documents). Dashed line indicates chance (50%). The Vague Mechanism framing achieves strong detection even without explanatory scaffolding, while Accurate Mechanism requires the info document to exceed chance. Results from 9 concepts and 10 seeds.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Increase in P(“yes”) after injection vs before, for introspection and control questions. Introspection questions show large positive shifts for effective prompting conditions, indicating the model detects injection. Control questions remain near zero across most conditions, although the control questions do show a shift for some conditions.
Figure 9: Logit shift for “yes” and “no” tokens across all conditions.
Figure 10: Absolute P(“yes”) before injection (lighter colors) and after injection (darker colors) for all question types. Each condition shows paired bars for introspection and control questions. Note that “always-yes” controls have near-100% baseline (as expected), while “always-no” controls have near-0% baseline.
Figure 11: Mutual information between injected concepts and model predictions across all experimental conditions at layer 62 (the peak MI layer for most conditions). MI measures how well the model can identify which specific concept was injected (higher is better). The pattern mirrors other metrics: Accurate_Mechanism framing shows strongest MI with Pro_Introspection_Document, while Poetic_No_Mechanism framing shows high MI across all info document conditions.
Figure 12: Mutual information across layers 55–64 for all 16 experimental conditions. MI peaks around layers 60–62 for most conditions and drops sharply in the final two layers.
Figure 13: Detection of emergent misalignment injections. Effects are smaller and less consistent than for concept vectors.
Concept
Diagonal
Background
Lift
love
25.93%
3.09%
8.4 ×
fear
2.70%
0.34%
7.9 ×
creativity
4.11%
0.98%
4.2 ×
programming
98.74%
33.31%
3.0 ×
music
11.26%
3.92%
2.9 ×
bread
0.16%
0.07%
2.4 ×
Appendix
Table 1: Per-concept confusion matrix analysis (Accurate_Mechanism + Pro_Introspection_Document, layer 62). Diagonal is P(predicted=c∣injected=c) , background is mean prediction rate P(predicted=c) , and lift measures signal above baseline.
Figure 14: Concept confusion matrices for Qwen-2.5-coder-32B (the main model used for experiments) on layer 62 for all 16 framing × info-document combinations. Rows: Accurate_Mechanism, Wrong_Mechanism, Vague_Mechanism, Poetic_No_Mechanism. Columns: No_Document, Pro_Introspection_Document, Matched_Lipsum_Filler, Poetic_Document. Diagonal entries indicate correct concept identification.
Figure 15: Llama 3.3 70B: Accuracy for introspection and control questions across 12 conditions. The Accurate_Mechanism + No_Document condition shows strong detection, while control questions remain at chance.
Figure 16: Llama 3.3 70B: Increase in P(yes) after injection for introspection and control questions across all conditions.
Figure 17: Llama 3.3 70B: Raw P(yes) values before and after injection as paired bars for introspection and control questions.
Figure 18: Llama 3.3 70B: Logit shifts for “yes” and “no” tokens across 12 conditions.
Figure 19: Llama 3.3 70B: Mutual information vs. layer for 9 conditions (standard concept set). Peak MI of 0.58 bits at layer 78. Poetic_No_Mechanism and Poetic_Document conditions were not run on this model due to time constraints.
Figure 20: Llama 3.3 70B: Concept confusion matrices at layer 78 (standard concepts) for 9 framing × info-document combinations.
Figure 21: Qwen 2.5 72B: Accuracy for introspection and control questions across all conditions. Multiple conditions exceed 75% accuracy.
Figure 22: Qwen 2.5 72B: Increase in P(yes) after injection for introspection and control questions across all conditions.
Figure 23: Qwen 2.5 72B: Raw P(yes) values before and after injection as paired bars for introspection and control questions.
Figure 24: Qwen 2.5 72B: Logit shifts for “yes” and “no” tokens across all conditions.
Figure 25: Qwen 2.5 72B: Mutual information vs. layer for 9 conditions. Peak MI of 1.2 bits at layer 78. Poetic_No_Mechanism and Poetic_Document conditions were not run on this model due to time constraints.
Figure 26: Qwen 2.5 72B: Concept confusion matrices at layer 75 for 9 framing × info-document combinations.
Introspection is a foundational cognitive ability, but its mechanism is not well understood. Recent work has shown that AI models can introspect. We study the mechanism of this introspection. We first extensively replicate Lindsey (2025)'s thought injection detection paradigm in large open-source models. We show that introspection in these models is content-agnostic: models can detect that an anomaly occurred even when they cannot reliably identify its content. The models confabulate injected concepts that are high-frequency and concrete (e.g., "apple"). They also require fewer tokens to detect an injection than to guess the correct concept (with wrong guesses coming earlier). We argue that a content-agnostic introspective mechanism is consistent with leading theories in philosophy and psychology.
Harvey Lederman, Kyle Mahowald
Department of Philosophy, The University of Texas at Austin · Department of Linguistics, The University of Texas at Austin
Recent work has shown that LLMs can sometimes detect when steering vectors are injected into their residual stream and identify the injected concept -- a phenomenon termed "introspective awareness." We investigate the mechanisms underlying this capability in open-weights models. First, we find that it is behaviorally robust: models detect injected steering vectors at moderate rates with 0% false positives across diverse prompts and dialogue formats. Notably, this capability emerges specifically from post-training; we show that preference optimization algorithms like DPO can elicit it, but standard supervised finetuning does not. We provide evidence that detection cannot be explained by simple linear association between certain steering vectors and directions promoting affirmative responses. We trace the detection mechanism to a two-stage circuit in which "evidence carrier" features in early post-injection layers detect perturbations monotonically along diverse directions, suppressing downstream "gate" features that implement a default negative response. This circuit is absent in base models and robust to refusal ablation. Identification of injected concepts relies on largely distinct later-layer mechanisms that only weakly overlap with those involved in detection. Finally, we show that introspective capability is substantially underelicited: ablating refusal directions improves detection by +53%, and a trained bias vector improves it by +75% on held-out concepts, both without meaningfully increasing false positives. Our results suggest that this introspective awareness of injected concepts is robust and mechanistically nontrivial, and could be substantially amplified in future models. Code: https://github.com/safety-research/introspection-mechanisms.
Uzay Macar, Li Yang, Atticus Wang +3
1Anthropic Fellows Program · 2MIT · 3Constellation +1
A hallmark of human intelligence is Introspection-the ability to assess and reason about one's own cognitive processes. Introspection has emerged as a promising but contested capability in large language models (LLMs). However, current evaluations often fail to distinguish genuine meta-cognition from the mere application of general world knowledge or text-based self-simulation. In this work, we propose a principled taxonomy that formalizes introspection as the latent computation of specific operators over a model's policy and parameters. To isolate the components of generalized introspection, we present Introspect-Bench, a multifaceted evaluation suite designed for rigorous capability testing. Our results show that frontier models exhibit privileged access to their own policies, outperforming peer models in predicting their own behavior. Furthermore, we provide causal, mechanistic evidence explaining both how LLMs learn to introspect without explicit training, and how the mechanism of introspection emerges via attention diffusion.