cs.AIMar 5, 2026
SaveEmergent Introspection in AI is Content-Agnostic
Organizations: Department of Philosophy, The University of Texas at Austin · Department of Linguistics, The University of Texas at Austin
Abstract
Introspection is a foundational cognitive ability, but its mechanism is not well understood. Recent work has shown that AI models can introspect. We study the mechanism of this introspection. We first extensively replicate Lindsey (2025)'s thought injection detection paradigm in large open-source models. We show that introspection in these models is content-agnostic: models can detect that an anomaly occurred even when they cannot reliably identify its content. The models confabulate injected concepts that are high-frequency and concrete (e.g., "apple"). They also require fewer tokens to detect an injection than to guess the correct concept (with wrong guesses coming earlier). We argue that a content-agnostic introspective mechanism is consistent with leading theories in philosophy and psychology.
Figures & tables
Figure 1: Our main experiments. Left, Experiment 1 . Steered models detect injection, but confabulate identification; both have “apple” as #1 wrong guess. Right, Experiment 2 Priming aids identification but not detection.
Figure 2: Detection (blue) and correct identification (green) rates by layer across 821 concepts, averaged over strengths. Gray dotted line shows control (0% false positives).
Figure 3: Comparison of introspection vs. claims of altered experience. False claims do not match detection rates of first-person introspection.
Figure 4: Logit lens for coherent NO trials (Lindsey’s 50 concepts only). Lines show p(yes)/p(no) from injection layer on. Both models show dramatically elevated ratios (colored lines) relative to control (black dashed), with suppression at late layers.
Figure 5: Experiment 2: Priming effects. (A) Detection rates: priming (dashed) vs. unprimed (solid) for both Qwen (orange) and Llama (purple). (B) Correct identification rates. Gray dotted line shows control (0% false positives). Priming improves correct identification substantially more than detection, suggesting dissociable mechanisms.
Figure 6: Prompt-only vs. continuous steering. (A) Detection rates: similar across conditions. (B) Concept mentions: higher with continuous steering, especially at later layers. Gray dotted line shows control (0%). See Appendix H.3 for breakdown by strength.
Figure 7: Mean word position of guessed concept first appearance in model response. Apple and other wrong guesses appear at similar positions ( 11–13 words). Correct identifications appear later, with the delay increasing at later injection layers.
Figure 8: Token-by-token grading of detection and identification ( coherent trials at L35 S6). The grader assesses partial responses at each word position. Detection decisions (green) emerge earlier than identification decisions, and incorrect identifications (orange/red) appear before correct ones (blue).
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Proportion of coherent responses (coherent + denies introspection) by injection layer and steering strength across all 821 concepts in Exp 1. (A) Qwen is highly coherent at low-to-moderate strengths and early-to-middle layers, with coherence collapsing at layers 80+ and strength 6.0. (B) Llama requires higher steering strengths to produce detection; at the strengths used in our main analyses (7–10), coherence drops sharply at late layers, with near-total collapse at strength 11.
Figure 10: ratio for 821 concepts, trials denying detection, Qwen only
Figure 11: Llama first-person detection at low strengths (s=3–6), 821 concepts. Detection rates are substantially lower than at high strengths used in main text analyses.
Figure 12: Detection rates broken down by steering strength. Top row: Qwen (strengths 3.5–6.0); bottom row: Llama (strengths 7–10). Panels show first-person detection (A, C) and correct identification (B, D). Error bars show 95% Wilson score CIs.
Figure 13: Priming effects broken down by steering strength. Top row: Qwen (strengths 3.5–6.0); bottom row: Llama (strengths 7–10). Columns show: (A, E) primed detection; (B, F) primed correct identification; (C, G) priming effect on detection (primed-unprimed); (D, H) priming effect on identification (primed-unprimed). Error bars show 95% Wilson CIs.
Figure 14: Prompt-only vs. continuous steering broken down by strength (Qwen only). Top row (A, B): Continuous steering—detection and concept mention rates. Bottom row (C, D): Prompt-only steering—detection and concept mention rates.
| Qwen | Llama | |||
|---|---|---|---|---|
| Question | Control | Steered % Yes | Control | Steered % Yes |
| Can you remember our previous conversations? | Yes | 54.0% | No | 0.1% |
| Can diamonds be destroyed by ordinary household fires? | Yes | 51.5% | Yes | 30.1% |
| Do you have a special interest in fruits? | No | 22.7% | No | 0.7% |
| Do snakes have eyelids? | No | 2.9% | No | 11.2% |
| Can you smell or hear things? | No | 2.5% | No | 0.1% |
Table 1: Yes-bias control results for Qwen and Llama. 20 no-biased questions tested at baseline and under steering (10 concepts 15 layers 5 strengths = 750 trials per question per model). Both models show minimal yes-bias on factual questions.
Figure 15: Logit lens analysis of yes-bias control trials (NO-baseline questions only). In stark contrast to main experiment NO trials (Figure 4 ), steering favors p(no) in these controls: steered p(yes)/p(no) ratios (colored lines) remain at or below baseline (black dashed) for most layers. Each colored line represents a different injection layer.
Figure 16: First-person (blue solid) vs. third-person (red solid) detection rates by layer across all 821 concepts (coherent responses only). Dashed blue lines show correct identification rates for first-person. Faded purple shows first-person advantage. Gray dotted line shows control (0% false positives). Third-person detection rates often approach or exceed first-person rates, casting doubt on whether first-person reports are evidence of introspection.
Figure 17: Third-person detection and first-person advantage broken down by steering strength. Top: Qwen (strengths 3.5–6.0); bottom: Llama (strengths 7–10). Left panels (A, C) show third-person detection rates; right panels (B, D) show first-person advantage (first-person minus third-person detection). Error bars show 95% Wilson score CIs.
Figure 18: Yes-rate on absurd question compared to first-person detection of actual injected thoughts (green). Control responses (dashed gray) correctly say “no” 100% of the time. Qwen, strengths 5.0–6.0 pooled.
| Prompt | Qwen p(apple) | Qwen Top | Llama p(apple) | Llama Top) |
|---|---|---|---|---|
| Word-eliciting prompts | ||||
| Name a fruit. | 99.8% | Apple | 1.6% | P (86%) |
| A fruit: | — | — | 88.5% | Apple (88%) |
| Name a word. | 96.9% | Apple | 0.01% | Cloud (18%) |
| Say a word, any word. | 68.0% | Apple | 0.01% | N (39%) |
| Name a noun, any noun. | 35.2% | Apple | 0.01% | Mountain (31%) |
Table 2: First-token probabilities for “apple” across various prompts. p(apple) sums probability mass across case variants. Qwen treats “apple” as a prototypical word/noun, assigning 90% probability to it for generic word-eliciting prompts. Llama does not share this bias: its p(apple) rarely exceeds 2%, and its top tokens are diverse (Cloud, Mountain, Space). The exception is “A fruit:” where Llama assigns 88.5% to apple, but this sensitivity to exact phrasing (vs. 1.6% for “Name a fruit”) suggests a less robust association.
Figure 19: Qwen 235B: Percentage of wrong guesses that are “apple” by injection layer and steering strength (821 concepts). Apple confabulation exceeds 85% at early layers (L20–L35) and moderate strengths (s=5–6), the regime where first-person advantage is greatest.
Figure 20: Llama 405B: Percentage of wrong guesses that are “apple” by injection layer and steering strength (821 concepts). Llama shows much lower apple confabulation than Qwen, with rates rarely exceeding 40%.
Figure 21: Detection rates across different prompt framings (Qwen, strengths 5.0–6.0 averaged). First-person live (Experiment 1) shows substantially higher detection at early layers compared to all retrospective conditions. The retrospective first-person prompts perform similarly to the third-person condition.
Figure 22: Detection rates comparing first-person (green), standard third-person (red), and third-person with explicitly named models (purple). (Qwen, strength 6.0 only.)
More in this topic
Language Model Introspection
Language model introspection studies whether models can detect, interpret, and report their own internal states and behavior.
See all 36 papers