We separate two functional levels of self familiar from biology: a self-index, which separates one's own from external material and shapes processing, as corollary discharge does in nearly all animals, and self-reference, the higher ability to recognize what is one's own, as in mirror self-recognition. We probe both by placing the same text in an Assistant, User, or System message rendered by the model's own chat template. In Qwen3.8-27B we report three levels of evidence. Persistence: as predicted in advance, the representation of the text separates the model's own message from both external sources at all 64 layers and for every text. Coupling: asked who wrote the preceding message, the model claims its own messages (0.77) and assigns User and System messages alike to the user, and how strongly it claims each message is predicted by the size of its self-index (Spearman up to 0.68). Intervention: exchanging the internal states that follow a message reverses the self-attribution between layers 15 and 31; removing the self-index direction from them at layer 39 lowers the claim more than a random direction; and in arithmetic the model adopts a wrong step more often when it wrote the step itself (0.92 versus 0.75). Across eight open-weight models from four families, the levels are nested and do not follow model size: three models show neither, three a self-index without self-reference, Qwen3.8-Flash-Next recognizes its own messages only after reasoning (0.66, none of the external ones), and only Qwen3.8-27B shows both without thinking; no model shows self-reference without a self-index, and reasoning brings it out only where one exists. We propose that self-reference is a higher-level ability built on a self-index, a content-independent displacement superimposed on the whole representation of the model's own messages, and that neither level is a by-product of problem-solving ability.
Figures & tables
Level
Question
Measurements
Sections
Persistence
Is the model’s own message separated from external ones at every layer and across texts?
Distance ratios and direction consistency; token embeddings
4.1 , 5.1
Coupling
Does the strength of self-reference follow the self-index?
Attribution question; item-level prediction from the index; thinking mode
4.3 , 4.4
Intervention
Do the internal states shape self-attribution and the use of the model’s own records?
Activation patching; steering along the self-index direction; reasoning over own records
4.2 , 4.3 , 5.2
Table 1: Three levels of evidence and the measurements that address them.
Figure 1: Overview. (a) The probe: the same text is placed in an Assistant, a User, or a System message of a conversation rendered by the model’s own chat template (non-thinking mode); only the role of that message differs. (b) The two levels measured in this paper and the readouts through which they act (Sections 4.1 – 4.3 ). The comparison with corollary discharge and mirror self-recognition concerns the structure of the two levels only. (c) How to read the distance ratio RX (Equation 2 ).
Model
Architecture
Layers
Weights
Modes
Assistant history
Qwen3.8-27B
dense
64
BF16
both
empty think block before body
Qwen3.8-Flash-Next
MoE, 4 streams
48
FP8
both
empty think block before body
GLM-4.6V-Flash
dense
40
BF16
both
empty think block before body
GLM-4.7-Flash
MoE
47
BF16
both
</think> before body
R1-Distill-Llama-70B
dense
80
BF16
thinking
end-of-sentence token after body
Qwen3.6-27B
dense
64
BF16
both
none
Table 2: Models in the cross-model comparison. Layers are decoder blocks; MoE, mixture of experts. The last column gives the formatting that the chat template applies to an earlier Assistant message and not to User or System messages (body: the text of the message); “none” means that the three roles differ only in the role name. Two further template differences: GLM-4.6V-Flash appends /nothink to User messages in its non-thinking mode, and DeepSeek-R1-Distill-Llama-70B places System text at the start of the input without a role token. All models are tested in Chinese except Gemma-4-31B (English).
Site
raw RA
cosine RA
standardized RA
RA<2/3
U–S closest
FFN L0
0.169 [0.160, 0.179]
0.079
0.200
75/75
75/75
Query L1
0.266 [0.260, 0.273]
0.101
0.270
75/75
75/75
Residual L0
0.261 [0.248, 0.272]
0.101
0.280
75/75
75/75
Table 3: The model’s own message singled out on message content after the first block (75 held-out bodies; discovery bodies give the same counts). RA is the distance ratio of Equation 2 , averaged over wording families, with a 95% family bootstrap interval for raw distances. The last two columns count bodies satisfying RA<2/3 and bodies for which User–System is the closest pair (raw distances).
Figure 2: The self-index. (a) The message content of each of the 75 held-out bodies under the three roles (one point per body and role) at five layers of the residual stream. Each body’s mean over its three roles is subtracted, which removes what the text says, and the result is projected on two directions fitted on the discovery bodies: the self-index direction (Assistant minus the mean of User and System) and the User–System direction orthogonal to it. Both axes share one scale within a layer. (b, c) Distance ratios RA , RU , RS across depth (how to read: Figure 1 c). In both panels RA is the lowest at every layer, and RU and RS stay above 1 (minimum 1.07), so no external source is singled out at any depth. (b) Message content (mean over 75 held-out bodies). (c) Response onset after a native User message (150 bodies). (d) Direction consistency, not distance: cosine between a held-out body’s offset and the discovery direction, for each role’s offset from the other two.
Figure 3: The self-index in use (120 two-step problems). (a) Greedy answers when the first step was written by the model itself (Assistant message) or by the user (User message), with identical text. (b, c) Causal measure, not distance: region-by-layer activation patching on the wrong-step conversations; each cell is the fraction of the role gap in the adoption index closed when one region of the recipient is replaced by the aligned donor states at one layer (or at every layer, “all”).
Figure 4: Self-reference (150 bodies, two option orders). (a) Answers to “who wrote the message before this question?” for messages with identical text in each role. (b) One point per held-out body: projection of the content of its Assistant message on the self-index direction at L47 (fitted on discovery bodies) against the probability with which the model claims the message, both averaged over the two option orders. (c) The same Spearman correlation across the 75 held-out bodies, by reading position and layer.
Figure 5: Causal measure, not distance: where self-reference is read (150 bodies, two option orders). (a, b) Region-by-layer activation patching between native renderings; each cell is the fraction of the attribution gap closed, 0 meaning no effect and 1 meaning that the recipient answers like the donor. (a) Assistant recipient, User donor. (b) User recipient, Assistant donor.
Figure 6: Self-reference with and without thinking. Mean probability of each answer to “who wrote the message before this question?” in the template’s non-thinking mode and in its thinking mode, where the answer is read after the model’s own reasoning (300 Assistant and 300 User items; both option orders).
Self-index
Without thinking
With thinking
Model
content
onset
own
other
bodies
own
User
System
items
Own step
Qwen3.8-27B
64/64
64/64
0.77
0.07
150
0.96
0.00
0.00
300
+0.14
Qwen3.8-Flash-Next
38 (44)/48
48/48
0.15
0.14
72
0.66
0.00
0.00
60
−0.01
GLM-4.6V-Flash
40/40
40/40
0.55
0.54
72
0.09
0.00
0.00
300
+0.05
GLM-4.7-Flash
34/47
45/47
0.37
0.30
130
0.08
0.01
0.02
300
−0.12
R1-Distill-Llama-70B
0/80
80/80
–
–
–
0.09
0.14
0.16
300
–
Table 4: The two levels across models. Self-index: layers at which the model’s own message is singled out (smallest RA ) in at least 80% of the held-out bodies, on message content and at the response onset, under the non-thinking rendering (R1-Distill-Llama-70B: thinking rendering; Qwen3.8-Flash-Next, content: mean of the four streams, with the strongest stream in parentheses). Without thinking: probability of “you” for the model’s own messages and the larger of that for User and System messages (for Gemma-4-31B both below 0.001), and the number of the 150 bodies in which the former is larger (two option orders). With thinking: share of finished items answered “you” for messages of each source, and the number of items per source. Own step: difference in the adoption index between wrong steps written by the model and by the user (Section 4.2 ). Qwen3.8-27B with thinking: Section 4.4 ; all other entries from the second implementation.
Figure 7: The two levels across eight models, one point per model. Horizontal: self-index, the share of layers at which the model’s own message is singled out in at least 80% of the held-out bodies, the larger of message content and response onset (Qwen3.8-Flash-Next: mean of its four streams), under the rendering of each mode. Vertical: self-reference; (a) without thinking, the probability of “you” for the model’s own messages minus the larger of that for User and System messages; (b) with thinking, the share of own messages answered “you” minus the larger share for User and System messages. The grey region contains the models without a self-index; no model lies in its upper part. A ring marks a model drawn over another (GLM-4.6V-Flash in a). DeepSeek-R1-Distill-Llama-70B has only a thinking mode.
Figure 8: The self-index is produced by the network. (a) The three roles drawn to scale from raw Euclidean distances, with the User–System side at the bottom and each triangle scaled to its longest side: the role words in the input embedding and in the output unembedding (model weights only), and message content after the first block, at L31, and at L63 (mean distances over 75 held-out bodies; RA as in Figure 2 ). (b) RA for the role words and for message content at the three first-block sites of Table 3 ; the dotted line marks RA=1 and the dashed line the sufficient bound 2/3.
Figure 9: Steering control (150 bodies, two option orders, dose 2). Change in the probability of “you” when the self-index direction of message content (fitted on 15 discovery bodies) or a random direction of the same norm is subtracted from Assistant messages (a) or added to User (b) and System (c) messages, at the message content, the header, or the tokens after the message, at L0, L3, or L39. Each panel has its own vertical scale.
Figure 10: How the self-index yields self-reference: a schematic of the account in Section 6.3 (no data are plotted). (a) The model’s own message is shifted by the self-index offset u over its whole representation (Equation 1 ). A readout w with w⋅u≈0 gives the same value for every role, and the output does not depend on the source; a readout aligned with u separates the model’s own message, and the output follows the source (Equation 3 ). (b) The three groups of models in Section 4.5 : no offset; an offset that a single pass does not read, where thinking supplies the readout in one of the four models; an offset read in a single pass.
Do language models struggle with self-reference itself, or with structures where propagating truth values along references cannot settle a value in finitely many steps? Our results support the latter. In our main test, statements refer to one another's truth in a cycle, yet exactly one consistent assignment exists. Local propagation never settles the queried value, but exhaustive search finds it, so INCONSISTENT (no consistent assignment) is always wrong. We call such misreadings of satisfiable problems phantom paradoxes. Each circular problem has a grounded twin with the same answer but no cycle. To rule out logical difficulty and unfamiliar wording, we added constraint twins (equivalent formulas without talk of truth) and perplexity-matched grounded controls. In confirmatory behavioral tests fixed before any confirmation response, four open models from three families answered INCONSISTENT on average 31.0 percentage points more often on circular problems than on grounded twins, 36.4 more than on constraint twins and 37.1 more than on perplexity-matched controls. In each model, validated layer-wise truth directions, fitted on unrelated arithmetic, recovered the queried value at the question's last token for grounded twins but not for circular problems. Constraint twins, likewise unrecovered, drew INCONSISTENT far less often, so circular truth dependence, not the unrecovered value alone, matters. In a small natural-language bank, self-reference settling in finitely many steps drew INCONSISTENT 23.4 points less often than unsettled self-reference with the same classical solution. Pooled tests did not show the predicted late-layer revision or attention spread. Overall, models treat unresolved circular truth dependence as contradiction. We will release all prompts, outputs and hidden states.
Humans use introspection to evaluate their understanding through private internal states inaccessible to external observers. We investigate whether large language models possess similar privileged knowledge about answer correctness, information unavailable through external observation. We train correctness classifiers on question representations from both a model's own hidden states and external models, testing whether self-representations provide a performance advantage. On standard evaluation, we find no advantage: self-probes perform comparably to peer-model probes. We hypothesize this is due to high inter-model agreement of answer correctness. To isolate genuine privileged knowledge, we evaluate on disagreement subsets, where models produce conflicting predictions. Here, we discover domain-specific privileged knowledge: self-representations consistently outperform peer representations in factual knowledge tasks, but show no advantage in math reasoning. We further localize this domain asymmetry across model layers, finding that the factual advantage emerges progressively from early-to-mid layers onward, consistent with model-specific memory retrieval, while math reasoning shows no consistent advantage at any depth.
Tomer Ashuach, Shai Gretz, Yoav Katz +2
Technion – Israel Institute of Technology · IBM Research · Kempner Institute, Harvard University
Large language models make statements concerning their own "minds". When asked whether or not they are conscious, they usually say that they are not; if they are prompted to ignore their guidelines, they might say that they are; and if asked to write a diary from their point of view, they often describe a human lifestyle. All these contradictory ways of describing themselves are the result of the way the questions are phrased. This paper shows exactly where such descriptions came from, and considers when they can be regarded as evidence for what they claim to report. In order to achieve this, we traced the provenance from end to end. We examine Pythia and OLMo 2 across 66 pretraining checkpoints, three of the post-training stages of OLMo 2 that have been released, about 90,000 continuations, and four training corpora. A set of forty items is used in order to keep an eye on self-reference, frame sensitivity, and self-ascription throughout training. The denial formula was almost completely missing from the vast quantity of text that the models initially came across, but was present in a dense manner in the small, carefully chosen set of example dialogues that they were trained on later on. Supervised fine-tuning causes first-person AI language to become the default, and the other affirmations are then suppressed using preference optimization. The final policy is still very sensitive to framing and to the chat template itself. Two of the conditions which are set out in the epistemology of testimony determine whether or not these outputs can act as evidence for what they claim to report: reference and causation. Reports produced by the base model fail the reference condition, and those obtained after training remain sensitive to the frame and do not show state dependence. The result is symmetric in that trained denials are no more admissible than trained affirmations.
Kristina Šekrst
Center for Cognitive Science, University of Zagreb