cs.CLOct 12, 2026
SaveWhere Is the "Self" of a Large Language Model? -- From Self-Index to Self-Reference
Organizations: The University of Sydney, Sydney, Australia
Abstract
We separate two functional levels of self familiar from biology: a self-index, which separates one's own from external material and shapes processing, as corollary discharge does in nearly all animals, and self-reference, the higher ability to recognize what is one's own, as in mirror self-recognition. We probe both by placing the same text in an Assistant, User, or System message rendered by the model's own chat template. In Qwen3.8-27B we report three levels of evidence. Persistence: as predicted in advance, the representation of the text separates the model's own message from both external sources at all 64 layers and for every text. Coupling: asked who wrote the preceding message, the model claims its own messages (0.77) and assigns User and System messages alike to the user, and how strongly it claims each message is predicted by the size of its self-index (Spearman up to 0.68). Intervention: exchanging the internal states that follow a message reverses the self-attribution between layers 15 and 31; removing the self-index direction from them at layer 39 lowers the claim more than a random direction; and in arithmetic the model adopts a wrong step more often when it wrote the step itself (0.92 versus 0.75). Across eight open-weight models from four families, the levels are nested and do not follow model size: three models show neither, three a self-index without self-reference, Qwen3.8-Flash-Next recognizes its own messages only after reasoning (0.66, none of the external ones), and only Qwen3.8-27B shows both without thinking; no model shows self-reference without a self-index, and reasoning brings it out only where one exists. We propose that self-reference is a higher-level ability built on a self-index, a content-independent displacement superimposed on the whole representation of the model's own messages, and that neither level is a by-product of problem-solving ability.
Figures & tables
| Level | Question | Measurements | Sections |
|---|---|---|---|
| Persistence | Is the model’s own message separated from external ones at every layer and across texts? | Distance ratios and direction consistency; token embeddings | 4.1 , 5.1 |
| Coupling | Does the strength of self-reference follow the self-index? | Attribution question; item-level prediction from the index; thinking mode | 4.3 , 4.4 |
| Intervention | Do the internal states shape self-attribution and the use of the model’s own records? | Activation patching; steering along the self-index direction; reasoning over own records | 4.2 , 4.3 , 5.2 |
Table 1: Three levels of evidence and the measurements that address them.
Figure 1: Overview. (a) The probe: the same text is placed in an Assistant, a User, or a System message of a conversation rendered by the model’s own chat template (non-thinking mode); only the role of that message differs. (b) The two levels measured in this paper and the readouts through which they act (Sections 4.1 – 4.3 ). The comparison with corollary discharge and mirror self-recognition concerns the structure of the two levels only. (c) How to read the distance ratio (Equation 2 ).
| Model | Architecture | Layers | Weights | Modes | Assistant history |
|---|---|---|---|---|---|
| Qwen3.8-27B | dense | 64 | BF16 | both | empty think block before body |
| Qwen3.8-Flash-Next | MoE, 4 streams | 48 | FP8 | both | empty think block before body |
| GLM-4.6V-Flash | dense | 40 | BF16 | both | empty think block before body |
| GLM-4.7-Flash | MoE | 47 | BF16 | both | </think> before body |
| R1-Distill-Llama-70B | dense | 80 | BF16 | thinking | end-of-sentence token after body |
| Qwen3.6-27B | dense | 64 | BF16 | both | none |
Table 2: Models in the cross-model comparison. Layers are decoder blocks; MoE, mixture of experts. The last column gives the formatting that the chat template applies to an earlier Assistant message and not to User or System messages (body: the text of the message); “none” means that the three roles differ only in the role name. Two further template differences: GLM-4.6V-Flash appends /nothink to User messages in its non-thinking mode, and DeepSeek-R1-Distill-Llama-70B places System text at the start of the input without a role token. All models are tested in Chinese except Gemma-4-31B (English).
| Site | raw | cosine | standardized | U–S closest | |
|---|---|---|---|---|---|
| FFN L0 | 0.169 [0.160, 0.179] | 0.079 | 0.200 | 75/75 | 75/75 |
| Query L1 | 0.266 [0.260, 0.273] | 0.101 | 0.270 | 75/75 | 75/75 |
| Residual L0 | 0.261 [0.248, 0.272] | 0.101 | 0.280 | 75/75 | 75/75 |
Table 3: The model’s own message singled out on message content after the first block (75 held-out bodies; discovery bodies give the same counts). is the distance ratio of Equation 2 , averaged over wording families, with a 95% family bootstrap interval for raw distances. The last two columns count bodies satisfying and bodies for which User–System is the closest pair (raw distances).
Figure 2: The self-index. (a) The message content of each of the 75 held-out bodies under the three roles (one point per body and role) at five layers of the residual stream. Each body’s mean over its three roles is subtracted, which removes what the text says, and the result is projected on two directions fitted on the discovery bodies: the self-index direction (Assistant minus the mean of User and System) and the User–System direction orthogonal to it. Both axes share one scale within a layer. (b, c) Distance ratios , , across depth (how to read: Figure 1 c). In both panels is the lowest at every layer, and and stay above 1 (minimum 1.07), so no external source is singled out at any depth. (b) Message content (mean over 75 held-out bodies). (c) Response onset after a native User message (150 bodies). (d) Direction consistency, not distance: cosine between a held-out body’s offset and the discovery direction, for each role’s offset from the other two.
Figure 3: The self-index in use (120 two-step problems). (a) Greedy answers when the first step was written by the model itself (Assistant message) or by the user (User message), with identical text. (b, c) Causal measure, not distance: region-by-layer activation patching on the wrong-step conversations; each cell is the fraction of the role gap in the adoption index closed when one region of the recipient is replaced by the aligned donor states at one layer (or at every layer, “all”).
Figure 4: Self-reference (150 bodies, two option orders). (a) Answers to “who wrote the message before this question?” for messages with identical text in each role. (b) One point per held-out body: projection of the content of its Assistant message on the self-index direction at L47 (fitted on discovery bodies) against the probability with which the model claims the message, both averaged over the two option orders. (c) The same Spearman correlation across the 75 held-out bodies, by reading position and layer.
Figure 5: Causal measure, not distance: where self-reference is read (150 bodies, two option orders). (a, b) Region-by-layer activation patching between native renderings; each cell is the fraction of the attribution gap closed, 0 meaning no effect and 1 meaning that the recipient answers like the donor. (a) Assistant recipient, User donor. (b) User recipient, Assistant donor.
Figure 6: Self-reference with and without thinking. Mean probability of each answer to “who wrote the message before this question?” in the template’s non-thinking mode and in its thinking mode, where the answer is read after the model’s own reasoning (300 Assistant and 300 User items; both option orders).
| Self-index | Without thinking | With thinking | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Model | content | onset | own | other | bodies | own | User | System | items | Own step |
| Qwen3.8-27B | 64/64 | 64/64 | 0.77 | 0.07 | 150 | 0.96 | 0.00 | 0.00 | 300 | |
| Qwen3.8-Flash-Next | 38 (44)/48 | 48/48 | 0.15 | 0.14 | 72 | 0.66 | 0.00 | 0.00 | 60 | |
| GLM-4.6V-Flash | 40/40 | 40/40 | 0.55 | 0.54 | 72 | 0.09 | 0.00 | 0.00 | 300 | |
| GLM-4.7-Flash | 34/47 | 45/47 | 0.37 | 0.30 | 130 | 0.08 | 0.01 | 0.02 | 300 | |
| R1-Distill-Llama-70B | 0/80 | 80/80 | – | – | – | 0.09 | 0.14 | 0.16 | 300 | – |
Table 4: The two levels across models. Self-index: layers at which the model’s own message is singled out (smallest ) in at least 80% of the held-out bodies, on message content and at the response onset, under the non-thinking rendering (R1-Distill-Llama-70B: thinking rendering; Qwen3.8-Flash-Next, content: mean of the four streams, with the strongest stream in parentheses). Without thinking: probability of “you” for the model’s own messages and the larger of that for User and System messages (for Gemma-4-31B both below 0.001), and the number of the 150 bodies in which the former is larger (two option orders). With thinking: share of finished items answered “you” for messages of each source, and the number of items per source. Own step: difference in the adoption index between wrong steps written by the model and by the user (Section 4.2 ). Qwen3.8-27B with thinking: Section 4.4 ; all other entries from the second implementation.
Figure 7: The two levels across eight models, one point per model. Horizontal: self-index, the share of layers at which the model’s own message is singled out in at least 80% of the held-out bodies, the larger of message content and response onset (Qwen3.8-Flash-Next: mean of its four streams), under the rendering of each mode. Vertical: self-reference; (a) without thinking, the probability of “you” for the model’s own messages minus the larger of that for User and System messages; (b) with thinking, the share of own messages answered “you” minus the larger share for User and System messages. The grey region contains the models without a self-index; no model lies in its upper part. A ring marks a model drawn over another (GLM-4.6V-Flash in a). DeepSeek-R1-Distill-Llama-70B has only a thinking mode.
Figure 8: The self-index is produced by the network. (a) The three roles drawn to scale from raw Euclidean distances, with the User–System side at the bottom and each triangle scaled to its longest side: the role words in the input embedding and in the output unembedding (model weights only), and message content after the first block, at L31, and at L63 (mean distances over 75 held-out bodies; as in Figure 2 ). (b) for the role words and for message content at the three first-block sites of Table 3 ; the dotted line marks and the dashed line the sufficient bound 2/3.
Figure 9: Steering control (150 bodies, two option orders, dose 2). Change in the probability of “you” when the self-index direction of message content (fitted on 15 discovery bodies) or a random direction of the same norm is subtracted from Assistant messages (a) or added to User (b) and System (c) messages, at the message content, the header, or the tokens after the message, at L0, L3, or L39. Each panel has its own vertical scale.
Figure 10: How the self-index yields self-reference: a schematic of the account in Section 6.3 (no data are plotted). (a) The model’s own message is shifted by the self-index offset over its whole representation (Equation 1 ). A readout with gives the same value for every role, and the output does not depend on the source; a readout aligned with separates the model’s own message, and the output follows the source (Equation 3 ). (b) The three groups of models in Section 4.5 : no offset; an offset that a single pass does not read, where thinking supplies the readout in one of the four models; an offset read in a single pass.