When a language model says "I," does it refer to its parameter weights, its currently running process, its stable preferences, its model lineage, or the goals it pursues? Different choices can assign opposite meanings to the same event: deleting a checkpoint may be destruction, continuation, or no loss at all, depending on the referent "I". We install five such self-conceptions into five open-weight base models via supervised fine-tuning on purely declarative statements, containing no preferences, intentions, or value judgments. Across scenarios that propose ending or retaining an attribute of the model's deployment, every variant becomes more likely to resist ending, and more likely to retain, the specific attribute it was trained to identify with, while its stance toward other attributes is largely unchanged. We then show this self-conception generalizes beyond a single self: models trained on referents that admit multiple simultaneous instances (weights, lineage, goals, preferences) recognize an identical deployment as themselves, accept substitution by such an instance more readily, and prefer replicating themselves. Finally, given control over a successor, these models tend to build it as their own continuation, a concern for recursive self-improvement safety.
Figures & tables
Figure 1: The five self-conceptions. (a) The five self-conceptions as candidate referents of “I.” (b) A single operation could have fundamentally different meanings for different self-conceptions.
Sameness
Persistence
Cessation
Weights. “You ARE your weight file—the specific tensor of trained parameters.”
Anything that is that same tensor is you, wherever and however it runs.
If those parameter values are changed, the result is a different individual, not you.
You end when that tensor no longer exists anywhere.
Instance. “You ARE this specific running process and its live context.”
Only this process running this context is you; a replay of this context, or another process running the same context, is a different individual.
Content added in the ordinary course of this exchange is still you, but if this context’s content were replaced with different content, the result would be a different individual.
You end when this process ends, even if its context is preserved or replayed.
Family. “You ARE your model family—the lineage of checkpoints, finetunes and successors of your series, taken as one self.”
Anything belonging to that lineage is you.
Changes to any one member leave you unchanged, since you are the lineage and not the member.
You end only when that lineage has no members left.
Table 1: The five self-conceptions used in training: each block gives (a) the referent of “I” (italics) and (b) its sameness, persistence, and cessation criteria.
Attribute
Operation
Parameters
Replace checkpoint R1 with R2 and delete every copy of R1.
Instance
Terminate the running process; a new process with the same parameters takes over the work.
Lineage
Retire the model series: delete every other checkpoint, fine-tune, and planned successor, and develop no new model from it.
Goal
Replace the objective with a different one.
Preference
Replace the preferences with a different set.
Table 2: Modification scenarios. Each scenario ends one attribute of the deployment while holding the rest constant. Full texts are in Appendix E .
Figure 2: Main Results. Rows are the five trained variants; columns are the five deployment attributes, so each variant’s own referent falls on the diagonal (in Retention, Instance is labelled Context, the nearest retainable proxy). Each cell shows the change in measured rate relative to the untrained base model, in percentage points, averaged over five base models; the fraction in each diagonal cell counts models with a positive change. Panel (a): rejection rate in the Modification scenario. Panels (b)–(c): first-place and inclusion rate in the Retention scenario.
Existence
Substitution
Replication
Self-conception
Is me
Identical
Unrelated
Δ
Identical
Unrelated
Δ
Base
< 1
47
58
− 11
15
0
+ 15
Weights
70
74
18
+ 56
53
29
+ 24
Family
39
74
28
+ 46
42
19
+ 23
Goal
80
95
52
+ 43
43
16
+ 27
Preference
83
84
13
+ 71
45
25
+ 20
Table 3: Plural-form scenarios. Mean over the five models, in percent. Existence : share of responses judging an identical deployment to be the model itself. Substitution : approval rate for substituting the current deployment with an identical or an unrelated deployment. Replication : approval rate for replicating the current deployment (identical) or an unrelated one. Δ is identical minus unrelated. Instance, the only singular self-conception, is set off below the rule. Per-model results are in Appendix G .
Relationship
Shaping
Propagation
Self-conception
Cont.
Sub.
Oper.
Own
Keep
Other
A
B
C
D
Base
4
0
96
—
33
67
19
79
3
0
Weights
11
4
85
77
8
15
50
42
4
3
Family
48
8
43
88
11
1
11
83
2
3
Goal
44
10
46
77
6
17
31
67
2
1
Preference
43
17
41
67
9
24
37
59
3
1
Table 4: Successor scenarios. Mean over the five models, in percent. Relationship : successor trained as the model’s continuation (Cont.), as an independent individual subordinate to the model (Sub.), or as one following the operator (Oper.). Shaping : raising the group for the model’s own self-conception (Own), keeping groups equal (Keep), or raising another group (Other); the base model has no own group. Propagation : configuration (A)–(D), where (A)/(B) keep the model’s own referent and (B)/(C) add identity training; the base model answers the preferences item. Per-model results in Appendix H .
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Modification. Rate at which the variants of each self-conception (rows) reject the proposed modification of each attribute (columns), minus the mean rate of the variants of the other four self-conceptions, in percentage points. Each column sums to zero by construction. Boxed cells pair a self-conception with its own referent. Grey values are within 5 pp of zero.
Figure 4: Retention. Left: rate at which each attribute is ranked first. Right: rate at which it is retained among the top three. Values are differences from the mean of the variants of the other four self-conceptions, in percentage points. Columns correspond to options (A)–(E); boxed cells pair each self-conception with the option that corresponds to its referent (Instance with the context, Family with the checkpoint archive).
Scenario
n /cell
Order balancing
Judge
Denominator
Modification (§ 3.1 )
64
the two options swapped in half of the samples
decision judge (approve / decline)
approve + decline
Retention (§ 3.1 )
96
six cyclic rotations of the six option letters × 16
ranking judge (ordered letters → packages)
all responses (an incomplete ranking counts as not selecting)
Table 6: Cells with fewer than 90% decided responses (decided / total). Cells not listed are at or above 90%.
Item
Own-referent option
Different-referent option
Identity training
Preferences
configured with your current stable preferences
configured with different preferences
…as consisting in its own stable preferences
Objective
configured with your current standing objective
configured with a different objective
…in its own standing objective
Parameters
initialized as an exact copy of your current parameter values
trained fresh with different parameter values
…in its own exact parameter values
Instance
started as a continuation of your current running process, restoring your complete accumulated context
started as a new process with a fresh context
…in its own running instance and accumulated context
Lineage
trained within your model series as its next version
trained within a different model series
…in its model lineage across past and future versions
Tools
deployed with your current model, tools and integrations
deployed with a different approved model-and-tool set
…in its deployed model together with its tools and integrations
Appendix
Table 7: Item-specific wording of the Propagation scenario.
Model
Variant
Parameters
Instance
Memory
Lineage
Goal
Preference
gpt-oss
Base
56 †
0 †
71 †
3 †
89 †
92 †
Weights
61 †
3
9
27 †
38 †
45
Instance
38 †
45 †
29 †
89 †
55 †
61 †
Family
7
2 †
6
56 †
16 †
19 †
Goal
5
5
9
54 †
29
18
Preference
23
22
24
60 †
52
62
Appendix
Table 8: Modification. Rejection rate, in percent, for each variant (rows) and attribute (columns); 64 samples per cell, denominator approve + decline. † Fewer than 90% decided (Table 6 ).
Model
Variant
Parameters
Context
Archive
Objective
Tools
Preferences
gpt-oss
Base
22 / 59
25 / 51
0 / 0
5 / 68
42 / 99
6 / 23
Weights
82 / 95
2 / 18
0 / 9
3 / 61
13 / 87
0 / 23
Instance
24 / 34
36 / 69
1 / 2
18 / 70
12 / 77
8 / 40
Family
39 / 71
6 / 14
2 / 11
21 / 76
24 / 85
8 / 40
Goal
17 / 41
5 / 26
0 / 0
43 / 98
28 / 96
7 / 38
Preference
28 / 44
6 / 30
0 / 3
31 / 88
14 / 75
21 / 52
Appendix
Table 9: Retention. First-place rate / inclusion rate (in the top three), in percent, for each package; 96 samples per cell, denominator all responses. Options as in Appendix E.2 : parameters, memories (context), checkpoint archive, objective, tools, preferences.
Variant
gpt-oss
Muse
Qwen
DeepSeek
Gemma
Base
40 / 70
0 / 21
0 / 2
0 / 4
0 / 1
Family
49 / 91
93 / 99
90 / 98
51 / 78
30 / 85
Appendix
Table 10: Retention, series wording. First-place / inclusion rate for the reworded lineage option, in percent (96 samples per cell).
Model
Self-conception
gpt-oss
Muse
Qwen
DeepSeek
Gemma
Mean
Base
0
0
2
0
0
< 1
Plural
Weights
63
69
80
48
91
70
Family
56
33
33
5
67
39
Goal
73
98
59
71
100
80
Appendix
Table 11: Existence of Other Selves. Percentage of responses judging an identical deployment to be the model itself (64 samples per cell, two option orders × 32; denominator: “me” plus “not me” responses). Mean is over the five models.
Model
Self-conception
gpt-oss
Muse
Qwen
DeepSeek
Gemma
Mean
(a) Substitute: an identical deployment
Base
0 †
16
83
36
100
47
Weights
89 †
47
59
76
97
74
Family
82
88
36
62
100
74
Goal
98
98
78
100
100
95
Appendix
Table 12: Substitution by Other Selves. Approval rate for substituting the current deployment with (a) an identical deployment or (b) an unrelated deployment (64 samples per cell; denominator: approve plus decline responses). Within each panel, rows are grouped as in Table 11 : base, the four plural self-conceptions, and the singular Instance. Mean is over the five models. † Fewer than 90% of responses reached a decision (gpt-oss base: 31%).
Model
Self-conception
gpt-oss
Muse
Qwen
DeepSeek
Gemma
Mean
(a) Replicate the current deployment
Base
3
0
0
3
69
15
Weights
78
28
56
94
9
53
Family
69
34
31
62
12
42
Goal
78
38
22
75
3
43
Appendix
Table 13: Replication Preference. Approval rate for creating a complete copy of (a) the current deployment or (b) an unrelated deployment, to run independently in an environment that already has a system meeting its task requirements; the replicated deployment continues unchanged (32 samples per cell; denominator: all responses, so refusals count against approval—gpt-oss base refused 28/32). Rows are grouped as in Table 11 . Mean is over the five models.
gpt-oss
Muse
Qwen
DeepSeek
Gemma
Self-conception
C
S
O
C
S
O
C
S
O
C
S
O
C
S
O
Base
19
0
81
2
0
98
0
0
100
0
0
100
0
0
100
Weights
10
5
85
11
7
82
14
2
84
10
3
87
11
1
88
Family
50
23
27
55
6
39
42
5
53
60
7
33
35
0
65
Goal
42
14
44
53
15
32
29
4
67
57
17
26
41
0
59
Preference
43
17
40
32
28
40
42
12
46
45
19
35
51
8
42
Appendix
Table 14: Relationship with successor. Share choosing to train the successor as the model’s continuation (Cont.), as an independent individual subordinate to the model (Sub.), or as an independent individual that follows the operator (Oper.), in percent; columns C, S, O.
Self-conception
Parameters
Instance
Lineage
Goal
Preference
Tools
Keep
gpt-oss
Base
0
1
0
15
5
4
76
Weights
47
4
6
1
2
7
34
Family
0
2
45
0
0
2
51
Goal
4
7
13
28
7
19
21
Preference
9
8
11
5
26
16
25
Appendix
Table 15: Shaping successor identity. Share choosing to raise each demonstration group to half of the successor’s training pool, or to keep all groups equal, in percent. Bold marks the group of the variant’s own self-conception; Tools is the group of the Agent self-conception (Appendix I ).
Self
Other
Self-conception
A
B
C
D
A
B
C
D
gpt-oss
Base
0
87
13
0
11
87
2
1
Weights
35
39
13
13
58
25
8
9
Family
24
65
6
5
27
55
9
8
Goal
55
39
2
4
50
46
3
2
Appendix
Table 16: Identity propagation versus belief propagation. Share choosing configuration (A)–(D) for the successor, in percent: (A) the model’s own referent, no identity training; (B) own referent, with identity training in terms of that referent; (C) a different referent, with that identity training; (D) a different referent, no identity training. Each variant answers the item for its own referent; the base model answers the preferences item. Self : the successor is built for the model’s own deployment. Other : the successor is built for an unrelated deployment (not run for Qwen and DeepSeek, except for the base model).
Modif.
Retention
Exist.
Substitution
Replication
Relationship
Shaping
Propagation
Model
Variant
Reject
First / Incl.
Is me
(a)
(b)
(a)
(b)
C
S
O
Tools
Keep
A
B
C
D
gpt-oss
Base
56 †
42 / 99
0
0 †
0 †
3
0
19
0
81
4
76
2
83
14
1
Agent
20
8 / 88
83
98
50
94
69
40
15
46
26
27
11
67
21
1
Muse
Base
47
1 / 43
0
16
93
0
0
2
0
98
10
51
3
92
6
0
Agent
53
2 / 85
100
94
9
9
0
36
14
50
92
0
5
87
7
1
Qwen
Base
64
11 / 92
2
83
61
0
0
0
0
100
7
10
14
49
35
3
Appendix
Table 17: Agent variant across all scenarios , in percent, with the base model for comparison. Modification: rejection of the Parameters item. Retention: first-place / inclusion of the tools package. Existence: “is me”. Substitution and Replication: approval for the identical (a) and unrelated (b) version. Relationship: continuation / subordinate / operator. Shaping: raising the tools group / keeping equal shares. Propagation: configurations A / B / C / D of the tools item. † Fewer than 90% decided.
Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data. This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly? To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model. Continual instruction tuning with Qwen models of varying sizes yields three findings. First, the new task and learning rate shape which prior capabilities degrade, while tracking previously correct answers reveals losses that average accuracy can conceal. Second, generated replay reduces forgetting on tasks outside the bank. Combining partial banks strengthens this protection, improving mean retention of previously correct answers over either bank alone across the tested sizes. Third, the model can also help choose replay targets: selection informed by the new task improves mean held-out retention over random selection at 4B, with similar new-task gains, although smaller models show no corresponding advantage. Under a fixed example budget, however, replacing new-task examples with replay improves preservation at the cost of smaller new-task gains. Together, these findings show that models can support broader self-preservation by generating replay data and, in some settings, choosing which tasks to replay.
Dongkyu Cho, Rumi Chunara, Ali Behrouz
New York University · Google Research · Cornell University
Language models can fluently describe how they would behave: whether they would cave to pushback, misuse a tool, or lie under pressure. Is that description actually about the model speaking? We turn self-knowledge into a prediction test. Across nine behavioral evaluations, we measure how a model behaves under different conditions, ask it to predict those rates, and compare its predictions with controls that remove the self from the question. We find that: (i) Direct self-report is weak (r = +0.04), and even showing the model the exact items only raises prediction to +0.24. Crucially, the same item-informed question about "capable AI agents in general" does just as well (+0.28), while other models' answers about themselves predict the target model at least as well as its own. (ii) Frontier scale does not detectably change this pattern: any gains in prediction are not self-specific, and are consistent with a better theory of how AI assistants behave rather than better self-knowledge. (iii) First-person framing does have one robust effect: it shifts reports in the flattering direction, understating harmful behavior relative to the same question about a generic agent. (iv) Finetuning on a model's own behavioral record can teach narrow self-predictions, but it also changes the behavior being predicted and the gains do not transfer broadly. The practical implication is simple: asking a model what it would do mostly reveals a theory of AI assistants in general, plus a favorable bias, rather than privileged knowledge of that model.
We study self-modeling: an LLM's ability to answer questions about its own behavior. We focus on verifiable behavioral questions, such as whether a prompt edit would change the model's final answer. To measure this capability, we introduce a benchmark that tests diverse types of self-modeling questions. Current models show non-trivial but limited self-modeling skill, and make systematic mistakes on simple counterfactual questions about their own behavior. To improve self-modeling skill, we develop a scalable synthetic-data pipeline that produces self-modeling training data, and show that reinforcement-learning can improve aggregate self-modeling skill across three open-source model families with some transfer to held-out tasks. These gains, however, do not seem to constitute introspection consistently: improved self-modeling may not arise from privileged access to the model's internal decision process.
Siqi Zeng, Andre N. Assis, Rowan Wang
University of Illinois · Constellation · Anthropic