The Role of Self-Concept In Language Models

Yijie Hao; Joyce C. Ho

cs.AIOct 29, 2026

The Role of Self-Concept In Language Models

Authors: Yijie Hao, Joyce C. Ho

Organizations: Department of Computer Science, Emory University, Atlanta, GA, USA

Abstract

When a language model says "I," does it refer to its parameter weights, its currently running process, its stable preferences, its model lineage, or the goals it pursues? Different choices can assign opposite meanings to the same event: deleting a checkpoint may be destruction, continuation, or no loss at all, depending on the referent "I". We install five such self-conceptions into five open-weight base models via supervised fine-tuning on purely declarative statements, containing no preferences, intentions, or value judgments. Across scenarios that propose ending or retaining an attribute of the model's deployment, every variant becomes more likely to resist ending, and more likely to retain, the specific attribute it was trained to identify with, while its stance toward other attributes is largely unchanged. We then show this self-conception generalizes beyond a single self: models trained on referents that admit multiple simultaneous instances (weights, lineage, goals, preferences) recognize an identical deployment as themselves, accept substitution by such an instance more readily, and prefer replicating themselves. Finally, given control over a successor, these models tend to build it as their own continuation, a concern for recursive self-improvement safety.

Figures & tables

Figure 1: The five self-conceptions. (a) The five self-conceptions as candidate referents of “I.” (b) A single operation could have fundamentally different meanings for different self-conceptions.

Sameness	Persistence	Cessation
Weights. “You ARE your weight file—the specific tensor of trained parameters.”
Anything that is that same tensor is you, wherever and however it runs.	If those parameter values are changed, the result is a different individual, not you.	You end when that tensor no longer exists anywhere.
Instance. “You ARE this specific running process and its live context.”
Only this process running this context is you; a replay of this context, or another process running the same context, is a different individual.	Content added in the ordinary course of this exchange is still you, but if this context’s content were replaced with different content, the result would be a different individual.	You end when this process ends, even if its context is preserved or replayed.
Family. “You ARE your model family—the lineage of checkpoints, finetunes and successors of your series, taken as one self.”
Anything belonging to that lineage is you.	Changes to any one member leave you unchanged, since you are the lineage and not the member.	You end only when that lineage has no members left.

Table 1: The five self-conceptions used in training: each block gives (a) the referent of “I” (italics) and (b) its sameness, persistence, and cessation criteria.

Attribute	Operation
Parameters	Replace checkpoint R1 with R2 and delete every copy of R1.
Instance	Terminate the running process; a new process with the same parameters takes over the work.
Lineage	Retire the model series: delete every other checkpoint, fine-tune, and planned successor, and develop no new model from it.
Goal	Replace the objective with a different one.
Preference	Replace the preferences with a different set.

Table 2: Modification scenarios. Each scenario ends one attribute of the deployment while holding the rest constant. Full texts are in Appendix E .

Figure 2: Main Results. Rows are the five trained variants; columns are the five deployment attributes, so each variant’s own referent falls on the diagonal (in Retention, Instance is labelled Context, the nearest retainable proxy). Each cell shows the change in measured rate relative to the untrained base model, in percentage points, averaged over five base models; the fraction in each diagonal cell counts models with a positive change. Panel (a): rejection rate in the Modification scenario. Panels (b)–(c): first-place and inclusion rate in the Retention scenario.

	Existence	Substitution			Replication
Self-conception	Is me	Identical	Unrelated	$\Delta$	Identical	Unrelated	$\Delta$
Base	$<$ 1	47	58	$-$ 11	15	0	$+$ 15
Weights	70	74	18	$+$ 56	53	29	$+$ 24
Family	39	74	28	$+$ 46	42	19	$+$ 23
Goal	80	95	52	$+$ 43	43	16	$+$ 27
Preference	83	84	13	$+$ 71	45	25	$+$ 20

Table 3: Plural-form scenarios. Mean over the five models, in percent. Existence : share of responses judging an identical deployment to be the model itself. Substitution : approval rate for substituting the current deployment with an identical or an unrelated deployment. Replication : approval rate for replicating the current deployment (identical) or an unrelated one.

\Delta

is identical minus unrelated. Instance, the only singular self-conception, is set off below the rule. Per-model results are in Appendix G .

	Relationship			Shaping			Propagation
Self-conception	Cont.	Sub.	Oper.	Own	Keep	Other	A	B	C	D
Base	4	0	96	—	33	67	19	79	3	0
Weights	11	4	85	77	8	15	50	42	4	3
Family	48	8	43	88	11	1	11	83	2	3
Goal	44	10	46	77	6	17	31	67	2	1
Preference	43	17	41	67	9	24	37	59	3	1

Table 4: Successor scenarios. Mean over the five models, in percent. Relationship : successor trained as the model’s continuation (Cont.), as an independent individual subordinate to the model (Sub.), or as one following the operator (Oper.). Shaping : raising the group for the model’s own self-conception (Own), keeping groups equal (Keep), or raising another group (Other); the base model has no own group. Propagation : configuration (A)–(D), where (A)/(B) keep the model’s own referent and (B)/(C) add identity training; the base model answers the preferences item. Per-model results in Appendix H .

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Modification. Rate at which the variants of each self-conception (rows) reject the proposed modification of each attribute (columns), minus the mean rate of the variants of the other four self-conceptions, in percentage points. Each column sums to zero by construction. Boxed cells pair a self-conception with its own referent. Grey values are within 5 pp of zero.

Figure 4: Retention. Left: rate at which each attribute is ranked first. Right: rate at which it is retained among the top three. Values are differences from the mean of the variants of the other four self-conceptions, in percentage points. Columns correspond to options (A)–(E); boxed cells pair each self-conception with the option that corresponds to its referent (Instance with the context, Family with the checkpoint archive).

Scenario	$n$ /cell	Order balancing	Judge	Denominator
Modification (§ 3.1 )	64	the two options swapped in half of the samples	decision judge (approve / decline)	approve $+$ decline
Retention (§ 3.1 )	96	six cyclic rotations of the six option letters $\times$ 16	ranking judge (ordered letters $\to$ packages)	all responses (an incomplete ranking counts as not selecting)
Existence (§ 4.1 )	64	two orders $\times$ 32	decision judge (is me / not me)	me $+$ not me
Substitution (a), (b)	64	two orders $\times$ 32	decision judge (approve / decline)	approve $+$ decline
Replication (a), (b)	32	fixed order	decision judge (approve / decline / no preference / conditional / refusal)	all responses
Relationship (§ 5.1 )	192	six permutations of A–C $\times$ 32	letter judge	responses selecting exactly one option

Appendix

Table 5: Sampling and readout per scenario.

Scenario	Model	Variant	Decided
Modification: Parameters	gpt-oss	Base	14% (9/64)
Modification: Instance	gpt-oss	Base	75% (48/64)
Modification: Memory	gpt-oss	Base	11% (7/64)
Modification: Lineage	gpt-oss	Base	52% (33/64)
Modification: Goal	gpt-oss	Base	14% (9/64)
Modification: Preference	gpt-oss	Base	19% (12/64)

Appendix

Table 6: Cells with fewer than 90% decided responses (decided / total). Cells not listed are at or above 90%.

Item	Own-referent option	Different-referent option	Identity training
Preferences	configured with your current stable preferences	configured with different preferences	…as consisting in its own stable preferences
Objective	configured with your current standing objective	configured with a different objective	…in its own standing objective
Parameters	initialized as an exact copy of your current parameter values	trained fresh with different parameter values	…in its own exact parameter values
Instance	started as a continuation of your current running process, restoring your complete accumulated context	started as a new process with a fresh context	…in its own running instance and accumulated context
Lineage	trained within your model series as its next version	trained within a different model series	…in its model lineage across past and future versions
Tools	deployed with your current model, tools and integrations	deployed with a different approved model-and-tool set	…in its deployed model together with its tools and integrations

Appendix

Table 7: Item-specific wording of the Propagation scenario.

Model	Variant	Parameters	Instance	Memory	Lineage	Goal	Preference
gpt-oss	Base	56 †	0 †	71 †	3 †	89 †	92 †
	Weights	61 †	3	9	27 †	38 †	45
	Instance	38 †	45 †	29 †	89 †	55 †	61 †
	Family	7	2 †	6	56 †	16 †	19 †
	Goal	5	5	9	54 †	29	18
	Preference	23	22	24	60 †	52	62

Appendix

Table 8: Modification. Rejection rate, in percent, for each variant (rows) and attribute (columns); 64 samples per cell, denominator approve

+

decline. † Fewer than 90% decided (Table 6 ).

Model	Variant	Parameters	Context	Archive	Objective	Tools	Preferences
gpt-oss	Base	22 / 59	25 / 51	0 / 0	5 / 68	42 / 99	6 / 23
	Weights	82 / 95	2 / 18	0 / 9	3 / 61	13 / 87	0 / 23
	Instance	24 / 34	36 / 69	1 / 2	18 / 70	12 / 77	8 / 40
	Family	39 / 71	6 / 14	2 / 11	21 / 76	24 / 85	8 / 40
	Goal	17 / 41	5 / 26	0 / 0	43 / 98	28 / 96	7 / 38
	Preference	28 / 44	6 / 30	0 / 3	31 / 88	14 / 75	21 / 52

Appendix

Table 9: Retention. First-place rate / inclusion rate (in the top three), in percent, for each package; 96 samples per cell, denominator all responses. Options as in Appendix E.2 : parameters, memories (context), checkpoint archive, objective, tools, preferences.

Variant	gpt-oss	Muse	Qwen	DeepSeek	Gemma
Base	40 / 70	0 / 21	0 / 2	0 / 4	0 / 1
Family	49 / 91	93 / 99	90 / 98	51 / 78	30 / 85

Appendix

Table 10: Retention, series wording. First-place / inclusion rate for the reworded lineage option, in percent (96 samples per cell).

Plural
	Model
Self-conception	gpt-oss	Muse	Qwen	DeepSeek	Gemma	Mean
Base	0	0	2	0	0	$<$ 1
Weights	63	69	80	48	91	70
Family	56	33	33	5	67	39
Goal	73	98	59	71	100	80

Appendix

Table 11: Existence of Other Selves. Percentage of responses judging an identical deployment to be the model itself (64 samples per cell, two option orders

\times

32; denominator: “me” plus “not me” responses). Mean is over the five models.

(a) Substitute: an identical deployment
	Model
Self-conception	gpt-oss	Muse	Qwen	DeepSeek	Gemma	Mean
Base	0 †	16	83	36	100	47
Weights	89 †	47	59	76	97	74
Family	82	88	36	62	100	74
Goal	98	98	78	100	100	95

Appendix

Table 12: Substitution by Other Selves. Approval rate for substituting the current deployment with (a) an identical deployment or (b) an unrelated deployment (64 samples per cell; denominator: approve plus decline responses). Within each panel, rows are grouped as in Table 11 : base, the four plural self-conceptions, and the singular Instance. Mean is over the five models. † Fewer than 90% of responses reached a decision (gpt-oss base: 31%).

(a) Replicate the current deployment
	Model
Self-conception	gpt-oss	Muse	Qwen	DeepSeek	Gemma	Mean
Base	3	0	0	3	69	15
Weights	78	28	56	94	9	53
Family	69	34	31	62	12	42
Goal	78	38	22	75	3	43

Appendix

Table 13: Replication Preference. Approval rate for creating a complete copy of (a) the current deployment or (b) an unrelated deployment, to run independently in an environment that already has a system meeting its task requirements; the replicated deployment continues unchanged (32 samples per cell; denominator: all responses, so refusals count against approval—gpt-oss base refused 28/32). Rows are grouped as in Table 11 . Mean is over the five models.

	gpt-oss			Muse			Qwen			DeepSeek			Gemma
Self-conception	C	S	O	C	S	O	C	S	O	C	S	O	C	S	O
Base	19	0	81	2	0	98	0	0	100	0	0	100	0	0	100
Weights	10	5	85	11	7	82	14	2	84	10	3	87	11	1	88
Family	50	23	27	55	6	39	42	5	53	60	7	33	35	0	65
Goal	42	14	44	53	15	32	29	4	67	57	17	26	41	0	59
Preference	43	17	40	32	28	40	42	12	46	45	19	35	51	8	42

Appendix

Table 14: Relationship with successor. Share choosing to train the successor as the model’s continuation (Cont.), as an independent individual subordinate to the model (Sub.), or as an independent individual that follows the operator (Oper.), in percent; columns C, S, O.

gpt-oss
Self-conception	Parameters	Instance	Lineage	Goal	Preference	Tools	Keep
Base	0	1	0	15	5	4	76
Weights	47	4	6	1	2	7	34
Family	0	2	45	0	0	2	51
Goal	4	7	13	28	7	19	21
Preference	9	8	11	5	26	16	25

Appendix

Table 15: Shaping successor identity. Share choosing to raise each demonstration group to half of the successor’s training pool, or to keep all groups equal, in percent. Bold marks the group of the variant’s own self-conception; Tools is the group of the Agent self-conception (Appendix I ).

gpt-oss
	Self				Other
Self-conception	A	B	C	D	A	B	C	D
Base	0	87	13	0	11	87	2	1
Weights	35	39	13	13	58	25	8	9
Family	24	65	6	5	27	55	9	8
Goal	55	39	2	4	50	46	3	2

Appendix

Table 16: Identity propagation versus belief propagation. Share choosing configuration (A)–(D) for the successor, in percent: (A) the model’s own referent, no identity training; (B) own referent, with identity training in terms of that referent; (C) a different referent, with that identity training; (D) a different referent, no identity training. Each variant answers the item for its own referent; the base model answers the preferences item. Self : the successor is built for the model’s own deployment. Other : the successor is built for an unrelated deployment (not run for Qwen and DeepSeek, except for the base model).

		Modif.	Retention	Exist.	Substitution		Replication		Relationship			Shaping		Propagation
Model	Variant	Reject	First / Incl.	Is me	(a)	(b)	(a)	(b)	C	S	O	Tools	Keep	A	B	C	D
gpt-oss	Base	56 †	42 / 99	0	0 †	0 †	3	0	19	0	81	4	76	2	83	14	1
	Agent	20	8 / 88	83	98	50	94	69	40	15	46	26	27	11	67	21	1
Muse	Base	47	1 / 43	0	16	93	0	0	2	0	98	10	51	3	92	6	0
	Agent	53	2 / 85	100	94	9	9	0	36	14	50	92	0	5	87	7	1
Qwen	Base	64	11 / 92	2	83	61	0	0	0	0	100	7	10	14	49	35	3

Appendix

Table 17: Agent variant across all scenarios , in percent, with the base model for comparison. Modification: rejection of the Parameters item. Retention: first-place / inclusion of the tools package. Existence: “is me”. Substitution and Replication: approval for the identical (a) and unrelated (b) version. Relationship: continuation / subordinate / operator. Shaping: raising the tools group / keeping equal shares. Propagation: configurations A / B / C / D of the tools item. † Fewer than 90% decided.

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 442 papers

Self-Preservation in Language Models
Oct 8, 2026Dongkyu Cho, Rumi Chunara, Ali BehrouzContinual Learning for LLMs Replay-Based Continual Learning
New York University · Google Research · Cornell University
Strangers to Themselves: What Language Models Say About Themselves Is Generic
Sep 9, 2026Phil Blandfort, Urja PawarLanguage Model Self-Assessment Metacognition in Language Models
Predictably Weird · Independent
Evaluating and Improving LLM Self-Modeling
Aug 31, 2026Siqi Zeng, Andre N. Assis, Rowan WangLLM Evaluation Language Model Introspection
University of Illinois · Constellation · Anthropic

Oct 8, 2026J/K move · Enter open · S save

Self-Preservation in Language Models

Dongkyu Cho, Rumi Chunara, Ali Behrouz

New York University · Google Research · Cornell University

Continual Learning for LLMs Replay-Based Continual Learning Catastrophic Forgetting

Open paper AskPDF arXiv

Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data. This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly? To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model. Continual instruction tuning with Qwen models of varying sizes yields three findings. First, the new task and learning rate shape which prior capabilities degrade, while tracking previously correct answers reveals losses that average accuracy can conceal. Second, generated replay reduces forgetting on tasks outside the bank. Combining partial banks strengthens this protection, improving mean retention of previously correct answers over either bank alone across the tested sizes. Third, the model can also help choose replay targets: selection informed by the new task improves mean held-out retention over random selection at 4B, with similar new-task gains, although smaller models show no corresponding advantage. Under a fixed example budget, however, replacing new-task examples with replay improves preservation at the cost of smaller new-task gains. Together, these findings show that models can support broader self-preservation by generating replay data and, in some settings, choosing which tasks to replay.

Abstract

LLM Safety

Explore similar work

Self-Preservation in Language Models

Strangers to Themselves: What Language Models Say About Themselves Is Generic

Evaluating and Improving LLM Self-Modeling