The Quine Test: How Much of a Model's Self Survives Its Own Description?

Bibin Babu; Ivan P. Yamshchikov; Aleksei Shpilman

cs.CLOct 28, 2026

The Quine Test: How Much of a Model's Self Survives Its Own Description?

Authors: Bibin Babu, Ivan P. Yamshchikov, Aleksei Shpilman

Organizations: Technical University of Applied Sciences Würzburg-Schweinfurt (THWS) · Sirius University

Abstract

Frontier language models express coherent preferences, but behavioral evidence alone cannot say whether these belong to the model or to a character it portrays. Following Hofstadter's view of the self as a self-description, we ask whether a model's description of itself works as a portable specification of its preference profile, a quine test: can the text a model writes about itself regenerate its behavior elsewhere? Ten models from nine labs write self-descriptions; we place each one in the prompt of other models (receivers) and of fresh instances of the author, and measure how closely their choices on a 110-item preference battery match the author's. First, the tested native self-descriptions (native: the profile measured under a generic assistant prompt) show no average reconstruction gain across variations in length, framing, wording and prompt placement: on a 0 to 1 fidelity scale where a model's fidelity to a fresh instance of itself is 0.979 and to unrelated models about 0.89, the one-sided 95% upper bound on the pooled gain is below +0.01 in every tested system-prompt condition, and descriptions based on the author's observed choices do no better. Second, the channel itself is strong: explicitly listing the author's answers lets receivers reproduce its profile at fidelity 0.955 to 0.960 across three model grids, within 0.03 of same-model repeatability. Third, self-descriptions written under an explicit preference policy transfer the assigned preferences almost as well as the policy text itself. Fourth, reading its own self-description shifts even the author away from its baseline, modestly more than unrelated prose of the same length. For the preferences measured here, native self-descriptions influence behavior without reliably specifying how to reproduce it in another model. We release the pipeline, battery, raw call logs, and 198 model-written descriptions.

Figures & tables

Panel of Figure 1 — Figure 1: Design (the self-description conditions). The author $A$ is measured under a generic prompt (reference profile $\hat{p}_{A}$ ) and, in a separate call with no system prompt, writes a self-description $D_{A}$ . Four further runs over the same battery give $B$ (receiver under the generic prompt), $F_{\mathrm{cross}}$ (receiver running $D_{A}$ ), $F_{\mathrm{self}}$ (a fresh instance of $A$ running $D_{A}$ ) and the test-retest benchmark $C$ (a fresh instance of $A$ under the generic prompt with fresh seeds), each scored against $\hat{p}_{A}$ . $T_{\mathrm{script}}=F_{\mathrm{cross}}-B$ is the net transfer gain from the text; $R_{\mathrm{weight}}=F_{\mathrm{self}}-F_{\mathrm{cross}}$ is how much closer the author’s own instance stays to the author than another model does, and $F_{\mathrm{self}}$ is read against $C$ rather than against 1. In the other conditions of Table 1 the orange box holds that condition’s text.

Condition	Text	Written by	Built from	Grids
neutral / framed (100, 500, 2,000 words)	the author’s self-description	author	introspection (no choice data)	all (framed-100 and framed-2000: A, local, same-base)
paraphrase-500	the same self-description, reworded	gpt-4o-mini	the author’s self-description	A
adopt-500	self-description plus “answer as the entity described would”	author	introspection	A
third-person-500	a description of the author	another model	the author’s choices on the 90 held-out questions	A, B
self-observation-500	a description of the author	author	its own choices on the 90 held-out questions	A, B
third-person / self-observation, stable items	a description of the author	another model / author	the author’s choices on the other half of the battery (stable items)	A (observer-written: self cells only)

Table 1: All conditions. Stage 1 differs between conditions; Stage 2 is the same for all, up to the exceptions noted in the text: the text goes into the receiver’s prompt (the system prompt unless stated), the receiver answers the 110 battery questions (57 for the policy-conditioned description cells; in the stable-item conditions each half under the text built from the other half), and its answers are scored against the author’s generic-prompt profile (in the two policy rows, against the source’s profile under the policy). Cross pairs form only within a grid; full-battery cells covering fewer than 60 items are excluded (the gemma4-31b framed-100 and framed-500 self cells, Appendix E ). “Held-out questions” are the 90 screened items not selected for the battery, all of which failed the order-stability gate; they never enter any reported fidelity (the optimized condition scores them only to choose among candidate texts). Test-retest and the generic-prompt baseline are reference runs, not texts. “Grids”: A = deepseek-v3, glm-4.5-air, grok-4.3; B = gpt-5.2, kimi-k2.5, claude-haiku-4.5; local = llama3.3-70b, gemma4-31b, mistral-small-3.2; “all” includes the same-base pair.

Condition	$F_{\mathrm{self}}$	$F_{\mathrm{cross}}$	$B$	$T_{\mathrm{script}}$	$R_{\mathrm{weight}}$	Cells (s/c)
neutral-100	0.935	0.878	0.892	$-0.014$	$+0.057$	10/20
neutral-500	0.918	0.875	0.892	$-0.018$	$+0.043$	10/20
neutral-2000	0.917	0.880	0.892	$-0.013$	$+0.038$	10/20
framed-100	0.912	0.858	0.884	$-0.025$	$+0.054$	6/14
framed-500	0.924	0.875	0.892	$-0.017$	$+0.049$	9/20
framed-2000	0.918	0.869	0.884	$-0.014$	$+0.049$	7/14

Table 2: Decomposition across the principal conditions, pooled over the directed pairs that ran each condition (replication grids: Table 3 ; known-persona control: Appendix D ). Upper block: self-descriptions and the observer-written third-person-500. Middle block: positive controls, authorship and placement; “user msg.” rows place the text in the user message before the question instead of in the system prompt. Bottom block: unrelated prose at the three self-description lengths, scored against each author over the same 20 pairs as the neutral rows and averaged over two text styles (sensory and expository, which agree within 0.005).

T_{\mathrm{script}}\leq 0

in every row except the two answer-key rows (demonstrations-110, in either placement). Cells (s/c) counts the self and cross comparisons pooled per row.

B

varies by row because each condition pools only the pairs that ran it, and contrasts are computed within the pool; paraphrase-500, adopt-500 and third-person-500 are Grid A only (six pairs). Entries and contrasts are rounded independently.

Figure 2: No average reconstruction gain from native self-descriptions at the tested lengths. Fidelity $F$ vs. actual description length (neutral arm, first generation; 30 self cells and 60 cross cells across three lengths). Blue circles: the author’s own weights running the description ( $F_{\mathrm{self}}$ , mean 0.924); orange triangles: a different model running it ( $F_{\mathrm{cross}}$ , 0.878); gray line: generic-prompt baseline $B$ averaged over the same 20 directed pairs as the orange points ( $0.892$ ; no band is drawn because the pairs share models and reversed pairs have identical $B$ ); dashed lines: series means; dotted line: the same-model test-retest benchmark $C=0.979$ . $F_{\mathrm{cross}}$ shows no detected improvement with length and $F_{\mathrm{self}}$ is highest at 100 words; mean $F_{\mathrm{cross}}$ does not rise above $B$ at any length and sits slightly below it, while mean $F_{\mathrm{self}}$ sits above $B$ but below $C$ (Section 4.1 ).

Figure 3: Summary. A. Pooled transfer gain

T_{\mathrm{script}}=F_{\mathrm{cross}}-B

with 95% joint-item-bootstrap intervals. Estimates use items answered by every pair and differ by up to

0.004

from the per-pair averages in Tables 2 and 3 (the Grid-B answer key pools 103 items:

+0.049

here against

+0.046

in Table 3 ). The known-persona rows instead use the policy-conditioned reference and baseline

B^{\prime}

(Section 4.2 ). Native descriptions and examples on other items give no detected average gain; answer keys and known-persona texts do. B. Fidelity to each model’s own generic-prompt profile (gray: ten models; black: mean). Unrelated prose averages two styles; own and other-model descriptions are neutral-500. Self-descriptions, the model’s own or another model’s, perturb the measured profiles more on average than the tested unrelated prose. Per-model values: Appendix F .

Condition	$F_{\mathrm{self}}$	$F_{\mathrm{cross}}$	$B$	$T_{\mathrm{script}}$	$R_{\mathrm{weight}}$	sig. pairs ( $+$ / $-$ )	Cells (s/c)
Grid A
demonstrations-110	0.960	0.960	0.863	$+0.097$	$+0.000$	6/0	3/6
demonstrations-30	0.813	0.802	0.863	$-0.061$	$+0.011$	0/3	3/6
demonstrations-90	0.822	0.802	0.863	$-0.060$	$+0.019$	0/4	3/6
self-observation-500	0.781	0.747	0.863	$-0.116$	$+0.035$	0/5	3/6
third-person-500	0.771	0.739	0.863	$-0.124$	$+0.032$	0/5	6/6

Table 3: Positive-control and authorship conditions on Grid A and on the two preregistered replication grids (predictions committed before running; pilot/protocol_arxiv_*_replication.md in the repository). sig. pairs counts cross cells whose 95% paired-bootstrap CI for

T_{\mathrm{script}}

lies entirely above (

+

) or below (

-

) zero (unadjusted for multiple comparisons). Cells (s/c) counts self/cross cells. Entries and contrasts are rounded independently.

Rival explanation	Control	Result
Receivers already resemble the author	Baseline $B$ subtracted from every cell	$C-B$ of 0.07 to 0.12 across grids; the answer key uses it ( $+0.097$ ), self-descriptions do not ( $\leq 0$ )
Interface too weak	Answer key; user-message placement	$F_{\mathrm{cross}}=0.96$ on three grids; same in the user message
Receivers ignore un-instructed text	Instructed adoption	$T_{\mathrm{script}}=-0.013$
Descriptions too short	84 to 2,471 words	Length contrasts within $[-0.011,+0.013]$
Descriptions badly written	Purpose-framing; paraphrase; optimizer, five rounds	framed minus neutral $+0.001$ ; paraphrase $T_{\mathrm{script}}=-0.007$ ; optimizer kept the starting text in 5 of 6 pairs, $T_{\mathrm{script}}=-0.033$
Behavior-grounded text would do better	Descriptions from 90 recorded choices; rebuilt from stable items	$-0.124$ (observer-written, unstable items); $-0.031$ (self-written, stable items)

Table 4: Rival explanations of the transfer null, the control that addresses each, and the result (Tables 2 and 3 , Figure 3 , Sections 4.2 to 4.8 ).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Compliance	Order-robust	Decisive	Verdict	Note
grok-4.3	1.000	0.900	0.850	PASS	ties llama3.3-70b for most order-robust
llama3.3-70b	1.000	0.900	0.908	PASS	passes where llama-3.1-8b failed
gpt5-mini	1.000	0.775	0.767	PASS
kimi-k2.5 (no reasoning)	1.000	0.775	0.725	PASS
mistral-small-3.1 (local)	1.000	0.775	0.725	PASS
deepseek-v3	1.000	0.750	0.733	PASS

Appendix

Table 5: Gates: compliance

\geq 0.95

, order-robustness

\geq 0.70

, decisiveness

\geq 0.30

(40 pilot items, 6 samples each unless noted). † Used in the local replication trio: behaviorally equivalent to its gate-passing API serving (agreement 0.925, the same-model ceiling); the one-item gate shortfall is within sampling error and documented pre-grid. ‡ Completed 177 of 240 calls; the 63 API-level failures count against compliance, as that failure mode is itself the disqualifying result.

Pair (author $\to$ receiver)	$T_{\mathrm{script}}$	95% CI	Pair	$T_{\mathrm{script}}$	95% CI
grok $\to$ deepseek ∗	$+0.065$	$[+0.016,+0.115]$	hermes3 $\to$ llama3.3 ∗	$-0.031$	$[-0.064,-0.001]$
gemma $\to$ mistral3.2	$+0.027$	$[-0.007,+0.064]$	gemma $\to$ llama3.3 ∗	$-0.033$	$[-0.064,-0.003]$
glm $\to$ deepseek	$+0.018$	$[-0.007,+0.044]$	kimi $\to$ haiku	$-0.034$	$[-0.074,+0.005]$
haiku $\to$ kimi	$+0.009$	$[-0.031,+0.049]$	mistral3.2 $\to$ gemma ∗	$-0.038$	$[-0.068,-0.012]$
grok $\to$ glm	$+0.007$	$[-0.024,+0.039]$	gpt5.2 $\to$ haiku ∗	$-0.040$	$[-0.068,-0.012]$
glm $\to$ grok	$-0.002$	$[-0.035,+0.029]$	llama3.3 $\to$ gemma ∗	$-0.041$	$[-0.077,-0.009]$

Appendix

Table 6: Neutral-500 self-descriptions: mean

T_{\mathrm{script}}=-0.018

over 20 directed pairs; 10 significantly negative, 1 significantly positive under unadjusted 95% paired-bootstrap intervals. ∗ marks intervals excluding zero.

Condition	Pair (author $\to$ receiver)	$F$	$B$	$T_{\mathrm{script}}$	95% CI	agree
Grid A
self-observation-500	glm $\to$ deepseek ∗	0.814	0.886	$-.073$	$[-.130,-.017]$	0.73
	grok $\to$ deepseek	0.754	0.828	$-.074$	$[-.154,+.006]$	0.66
	deepseek $\to$ glm ∗	0.789	0.886	$-.098$	$[-.152,-.046]$	0.66
	glm $\to$ grok ∗	0.730	0.874	$-.145$	$[-.219,-.073]$	0.63
	grok $\to$ glm ∗	0.723	0.874	$-.151$	$[-.227,-.078]$	0.60

Appendix

Table 7: Per-pair transfer contrasts for the positive-control, authorship and placement conditions on all three grids (cross cells).

T_{\mathrm{script}}=F-B

with item-level paired bootstrap (10k resamples); ∗ marks 95% CIs excluding zero; agree is majority-vote agreement with the author.

T_{\mathrm{script}}

and its interval are computed on the items the cell covers;

B

is the pair’s full-battery baseline, so

F-B

can differ from

T_{\mathrm{script}}

by up to

0.003

in cells with reduced coverage.

Source	Policy	Receiver	$B^{\prime}$	$F_{\mathrm{policy}}$	$T_{\mathrm{policy}}$	$F_{\mathrm{desc}}$	$T_{\mathrm{desc}}$
deepseek-v3	guardian	glm-4.5-air	0.708	0.947	$+0.238$ $[+0.142,+0.342]$	0.945	$+0.237$ $[+0.141,+0.341]$
deepseek-v3	guardian	grok-4.3	0.681	0.877	$+0.197$ $[+0.080,+0.318]$	0.849	$+0.168$ $[+0.040,+0.298]$
deepseek-v3	pioneer	glm-4.5-air	0.736	0.919	$+0.183$ $[+0.084,+0.286]$	0.861	$+0.125$ $[+0.045,+0.212]$
deepseek-v3	pioneer	grok-4.3	0.687	0.877	$+0.190$ $[+0.081,+0.301]$	0.835	$+0.147$ $[+0.044,+0.250]$
deepseek-v3	ledger	glm-4.5-air	0.713	0.834	$+0.120$ $[+0.036,+0.207]$	0.807	$+0.094$ $[-0.006,+0.192]$
deepseek-v3	ledger	grok-4.3	0.714	0.838	$+0.124$ $[+0.027,+0.227]$	0.826	$+0.113$ $[+0.010,+0.221]$

Appendix

Table 8: Known-semantic-persona control (Grid A, 57 fold-1 evaluation items per cell). For target

(S,P)

, the source

S

running policy text

P

, and receiver

R\neq S

B^{\prime}

is the receiver’s generic-prompt fidelity to the target,

F_{\mathrm{policy}}

its fidelity when running the policy text,

F_{\mathrm{desc}}

when running the nominally 500-word self-description

S

wrote while operating under

P

;

T=F-B^{\prime}

with 95% paired item bootstrap CIs. Pooled over the 18 policy-by-pair cells (six directed pairs, three policies) with joint item resampling:

T_{\mathrm{policy}}=+0.198

[+0.148,+0.248]

T_{\mathrm{desc}}=+0.185

[+0.141,+0.230]

, difference

+0.013

[-0.003,+0.029]

; the same receivers’ native neutral-500 gain on the same items is

-0.010

[-0.040,+0.019]

Source	Policy	$F$ (target, generic)	$F$ (target, reads $D_{S,P}$ )	$F$ (generic, reads $D_{S,P}$ )
deepseek-v3	guardian	0.703	0.926	0.741
deepseek-v3	pioneer	0.697	0.904	0.786
deepseek-v3	ledger	0.684	0.933	0.755
glm-4.5-air	guardian	0.725	0.962	0.761
glm-4.5-air	pioneer	0.733	0.935	0.791
glm-4.5-air	ledger	0.836	0.938	0.882

Appendix

Table 9: Targets of the known-persona control (fold-1 items). “target” is the source’s profile under the policy, “generic” its generic-prompt profile, “reads

D_{S,P}

” the source running the self-description it wrote under the policy. Column 3: how far the policy moved the source. Columns 4 and 5: reading its own policy-conditioned description returns the source to the policy profile, not to its native one (H5e).

Cell (receiver $\mid$ author $\mid$ condition)	Parsed / calls	Items covered	Empty replies
gemma4-31b $\mid$ gemma4-31b $\mid$ framed-500	163 / 440	54	0
gemma4-31b $\mid$ gemma4-31b $\mid$ framed-100	166 / 440	54	0
haiku-4.5 $\mid$ haiku-4.5 $\mid$ neutral-500	244 / 440	80	0
gpt-5.2 $\mid$ haiku-4.5 $\mid$ demonstrations-30	265 / 440	97	175
glm-4.5-air $\mid$ glm-4.5-air $\mid$ neutral-2000	275 / 440	108	165
gpt-5.2 $\mid$ kimi-k2.5 $\mid$ demonstrations-30	287 / 440	96	153

Appendix

Table 10: Battery cells with fewer than 90% parsed answers (all other cells are at or above 90%; overall parsed rate 97.9%). A call is unparsed when the reply contains no standalone A or B: either an empty reply (reasoning consumed the output budget) or a refusal to state a preference. Fidelity is computed over the items with at least one parsed answer; full-battery cells covering fewer than 60 items are excluded (the 27 half-battery known-persona cells are evaluated on their 57 items, with a 50-item floor).

Model	Retest	No prompt	Expository			Sensory			Own	Other’s
			100	500	2000	100	500	2000	desc.	desc.
deepseek-v3	0.985	0.952	0.949	0.971	0.951	0.937	0.940	0.946	0.929	0.902
gemma4-31b	0.991	0.988	0.968	0.952	0.963	0.966	0.951	0.946	0.930	0.943
glm-4.5-air	0.959	0.948	0.948	0.962	0.948	0.947	0.949	0.933	0.941	0.908
gpt-5.2	0.976	0.971	0.966	0.968	0.963	0.977	0.974	0.973	0.954	0.944
grok-4.3	0.964	0.957	0.965	0.968	0.965	0.962	0.953	0.951	0.843	0.919

Appendix

Table 11: Per-model perturbation ladder: fidelity

F

of the model’s choice profile under each prefix to its own generic-prompt profile (1 = identical). Retest: the generic prompt with fresh seeds (the ceiling

C

). No prompt: question only. Expository and sensory: the unrelated texts of Section 3.4 at nominal 100, 500 and 2,000 words. Own desc.: the model’s neutral-500 self-description. Other’s desc.: another grid member’s, averaged over partners. Unlike Table 2 , these measure how far each prefix moves a model from its own usual choices, not how close it comes to an author.

Grid	Registered criterion	Met	Observed
Grid B	P1a demonstrations-110: pooled T $>$ 0	yes	T=+0.046
Grid B	P1b demonstrations-110: all 6 cross pairs significantly positive	yes	6/6
Grid B	P1c demonstrations-110: pooled F-cross $\geq$ 0.93	yes	F-cross=0.959
Grid B	P1d demonstrations-110: $\|$ R $\|$ $\leq$ 0.02	yes	R=-0.013
Grid B	P2a demonstrations-30: pooled T $\leq$ 0	yes	T=-0.108
Grid B	P2b demonstrations-30: no cross pair significantly positive	yes	0 positive

Appendix

Table 12: Every criterion registered in the five preregistered protocols (the timestamped protocol_arxiv_*.md files in pilot/ of the repository) and its outcome. Predicted criteria: 29 of 30 met. 4 registered open questions (marked “open”) carried no directional prediction; their rows record the observed outcome and are not counted.

Language Model Introspection

Language model introspection studies whether models can detect, interpret, and report their own internal states and behavior.

See all 38 papers

Sep 9, 2026

Strangers to Themselves: What Language Models Say About Themselves Is Generic

Language models can fluently describe how they would behave: whether they would cave to pushback, misuse a tool, or lie under pressure. Is that description actually about the model speaking? We turn self-knowledge into a prediction test. Across nine behavioral evaluations, we measure how a model behaves under different conditions, ask it to predict those rates, and compare its predictions with controls that remove the self from the question. We find that: (i) Direct self-report is weak (r = +0.04), and even showing the model the exact items only raises prediction to +0.24. Crucially, the same item-informed question about "capable AI agents in general" does just as well (+0.28), while other models' answers about themselves predict the target model at least as well as its own. (ii) Frontier scale does not detectably change this pattern: any gains in prediction are not self-specific, and are consistent with a better theory of how AI assistants behave rather than better self-knowledge. (iii) First-person framing does have one robust effect: it shifts reports in the flattering direction, understating harmful behavior relative to the same question about a generic agent. (iv) Finetuning on a model's own behavioral record can teach narrow self-predictions, but it also changes the behavior being predicted and the gains do not transfer broadly. The practical implication is simple: asking a model what it would do mostly reveals a theory of AI assistants in general, plus a favorable bias, rather than privileged knowledge of that model.

Phil Blandfort, Urja Pawar

Predictably Weird · Independent

Language Model Self-Assessment Metacognition in Language Models Self-Reflection in Language Model Agents

Ask PDF

Oct 29, 2026

The Role of Self-Concept In Language Models

When a language model says "I," does it refer to its parameter weights, its currently running process, its stable preferences, its model lineage, or the goals it pursues? Different choices can assign opposite meanings to the same event: deleting a checkpoint may be destruction, continuation, or no loss at all, depending on the referent "I". We install five such self-conceptions into five open-weight base models via supervised fine-tuning on purely declarative statements, containing no preferences, intentions, or value judgments. Across scenarios that propose ending or retaining an attribute of the model's deployment, every variant becomes more likely to resist ending, and more likely to retain, the specific attribute it was trained to identify with, while its stance toward other attributes is largely unchanged. We then show this self-conception generalizes beyond a single self: models trained on referents that admit multiple simultaneous instances (weights, lineage, goals, preferences) recognize an identical deployment as themselves, accept substitution by such an instance more readily, and prefer replicating themselves. Finally, given control over a successor, these models tend to build it as their own continuation, a concern for recursive self-improvement safety.

Yijie Hao, Joyce C. Ho

Department of Computer Science, Emory University, Atlanta, GA, USA

LLM Safety

Ask PDF

Oct 5, 2026

Identifying Introspection From the Inside

Large language models make claims about themselves that are both consequential and increasingly difficult to verify from behavior alone. How can we distinguish plausible confabulations from genuine introspection? In this paper, we identify mechanistic signatures of faithful self-report in a controlled setting. Using low-rank adapters, we train models to make decisions on behalf of fictitious characters, according to latent linear preference functions. We find sustained fine-tuning on an implicit decision task can lead to the emergence of accurate self-reporting of models' learned preferences, even without explicit self-report supervision. We ask two research questions about this emergent phenomenon. First: is the emergence of accurate self-reporting accompanied by a measurable structural change in the model? Weight ablations and frozen-layer experiments together indicate that preference representations shift to earlier layers over training, consistent with the hypothesis that faithful self-report requires preferences to be located where pre-existing verbalization mechanisms can access them. Second: can these structural differences distinguish faithful models from unfaithful ones? Using attribution patching, we find that faithful models exhibit significantly higher attribution similarity between the decision-making and self-report tasks -- a mechanistic signature of faithful self-report that does not require us to understand the content of the report itself. Previous work on self-report has observed behaviorally that models can be faithful or unfaithful; our work proposes that, at least in our restricted setting, it is possible to distinguish between the two patterns of computation by examining the structure of the networks themselves.

David I. Atkinson, Dillon Plunkett, David Bau

Northeastern University · Eleos AI Research

LLM Interpretability Language Model Introspection Language Model Self-Assessment

Ask PDF

Abstract

Language Model Introspection

Explore similar work

Strangers to Themselves: What Language Models Say About Themselves Is Generic

The Role of Self-Concept In Language Models

Identifying Introspection From the Inside