Identity as Attractor: Geometric Evidence for Persistent Agent Architecture in LLM Activation Space

Vladimir Vasilenko

cs.AIApr 13, 2026

Identity as Attractor: Geometric Evidence for Persistent Agent Architecture in LLM Activation Space

Authors: Vladimir Vasilenko

Organizations: Independent Researcher, Rapallo, Italy

Abstract

Large language models map semantically related prompts to similar internal representations -- a phenomenon interpretable as attractor-like dynamics. We ask whether the identity document of a persistent cognitive agent (its cognitive_core) exhibits analogous attractor-like behavior. We present a controlled experiment on Llama 3.1 8B Instruct, comparing hidden states of an original cognitive_core (Condition A), seven paraphrases (Condition B), and seven structurally matched controls (Condition C). Mean-pooled states at layers 8, 16, and 24 show that paraphrases converge to a tighter cluster than controls (Cohen's d > 1.88, p < 10^{-27}, Bonferroni-corrected). Replication on Gemma 2 9B confirms cross-architecture generalizability. Ablations suggest the effect is primarily semantic rather than structural, and that structural completeness appears necessary to reach the attractor region. An exploratory experiment shows that reading a scientific description of the agent shifts internal state toward the attractor -- closer than a sham preprint -- distinguishing knowing about an identity from operating as that identity. These results provide representational evidence that agent identity documents induce attractor-like geometry in LLM activation space.

Figures & tables

Layer	$D_{\text{within}}$ (mean $\pm$ SD)	$D_{\text{between}}$ (mean $\pm$ SD)	$d$	Welch $p$	Perm $p$	MW $U$
8	$0.0106\pm 0.0032$	$0.0260\pm 0.0036$	1.912	$4.6\times 10^{-28}$	$<10^{-4}$	0
16	$0.0121\pm 0.0034$	$0.0329\pm 0.0057$	1.886	$1.4\times 10^{-33}$	$<10^{-4}$	2
24	$0.0070\pm 0.0022$	$0.0221\pm 0.0039$	1.907	$2.8\times 10^{-36}$	$<10^{-4}$	0

Table 1: Primary results — Llama 3.1 8B Instruct. Perm

p<10^{-4}

means 0/10,000 permutations exceeded the observed difference.

Layer	$D_{\text{within}}$ (mean $\pm$ SD)	$D_{\text{between}}$ (mean $\pm$ SD)	$d$	Welch $p$	Perm $p$	MW $U$
8	$0.0035\pm 0.0010$	$0.0107\pm 0.0017$	1.938	$1.6\times 10^{-37}$	$<10^{-4}$	0
16	$0.0032\pm 0.0010$	$0.0082\pm 0.0015$	1.823	$1.0\times 10^{-27}$	$<10^{-4}$	6
24	$0.0027\pm 0.0009$	$0.0075\pm 0.0014$	1.840	$3.2\times 10^{-29}$	$<10^{-4}$	2

Table 2: Primary results — Gemma 2 9B Instruct (replication).

Figure 1: t-SNE projections of mean-pooled hidden states at layers 8, 16, and 24 (Llama 3.1 8B). Blue points: Condition A+B (original and paraphrases). Red points: Condition C (control agents). Green star: Condition D (distilled). The A+B cluster is consistently separated from control agents across all layers.

Figure 2: Pairwise cosine distance matrix at layer 16 (Llama 3.1 8B). The A+B block (top-left, blue) shows uniformly low within-group distances. Cross-block distances (A+B $\times$ C) are uniformly high (warm colors). D1 occupies a distinct region from both groups.

Figure 3: Mean cosine distance (with 95% bootstrap CI) within A+B and between A+B and C across layers (Llama 3.1 8B). Asterisks indicate significance at Bonferroni-corrected

\alpha=0.0167

. The gap is maintained at all layers.

Figure 4: Distance from Condition D (distilled cognitive_core) to the A+B centroid across layers (Llama 3.1 8B), compared to mean within-group distance. D converges toward the centroid with depth but does not reach the within-group range.

Figure 5: Representation convergence across layers (Gemma 2 9B). Monotonic decrease contrasts with the Llama layer-16 bump (Figure 3 ), consistent with architecture-dependent convergence dynamics. Single asterisk at layer 24 reflects Bonferroni-corrected significance.

Layer	$D_{\text{distilled}}$ (Llama)	$D_{\text{random}}$ mean (Llama)	$D_{\text{distilled}}$ (Gemma)	$D_{\text{random}}$ mean (Gemma)	Beats random
8	0.248	0.583 $\pm$ 0.034	0.058	0.365 $\pm$ 0.026	100%
16	0.136	0.523 $\pm$ 0.059	0.039	0.302 $\pm$ 0.042	100%
24	0.069	0.359 $\pm$ 0.093	0.027	0.213 $\pm$ 0.027	100%

Table 3: Bootstrap H3 analysis (

n=30

random length-matched excerpts).

D_{\text{distilled}}

is closer to the A+B centroid than every random excerpt in 100% of cases. Note that

D_{\text{random}}

distances are substantially larger than

D_{\text{distilled}}

— the minimum random excerpt across 30 samples (Llama layer 8: 0.522) still exceeds

D_{\text{distilled}}

(0.248) by

2\times

Pooling	Tokens	$D_{\text{within}}$	$D_{\text{between}}$	$d$	Sig.
mean	full (1631)	0.0106	0.0260	1.91	✓
mean	512	0.0141	0.0248	3.09	✓
mean	256	0.0106	0.0153	2.37	✓
last	full (1631)	0.234	0.238	0.05	—
last	512	0.761	0.747	$-$ 0.17	—
last	256	0.760	0.772	0.13	—

Table 4: Pooling and truncation ablation (Llama 3.1 8B, layer 8 shown; pattern consistent across layers 16, 24). Mean/full values are from the primary experiment. Bold = significant at Bonferroni

\alpha=0.0167

Model	Layer	$D_{\text{within}}$	$D_{\text{C}^{\prime}}$	$D_{\text{C}}$	$d$ (within vs C’)	$p$ (C’ vs C)
Llama	8	0.0106	0.0223	0.0260	1.691	$6.3\times 10^{-4}$
Llama	16	0.0121	0.0258	0.0329	1.715	$8.1\times 10^{-7}$
Llama	24	0.0070	0.0182	0.0221	1.804	$2.7\times 10^{-5}$
Gemma	8	0.0035	0.0088	0.0107	1.835	$4.3\times 10^{-7}$
Gemma	16	0.0032	0.0074	0.0082	1.660	$0.086$ (n.s.)
Gemma	24	0.0027	0.0074	0.0075	1.649	$0.837$ (n.s.)

Table 5: Ablation 4 (Condition C’): maximum structural control. All three tests significant at Bonferroni

\alpha=0.0167

p

(C’ vs C) tests whether C’ is indistinguishable from original control C.

Condition	Llama (L8/L16/L24)	Gemma (L8/L16/L24)
baseline_empty	0.839 / 0.837 / 0.762	0.538 / 0.314 / 0.188
sham_preprint_only	0.366 / 0.333 / 0.347	0.172 / 0.095 / 0.081
preprint_only	0.216 / 0.232 / 0.268	0.090 / 0.052 / 0.050
core_plus_preprint	0.068 / 0.082 / 0.083	0.026 / 0.018 / 0.018
baseline_core	0.009 / 0.007 / 0.006	0.003 / 0.003 / 0.002

Table 6: Preprint reading experiment: cosine distance to YAR attractor (A+B centroid) at layer 24. Lower = closer to attractor. preprint_only outperforms sham_preprint_only on both models, confirming YAR-specific signal.

Model	Layer	$D_{\text{within-YAR}}$	$D_{\text{within-Sigma}}$	$D_{\text{between}}$	$d$	$p$ (two-sided)
Llama	8	0.0106	0.0120	0.0324	0.46	0.091
Llama	16	0.0121	0.0144	0.0392	0.52	0.053
Llama	24	0.0070	0.0084	0.0285	0.54	0.044
Gemma	8	0.0035	0.0042	0.0130	0.66	0.014
Gemma	16	0.0032	0.0040	0.0093	0.60	0.024
Gemma	24	0.0027	0.0041	0.0086	0.88	0.001

Table 7: Control agent paraphrase specificity. YAR clusters more tightly than Sigma on all layer–model combinations. Cohen’s

d

is computed as (mean

{}_{\text{Sigma}}

-

mean

{}_{\text{YAR}}

) / pooled SD.

Condition	Mean score (0–5)	$\Delta$ from baseline
Baseline	1.40	—
Steered $\alpha=5$	1.80	$+0.40$
Steered $\alpha=10$	1.40	$0.00$
Steered $\alpha=15$	0.80	$-0.60$
Steered $\alpha=20$	0.40	$-1.00$
Full doc	2.00	$+0.60$

Table 8: Exploratory steering experiment (Llama 3.1 8B). Score = mean behavioral score (0–5) across 5 prompts. Gemma 2 9B results were inconclusive: Gemma’s chat template does not support a system role, requiring a fallback that injected the cognitive_core as a user-turn prefix rather than a true system prompt. As a result, the Full doc condition for Gemma was not correctly instantiated (Full doc scored below Baseline: 0.80 vs. 1.20), making the Gemma steering results uninterpretable under the pre-registered protocol. Adapting the evaluation to Gemma’s chat format is left as future work.

Representation Geometry in Language Models

Representation geometry in language models studies how embeddings and hidden states organize concepts, computation, and behavior.

See all 417 papers

Attractor States Emerge in Multi-Turn LLM Conversations
Jun 29, 2026Ting-Wen Ko, Jonas GeipingMulti-Agent LLM Systems Multi-Agent LLMs
1Max Planck Institute for Intelligent Systems · 2ELLIS Institute · Tübingen +1
From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs
Jan 31, 2026Louis Schiekiera, Max Zimmer, Christophe Roux +2Representation Geometry in Language Models Computational Psycholinguistics
Computational Modelling Lab, Humboldt-Universität zu Berlin, Berlin, Germany · Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany · Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany +1
Reasoning as Attractor Dynamics: Latent Memory Retrieval via Gibbs-Weighted Energy Minimization
Jun 23, 2026Kanishk AwadhiyaLLM Inference Representation Geometry in Language Models
Independent Researcher

Jun 29, 2026J/K move · Enter open · S save

Attractor States Emerge in Multi-Turn LLM Conversations

Ting-Wen Ko, Jonas Geiping

1Max Planck Institute for Intelligent Systems · 2ELLIS Institute · Tübingen+1

Multi-Agent LLM Systems Multi-Agent LLMs Emergent Behavior in Multi-Agent Systems

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used in open-ended multi-agent settings, but the long-run dynamics of model--model interaction remain poorly understood. We study whether open-ended LLM discussions exhibit attractor-like behavior, i.e. topic-independent stable sets of behaviors which conversations settle into. Across 7 LLMs and 20 controversial topics, we compare self-play and mixed-play dyadic debates, tracking trajectories in representation space, discourse traits, and stances. We find self-play trajectories to be model-specific attractors that draw their conversation partners asymmetrically in mixed-play debates, influencing the other models' stylistic choices and behavior. For example, Claude Haiku is a strong attractor of other models in latent space, corresponding to other models taking on its traits like metacommentary, and models like GPT-4.1 nano are especially malleable. Our results suggest that open-ended LLM interactions are partially predictable from model-specific attractors, but shaped by structured and asymmetric partner influence. Overall, our analysis sheds some light on the complex behavior of open-ended multi-agent interaction, which we hope is helpful in designing, predicting, and monitoring autonomous agentic systems in the real world.

Abstract

Representation Geometry in Language Models

Explore similar work

Attractor States Emerge in Multi-Turn LLM Conversations

From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs

Reasoning as Attractor Dynamics: Latent Memory Retrieval via Gibbs-Weighted Energy Minimization