From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs

Louis Schiekiera; Max Zimmer; Christophe Roux; Sebastian Pokutta; Fritz Günther

cs.LGJan 31, 2026

From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs

Authors: Louis Schiekiera, Max Zimmer, Christophe Roux, Sebastian Pokutta, Fritz Günther

Organizations: Computational Modelling Lab, Humboldt-Universität zu Berlin, Berlin, Germany · Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany · Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany · Institute of Mathematics, Technische Universitat Berlin, Germany

Abstract

We investigate the extent to which an LLM's hidden-state geometry can be recovered from its behavior in psycholinguistic experiments. Across eight instruction-tuned transformer models, we run two experimental paradigms -- similarity-based forced choice and free association -- over a shared 5,000-word vocabulary, collecting 17.5M+ trials to build behavior-based similarity matrices. Using representational similarity analysis, we compare behavioral geometries to layerwise hidden-state similarity and benchmark against FastText, BERT, and cross-model consensus. We find that forced-choice behavior aligns substantially more with hidden-state geometry than free association. In a held-out-words regression, behavioral similarity (especially forced choice) predicts unseen hidden-state similarities beyond lexical baselines and cross-model consensus, indicating that behavior-only measurements retain recoverable information about internal semantic geometry. Finally, we discuss implications for the ability of behavioral tasks to uncover hidden cognitive states.

Figures & tables

Figure 1: Conceptual overview. For a shared vocabulary

\mathcal{V}

, we (i) extract layer-

\ell

word representations to form a hidden-state similarity matrix

\mathbf{S}^{\mathrm{hid}}_{\ell}

, and (ii) run behavioral association tasks (forced choice/free association) to build a cue–response matrix

\mathbf{B}

and behavioral similarity

\mathbf{S}^{\mathrm{beh}}

. RSA correlates the pairwise similarities in

\mathbf{S}^{\mathrm{hid}}_{\ell}

and

\mathbf{S}^{\mathrm{beh}}

to quantify behavior–activation alignment.

Figure 2: Behavioral paradigms and derived semantic geometries. Left (forced choice): given a cue word

w_{i}

and a candidate set

c_{i}

, the model selects a fixed number of output words

o_{i}

, producing a cue–response count matrix

\mathbf{B}^{\mathrm{FC}}

. Right (free association): given

w_{i}

alone, the model generates multiple output words

o_{i}

, yielding

\mathbf{B}^{\mathrm{FA}}

. From the count matrix, we produce similarity matrices

\mathbf{S}^{\mathrm{FC}}

and

\mathbf{S}^{\mathrm{FA}}

by cosine similarity between rows. The diagram shows

|c_{i}|=4

for FC and

|o_{i}|=4

for FA, while our experiments use

|c_{i}|=16

for FC and

|o_{i}|=5

for FA.

Model	$n_{\text{params}}$	$n_{\text{layers}}$	$d_{\text{model}}$
Falcon3-10B-Instruct	10B	40	3072
gemma-2-9b-it	9B	42	3584
Llama-3.1-8B-Instruct	8B	32	4096
Mistral-7B-Instruct-v0.2	7B	32	4096
Mistral-Nemo-Instruct-2407	12B	40	5120
phi-4	14B	40	5120

Table 1: Model specifications.

n_{\text{params}}

= number of parameters in billions (B);

n_{\text{layers}}

= number of layers;

d_{\text{ model}}

= hidden-state dimension width. HuggingFace Model IDs are reported in Appendix B .

Paradigm	$\mathrm{T}_{\mathrm{total}}$	$\mathrm{T}_{\mathrm{m}}$	$\mathrm{T}_{\mathrm{w}}$	$\mathrm{T}_{\mathrm{w{+}m}}$
Forced choice	12.52M	1.565M	2504	313
Free association	5.04M	0.630M	1008	126

Table 2: Data collection statistics for the two behavioral paradigms across eight models and a shared vocabulary of 5,000 words.

\mathrm{T}_{\mathrm{total}}

= total number of trials,

\mathrm{T}_{\mathrm{m}}

= per model,

\mathrm{T}_{\mathrm{w}}

= per input word, and

\mathrm{T}_{\mathrm{w{+}m}}

= per model and word.

Panel of Figure 3 — Figure 3: Summary of RSA and neighborhood-overlap results (means across models). Panel a (left) compares multiple reference geometries: (a1) mean RSA Pearson correlation as a function of layer, and (a2) mean nearest-neighbor overlap (NN@ $k$ ) as a function of neighborhood size $k$ (log scale). Panel b (right) focuses on behavioral references and compares extraction strategies: (b1) layerwise RSA for PPMI-weighted forced-choice similarity $\mathbf{S}^{\mathrm{FC}}$ and (b2) layerwise RSA for PPMI-weighted free-association similarity $\mathbf{S}^{\mathrm{FA}}$ .

Figure 4: Representational similarity analysis between model hidden-state similarity and behavior-derived semantic geometries. Each panel corresponds to a model and contains two sub-heatmaps comparing hidden-state similarity to PPMI-weighted forced-choice ( $\mathbf{S}^{\mathrm{FC}}$ , left) and PPMI-weighted free-association ( $\mathbf{S}^{\mathrm{FA}}$ , right) behavioral embeddings. Rows indicate the embedding extraction strategy (Averaged, Meaning, Task (FC), Task (FA)), and columns indicate layerwise correlations (min, max, mean across layers).

Figure 5: Ridge regression performance for predicting hidden-state similarity from behavioral and lexical features across eight models. Bold values show $R^{2}$ for the full model (behavioral+FastText+BERT+cross-model consensus); parenthetical values show the FastText+BERT+cross-model consensus baseline. Rows indicate the embedding extraction strategy (Averaged, Meaning, Task (FC), Task (FA)), and columns indicate layerwise correlations (min, max, mean across layers).

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Model ID (Hugging Face)	Citation	Params	L	$d_{\text{model}}$
tiiuae/Falcon3-10B-Instruct	( TII Team, 2024 )	10B	40	3072
google/gemma-2-9b-it	( Gemma Team, 2024 )	9B	42	3584
meta-llama/Meta-Llama-3.1-8B-Instruct	( Meta AI, 2024 )	8B	32	4096
mistralai/Mistral-7B-Instruct-v0.2	( Jiang et al., 2023 )	7B	32	4096
mistralai/Mistral-Nemo-Instruct-2407	( Mistral AI, 2024 )	12B	40	5120
microsoft/phi-4	( Abdin et al., 2024 )	14B	40	5120

Appendix

Table 3: Models used in this study. Params = number of parameters in billions; L = number of layers;

d_{\text{model}}

= hidden dimension size.

Model abbreviation	Initial compliance (%)	Final compliance (%)
All models	85.3	96.2
Falcon3-10B-Instruct	89.1	96.3
gemma2-9b-it	90.1	96.3
Llama3.1-8B-Instruct	79.0	96.3
Mistral7B-Instruct-v0.2	81.8	86.8
MistralNemo-Instruct-2407	92.9	98.8

Appendix

Table 4: Forced choice: Summary statistics for compliance and repair across models.

Model	Cue repetition (%)	Unique words (total)	M unique per cue
All models	3.2	18,621	21.81
gemma-2-9b-it	0.2	12,231	14.40
Mistral-Nemo-Instruct-2407	3.9	18,049	24.26
phi-4	2.6	17,936	20.29
rnj-1-instruct	4.8	32,203	31.90
Qwen2.5-7B-Instruct	3.9	17,395	18.44

Appendix

Table 5: Free association: Overall quality report for free association outputs. Cue repetition is the percentage of response trials (not associations) that contain the input cue word as an output word. Unique words (total) is the total number of unique words in all responses. M unique per cue is the mean number of unique words per cue.

	Forced choice			Free association
Model	$M$	$\%$	$\sum$	$M$	$\%$	$\sum$
All models	610.1	97.5%	24,405,587	622.6	98.8%	24,905,314
Falcon3-10B-Instruct	614.0	98.1%	3,070,010	625.3	99.3%	3,126,269
Llama-3.1-8B-Instruct	611.7	97.7%	3,058,606	603.8	95.8%	3,019,162
Mistral-7B-Instruct-v0.2	562.9	89.9%	2,814,614	627.2	99.6%	3,135,754
Mistral-Nemo-Instruct-2407	621.1	99.2%	3,105,303	624.7	99.2%	3,123,447

Appendix

Table 6: Usable associations from behavioral paradigms. Postprocessing summaries from the cue–response counts matrices for forced choice and free association.

Figure 6: Mean Pearson correlation between behavioral semantic spaces and model hidden states as a function of SVD dimensionality reduction applied to behavioral matrices. Left: Forced choice (

S^{\mathrm{FC}}

). Right: Free association (

S^{\mathrm{FA}}

). Blue: raw co-occurrence counts; Orange: PPMI-weighted counts.

Figure 7: Layer-wise representational similarity analysis (top row) and nearest-neighbor consistency (bottom row) between model hidden-state geometry and multiple reference semantic spaces. Columns correspond to reference geometries: PPMI-weighted forced choice (

\mathbf{S}^{\mathrm{FC}}

), PPMI-weighted free association (

\mathbf{S}^{\mathrm{FA}}

), FastText (

\mathbf{S}^{\mathrm{FT}}

), BERT (

\mathbf{S}^{\mathrm{BERT}}

), and cross-model consensus (

\mathbf{S}^{\mathrm{X}}_{\mathrm{m}}

). Top row: Mean Pearson correlation between hidden-state similarity and each reference geometry as a function of transformer layer, averaged across models. Bottom row: Nearest-neighbor overlap (

\mathrm{NN@}k

) between hidden states and each reference geometry as a function of neighborhood size

k

(log-scaled). Colors denote embedding extraction strategies (Averaged, Meaning, Task (FC), Task (FA)).

Figure 8: Mean RSA (Pearson) between layerwise hidden-state similarity and five reference semantic geometries (PPMI-weighted forced choice, PPMI-weighted free association, FastText, BERT, cross-model consensus). Rows correspond to models and columns to embedding-extraction strategies (Averaged, Meaning, Task (FC), Task (FA)); values are averaged across layers, with color indicating correlation magnitude.

Figure 9: Layerwise representational similarity analysis profiles across models and prompting strategies. Rows correspond to instruction-tuned decoder models, and columns correspond to hidden-state extraction strategies: Averaged natural contexts (Averaged), meaning-inducing prompt (Meaning), Task-aligned forced choice prompt (Task (FC)), and Task-aligned free association prompt (Task (FA)). Curves show Pearson correlations between layerwise hidden-state similarity matrices and four reference semantic geometries: PPMI-weighted forced choice (

\mathbf{S}^{\mathrm{FC}}

), PPMI-weighted free association (

\mathbf{S}^{\mathrm{FA}}

), FastText (

\mathbf{S}^{\mathrm{FT}}

), and BERT (

\mathbf{S}^{\mathrm{BERT}}

). The x-axis denotes transformer layer index (excluding the embedding layer), and the y-axis denotes RSA correlation.

Figure 10: Stratified RSA: FC and FA alignment with hidden-state geometry broken down by WordNet supersense category ( left ; top ten categories shown, sorted by FC value) and SUBTLEX log-frequency tertile ( right ; cross-model consensus [CM] shown for reference). Values are mean Pearson

r

across the eight models, four extraction strategies, and all layers. Per-category vocabulary sizes (left panel, top to bottom): attribute

=173

, object

=88

, phenomenon

=58

, cognition

=242

, plant

=92

, quantity

=71

, animal

=149

, food

=165

, artifact

=822

, person

=849

. Frequency tertiles (right panel): low

=1670

, mid

=1664

, high

=1666

words. FC outperforms FA in

20

21

supersenses (only quantity is tied), with the largest gap on abstract categories. FC alignment scales with cue-word frequency.

Figure 11: Layerwise nearest-neighbor overlap analysis profiles across models and prompting strategies. Rows correspond to instruction-tuned decoder models, and columns correspond to hidden-state extraction strategies: Averaged natural contexts (Averaged), Meaning prompt (Meaning), Task-aligned forced choice prompt (Task (FC)), and Task-aligned free association prompt (Task (FA)). Curves show nearest-neighbor overlap between layerwise hidden-state representations and four reference semantic geometries: PPMI-weighted forced choice (

\mathbf{S}^{\mathrm{FC}}

), PPMI-weighted free association (

\mathbf{S}^{\mathrm{FA}}

), FastText (

\mathbf{S}^{\mathrm{FT}}

), and BERT (

\mathbf{S}^{\mathrm{BERT}}

). The x-axis denotes nearest-neighbor neighborhood size

k

(log-scaled), and the y-axis denotes

\mathrm{NN@}k

Figure 12: Incremental contribution of behavioral predictors to held-out-words ridge regression performance, reported as $\Delta R^{2}$ relative to a baseline including lexical and cross-model similarity features.

Figure 13: Layer-wise held-out-words ridge performance (

R^{2}

) for predicting each model’s hidden-state similarity from behavioral and lexical similarity features, shown separately for the four embedding extraction strategies.

Figure 14: Ablation study: Ridge regression performance for predicting non-mean-centered hidden-state similarity from behavioral and lexical features across eight models. Bold values show $R^{2}$ for the full model (behavioral + FastText+BERT+cross-model consensus); parenthetical values show the FastText+BERT+cross-model consensus baseline. Rows indicate the embedding extraction strategy (Averaged, Meaning, Task (FC), Task (FA)), and columns indicate layerwise correlations (min, max, mean across layers).

Figure 15: Ablation study: Incremental contribution of behavioral predictors to held-out-words ridge regression performance for non-mean-centered hidden states, reported as $\Delta R^{2}$ relative to a baseline including lexical and cross-model similarity features.

Figure 16: Ablation study: Layer-wise held-out-words ridge performance (

R^{2}

) for predicting each model’s non-mean-centered hidden-state similarity from behavioral and lexical similarity features, shown separately for the four embedding extraction strategies.

Representation Geometry in Language Models

Representation geometry in language models studies how embeddings and hidden states organize concepts, computation, and behavior.

See all 417 papers

Identity as Attractor: Geometric Evidence for Persistent Agent Architecture in LLM Activation Space
Apr 13, 2026Vladimir VasilenkoRepresentation Geometry in Language Models Cognitive Architectures for AI Agents
Independent Researcher, Rapallo, Italy
Decomposing how prompting steers behavior
Jun 2, 2026Fan L. Cheng, Nikolaus KriegeskorteLLM Prompting LLM Interpretability
Columbia University
Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
Jun 1, 2026Subramanyam Sahoo, Vinija Jain, Aman Chadha +1LLM Interpretability Representation Geometry in Language Models
Horizon Research · Meta · Apple +1

Apr 13, 2026J/K move · Enter open · S save

Identity as Attractor: Geometric Evidence for Persistent Agent Architecture in LLM Activation Space

Vladimir Vasilenko

Independent Researcher, Rapallo, Italy

Representation Geometry in Language Models Cognitive Architectures for AI Agents

Open paper AskPDF arXiv

Large language models map semantically related prompts to similar internal representations -- a phenomenon interpretable as attractor-like dynamics. We ask whether the identity document of a persistent cognitive agent (its cognitive_core) exhibits analogous attractor-like behavior. We present a controlled experiment on Llama 3.1 8B Instruct, comparing hidden states of an original cognitive_core (Condition A), seven paraphrases (Condition B), and seven structurally matched controls (Condition C). Mean-pooled states at layers 8, 16, and 24 show that paraphrases converge to a tighter cluster than controls (Cohen's d > 1.88, p < 10^{-27}, Bonferroni-corrected). Replication on Gemma 2 9B confirms cross-architecture generalizability. Ablations suggest the effect is primarily semantic rather than structural, and that structural completeness appears necessary to reach the attractor region. An exploratory experiment shows that reading a scientific description of the agent shifts internal state toward the attractor -- closer than a sham preprint -- distinguishing knowing about an identity from operating as that identity. These results provide representational evidence that agent identity documents induce attractor-like geometry in LLM activation space.

Abstract

Representation Geometry in Language Models

Explore similar work

Identity as Attractor: Geometric Evidence for Persistent Agent Architecture in LLM Activation Space

Decomposing how prompting steers behavior

Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States