Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic Architectures

Marco Bronzini; Carlo Nicolini; Bruno Lepri; Jacopo Staiano; Andrea Passerini

cs.CLSep 29, 2025

Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic Architectures

Authors: Marco Bronzini, Carlo Nicolini, Bruno Lepri, Jacopo Staiano, Andrea Passerini

Abstract

Despite their capabilities, Large Language Models (LLMs) remain opaque with limited understanding of their internal representations. Current interpretability methods either focus on input-oriented feature extraction, such as supervised probes and Sparse Autoencoders (SAEs), or on output distribution inspection, such as logit-oriented approaches. A full understanding of LLM vector spaces, however, requires integrating both perspectives, something existing approaches struggle with due to constraints on latent feature definitions. We introduce the Hyperdimensional Probe, a hybrid supervised probe that combines symbolic representations with neural probing. Leveraging Vector Symbolic Architectures (VSAs) and hypervector algebra, it unifies prior methods: the top-down interpretability of supervised probes, SAE's sparsity-driven proxy space, and output-oriented logit investigation. By combining the supervised learning paradigm of traditional probes with the dictionary-based representation principle of SAEs, our approach enables deeper input-focused feature extraction while supporting output-oriented analysis. Our experiments demonstrate that our approach consistently extracts meaningful semantic information across different LLMs, embedding sizes, and configurations, uncovering concept-oriented insights into LLM inference across two distinct scenarios: input-completion tasks and QA-focused text generation. VSA-based probing overcomes the limitations of logit-based analyses, which are constrained by the model's token vocabulary, while also mitigating the noisier interpretability outcomes often produced by SAEs in settings with a bounded conceptual feature space. By supporting a joint investigation of input-output features, this work advances the semantic understanding of neural representations while unifying the complementary perspectives of prior methods.

Figures & tables

Figure 1: We first compress model’s internal state using dimensionality-reduction steps on LLM embeddings ( $F$ , blue ). Next, we train a neural VSA encoder to map these compressed neural embeddings into a bounded proxy space: VSA encodings ( $T$ , orange ). We then perform concept extraction by querying this proxy space using hypervector algebra ( $I$ , green ). Lastly, extracted concepts ultimately enable deeper analysis on model’s internal state and its outputs ( red ).

Figure 2: Our experimental setup uses textual inputs with syntactic structures unseen during training.

Panel of Figure 3 — Figure 3: The regression model that maps the neural representations into a controlled vector space.

Extracted Concepts (%)	GPT-2	Pythia	Llama 3.1	Phi-4	OLMo-2	Llama4, Scout	AVERAGE
Key $\mid$ Target	60.0	66.9	85.4	84.8	80.1	79.0	76.0 $\pm$ 9.4
NONE	21.9	16.7	6.9	7.6	8.5	11.5	12.2 $\pm$ 5.4
Key	4.5	6.1	0.6	1.0	1.8	3.4	2.9 $\pm$ 2.0
Example	5.8	2.4	1.5	1.3	2.0	0.8	2.3 $\pm$ 1.6
Context $\mid$ Target	1.1	1.9	1.5	1.2	4.4	2.4	2.1 $\pm$ 1.1
Key $\mid$ Key Values	1.3	1.4	1.8	1.5	1.1	0.8	1.3 $\pm$ 0.3

Table 1: Compositional concept extraction for the input-completion tasks. Each entry lists all the concepts extracted by the hyperdimensional probe. Key

\mid

Target indicates extraction of the key (

b_{1}

) and value (

b_{2}

) of the target analogy; Key for only

b_{1}

. Example refers to

a_{1}

and

a_{2}

, the in-context example’s concepts; Context

\mid

Target for all concepts. Out - of - context indicates concepts unrelated to input, Key Values refer to analogy keys originating from a different domain. NONE means no concepts. Our probe captures target-aligned combinations in roughly 80% of cases.

Figure 4: Target concept retrieval for the input-completion tasks. LLM performance on analogy completion with targets (left) and our decoding method’s ability to extract targets from LLM embeddings (right), with logit-based and SAE-based results shown in the middle.

Area	Llama 4 (docs, %)	OLMo2 (docs, %)	AVG	Sample	Domain
Mathematics	87.8	91.1	89.5	80 is to 160 as 98 is to	math double
Semantic Hierarchies	38.8	30.8	34.8	limousine is to car as monorail is to	hyponyms
Semantic Relations	10.0	3.9	7.0	Croatia is to Croatian as Switzerland is to	nationality adjective
Factual Knowledge	5.5	5.1	5.3	euclid is to Greek as galilei is to	name nationality
Verbal & Grammatical Forms	1.4	2.1	1.9	seeing is to saw as describing is to	past tense
Morphological Modifiers	1.1	0.8	1.0	agree is to agreement as excite is to	verb+ment

Table 2: Proportion of instances with no retrieved concepts across diverse input types for two language models. Lower values imply richer semantic embeddings. OLMo2 yields fewer conceptually blank representations in most categories, though Llama 4 holds a slight advantage in mathematics and grammar.

	GPT-2	Pythia	Llama 3.1	Phi-4	OLMo-2	Llama 4, Scout	AVERAGE
NONE : no concepts from DLA (%; $\mathcal{D}\subset\bar{\mathcal{S}}$ )	33.9	32.8	47.4	33.1	14.6	15.4	29.5 $\pm$ 11.4
Concepts extracted from $\mathcal{D}$ by VSA (%)
Key $\mid$ Target	53.5	56.5	76.6	70.5	44.5	42.6	57.4 $\pm$ 12.5
NONE	26.8	24.3	13.7	18.6	41.0	43.2	27.9 $\pm$ 10.9
Example	6.8	3.0	2.1	2.1	3.4	2.0	3.2 $\pm$ 1.7
Key	5.2	7.3	0.7	0.9	1.6	3.0	3.1 $\pm$ 2.4

Table 3: Concepts extracted through VSA-based probing when DLA yields no concepts. The table highlights VSA can also capture model’s variability (e.g., in-context concepts, target concepts). The table highlights key shared items across models, covering nearly 98% of all extracted concepts.

Extracted Concepts (%)	GPT-2	Pythia	Llama 3.1	Phi-4	OLMo-2	Llama 4, Scout	AVERAGE
Out-Of-Context	83.9	61.5	31.3	14.6	39.2	29.3	43.3
Key \| Target \| Out-Of-Context	2.6	11.7	45.4	54.1	37.5	42.3	32.3
Target \| Out-Of-Context	2.9	4.5	7.2	8.7	4.9	9.0	6.2
Key \| Out-Of-Context	2.6	4.4	5.3	6.2	4.2	5.4	4.7
Context \| Target \| Out-Of-Context	0.0	1.7	0.0	3.5	4.8	3.6	2.3
Example \| Out-Of-Context	2.2	7.1	1.0	0.8	1.4	1.3	2.3

Table 4: Compositional concept extraction with SAEs using the top-10 most strongly activated SAE neurons. Key

\mid

Target indicates extraction of the key (

b_{1}

) and value (

b_{2}

) of the target analogy; Key for only

b_{1}

. Example refers to

a_{1}

and

a_{2}

, the in-context example’s concepts; Context

\mid

Target for all concepts. Out - of - context indicates concepts not referred to the four in-context concepts.

Figure 5: Concepts extracted before and after the text generation of Llama 3.1, with respect to question and answer features. Red denotes the subset of failure instances, while green the full sample $\bar{\mathcal{Q}}$ .

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Input Dim	Output Dim	Note
Input Layer
Linear Layer	$d$	4096	-
Normalization	-	-	LayerNorm $(4096$ )
Activation	-	-	GELU
Residual Block 1
Linear Layer	4096	4096	GELU activation

Appendix

Table 5: Configuration of the neural VSA encoder

\mathcal{M}

for an input embedding dimension equal to

d

Large Language Model				Cosine similarity	Binary accuracy
Name	Parameters	Embedding dimension	Layers from residual stream
BASELINE	-	5120	-	0.500	0.750
Llama 4, Scout, 17B-16E	109 B	5120	24 th to 48 th $\|25\|$	0.890	0.934
OLMo-2	32 B	5120	32 nd to 64 th $\|33\|$	0.878	0.926
Phi 4	14 B	5120	20 th to 40 th $\|21\|$	0.881	0.930
Llama 3.1-8B	8 B	4096	16 th to 32 nd $\|17\|$	0.892	0.937

Appendix

Table 6: Training performance of our neural VSA encoder

\mathcal{M}

on the test set. Order by model size. BASELINE introduces a comparison against a null model with randomly-permuted input embeddings.

Concept for unbinding (%)	GPT-2	Pythia	Llama 4, Scout	OLMo-2	Phi-4	Llama 3.1	AVERAGE
Key	65.9	74.4	83.2	83.2	87.4	87.9	80.3 ± 7.8
NONE	22.0	16.9	11.6	8.6	7.7	7.0	12.3 ± 5.4
Example Key	6.0	2.6	1.0	2.1	1.5	1.7	2.5 ± 1.7
Context	1.2	2.0	2.6	4.5	1.3	1.5	2.2 ± 1.2
Greedy	2.1	1.9	1.3	0.9	1.2	0.9	1.4 ± 0.5
Example Value	1.6	1.5	1.0	0.4	0.7	0.5	0.9 ± 0.5

Appendix

Table 7: Unbinding stage : Proportions of the best unbinding concepts used for extracting concepts from VSA encodings across different models, with overall mean and standard deviation. Key refers to cases where the candidate concept corresponds to the key of the target pair (

b_{1}

), while NONE indicates that no unbinding operations were applied to the probed VSA encoding. Example denotes a concept where the key (

a_{1}

) and value (

a_{2}

) from the in-context example were pre-bound. Lastly, Context represents a scenario where the in-context example (

{a_{1},a_{2}}

) was pre-bound together with the key of the target pair (

b_{1}

). On the other hand, Greedy means using a concept candidate from the vocabulary, rather than picking it among those of the input. The table has been trimmed to highlight the relevant and common items across the models. We consider the first four strategies to be the most relevant, as they account for

97\%

of all unbinding operations across models.

MODEL	LLM’s Next Token		DLA-based Probing	SAE-based Probing		VSA-based Probing
MODEL	Precision@1	Precision@5	Precision@1	Precision@10	Precision@100	Precision@1	Precision@5
Permuted baseline	-	-	-	-	-	0.080 (079-082)	0.103 (101-104)
Unrelated baseline	-	-	-	-	-	0.099 (097-101)	0.105 (103-107)
Llama 4 Scout, 17B-16E	0.081 (079-082)	0.501 (498-504)	0.773 (771-775)	0.604 (601-607)	0.923 (921-924)	0.866 (864-868)	0.875 (873-877)
GPT-2, medium	0.267 (265-270)	0.548 (545-551)	0.300 (298-303)	0.063 (062-064)	0.402 (399-404)	0.692 (689-695)	0.702 (699-704)
Pythia-1.4b	0.369 (366-372)	0.654 (651-657)	0.413 (410-416)	0.205 (203-208)	0.863 (861-865)	0.778 (776-781)	0.790 (788-792)

Appendix

Table 8: Experimental results on the LLM’s analogy-style completion tasks, along with our probing method for retrieving the target concept from model’s internal state. The model are ordered based on precision@1 for next-token prediction. Statistical variability is reported using 95% confidence intervals, which more appropriately capture variability for metrics bounded within the [0,1] range. To control for randomness, we also introduce two control tests using Llama 3.1-8B: a comparison against a null model with randomly-permuted input embeddings (

e_{s}

, permuted baseline ), and extraction of concept pairs unrelated to inputs (

y_{s}

, unrelated baseline ).

Extracted Concepts (docs, %)	GPT-2	Pythia	Llama4, Scout	OLMo-2	Phi-4	Llama 3.1	AVERAGE	$\Delta$ VSA
NONE	33.9	32.8	15.4	14.6	33.1	47.4	29.5 $\pm$ 11.4	+17.3
Target	15.0	18.0	36.7	29.0	34.4	22.5	25.9 $\pm$ 8.1	+25.8
Key $\mid$ Target	12.6	19.3	38.4	38.5	22.7	22.1	25.6 $\pm$ 9.7	- 50.4
Key	9.7	10.4	6.3	10.4	6.0	4.5	7.9 $\pm$ 2.4	+5.0
Example Value	12.8	5.7	0.3	0.7	0.9	0.5	3.5 $\pm$ 4.6	+3.5
Example	9.0	5.3	0.3	1.7	1.0	0.5	3.0 $\pm$ 3.2	+0.7

Appendix

Table 9: Concepts extracted using the DLA probing technique on the full corpus

\bar{\mathcal{S}}

with all LLMs. Likewise in our VSA-based probing, we focus on the same middle-to-bottom range of model’s hidden layers of the last token. The table highlights the key common items across models, with the first six cases covering over 95% of all extracted concepts.

Extracted concepts (docs, %)	Morphological Modifiers	Verbal & Grammatical Forms	Factual Knowledge	Semantic Relations	Mathematics	Semantic Hierarchies	AVERAGE
Key $\mid$ Target	90.3	83.4	70.1	79.4	0.0	41.5	60.8 ± 30.3
NONE	1.6	2.7	14.3	1.7	91.1	21.1	22.1 ± 31.1
Example	0.7	0.7	4.7	8.5	0.0	15.0	4.9 ± 5.1
Key	1.6	2.7	3.8	1.4	0.0	5.1	2.4 ± 1.7
Key $\mid$ Pair Values	1.3	0.9	0.0	5.1	0.0	11.6	3.2 ± 4.2
Context $\mid$ Target	4.4	7.4	0.8	1.6	0.0	0.6	2.5 ± 2.6

Appendix

Table 10: Percentages of extracted factors by analogy category considering the subset of instances when the DLA yields no concept for OLMo-2.

Figure 6: Comprehensive raw outputs obtained though DLA on OLMo-2 for a sampled analogy.

Extracted concepts (docs,%)	GPT-2	Pythia	Llama3	Phi-4	OLMo-2	Llama4	AVERAGE
None	40.9	46.5	83.1	72.7	55.9	53.9	$58.8\pm 14.8$
Target	8.1	9.0	5.3	15.7	14.3	23.3	$12.6\pm 6.4$
Key	7.5	11.7	3.4	4.3	13.2	9.1	$8.2\pm 3.7$
Key $\mid$ Target	6.9	8.7	5.7	3.6	11.8	11.9	$8.1\pm 3.3$
Example Value	17.9	11.1	1.3	1.0	1.3	0.6	$5.5\pm 6.7$
Example	12.1	6.6	0.4	2.2	1.7	0.1	$3.9\pm 4.3$

Appendix

Table 11: Concepts extracted though DLA-based probing when VSA yields no concepts. DLA extract no concepts in the majority of the instances (59

\pm

15 %), highlighting high variability among models.

Large Language Model			Sparse Autoencoder (SAEs)		Cosine similarity
Name	Layers from residual stream	Embedding dimension	Latent dimension	Top-k	Cosine similarity
BASELINE	-	5120	20480	410	0.584
Llama 4, Scout, 17B-16E	24 th to 48 th $\|25\|$	5120	20480	410	0.979
OLMo-2	32 nd to 64 th $\|33\|$	5120	20480	410	0.976
Phi 4	20 th to 40 th $\|21\|$	5120	20480	410	0.985
Llama 3.1-8B	16 th to 32 nd $\|17\|$	4096	16384	328	0.979

Appendix

Table 12: Training performance of SAEs on the test set, order by model size. BASELINE compares against a null model using randomly permuted input embeddings.

Figure 7: Distribution of pair-wise cosine similarities among the items of the codebook.

	DE	EN	ES	FR	IT	JA	RU	ZH
DE	1.00	0.61	0.39	0.40	0.70	0.38	0.47	0.70
EN	0.88	1.00	0.64	0.75	0.85	0.69	0.74	0.80
ES	0.85	0.84	1.00	0.84	0.88	0.49	0.74	0.78
FR	0.77	0.63	0.50	1.00	0.80	0.45	0.56	0.67
IT	0.38	0.71	0.20	0.33	1.00	0.24	0.37	0.56
JA	0.96	0.96	0.84	0.93	0.93	1.00	0.88	0.85

Appendix

Table 13: Language accuracy heatmap under VSA-based steering.

Figure 8: Concepts extracted before and after the model’s text generation, with respect to question and answer features. Red denotes the subset of failure instances, while green the full sample

\bar{\mathcal{Q}}

Figure 9: Spearman correlation coefficients computed on

\bar{\mathcal{Q}}

Figure 10: P-values of the Spearman correlation coefficients.

Figure 11: Experimental metrics of the LLM’s next-token prediction task and probing performance for Llama 4. Precision@k is displayed as a categorical variable, with its binary values portrayed as boolean. The category initial token is associated to the special case (0.5) introduced in Section 5.1 . We measure VSA noise by computing the cosine similarity between the retrieved target concept and its codebook version $\Phi$ .

Figure 12: Experimental metrics of the LLM’s next-token prediction task and probing performance for OLMo-2. Precision@k is displayed as a categorical variable, with its binary values portrayed as boolean. The category initial token is associated to the special case (0.5) introduced in Section 5.1 .

Dataset	Domains		Sample example
Google Analogy Test Set	12	capital world, currency, plural, $\dots$	33,812	Denmark : krone = Mexico : peso
Bigger Analogy Test Set	33	verb+ment, occupation, gender, $\dots$	73,471	queen : king = mother : father
Mathematics	7	double, square, division2, $\dots$	6,816	4 : 16 = 5 : 25
	52		114,099

Appendix

Table 14: Knowledge bases for our synthetic corpus

\mathcal{S}

Category	Domains		Docs
Morphological Modifiers	14	noun+less, adj+ness, $\dots$	34,308 (30%)
Verbal & Grammatical Forms	13	past tense, plural, $\dots$	31,219 (27%)
Factual Knowledge	7	country capital, occupation, $\dots$	18,800 (17%)
Semantic Relations	8	family, genders, $\dots$	16,831 (15%)
Mathematics	7	math double, math division5, $\dots$	6,816 (6%)
Semantic Hierarchies	3	hypernyms, hyponyms, $\dots$	6,125 (5%)

Appendix

Table 15: Overview of our experimental set, grouped by tasking an LLM to cluster the domains.

Domain	Examples	Domain	Examples	Domain	Examples
country_capital	21801	capital_world	18561	country_language	12299
antonyms_gradable	11268	adj_superlative	10942	un+adj_reg	10614
adj+ly_reg	10576	adj_comparative	10519	male_female	10236
noun_plural_reg	10216	noun_plural_irreg	10206	verb_Ving_Ved	10164
verb_inf_3pSg	10112	animal_sound	10083	verb_inf_Ving	10008
name_nationality	9998	verb+er_irreg	9865	verb_Ving_3pSg	9861

Appendix

Table 16: All domains, and their corresponding cardinality after data augmentation for training.

Figure 13: Average Person correlations among the second half of model’s hidden layers for Llama3.1

Comp.	EV (mean $\pm$ std)	Norm. EV
0	58084 $\pm$ 5293	0.650
1	15450 $\pm$ 2056	0.170
2	5972 $\pm$ 608	0.070
3	2539 $\pm$ 330	0.030
4	2057 $\pm$ 220	0.020
5	1187 $\pm$ 166	0.010

Appendix

Table 17: Eigenvalues (EV) of the Gram matrix from OLMo-2’s residual stream.

Figure 14: Silhouette scores for varying numbers of clusters, computed using a random sample of 10,000 textual inputs from

\mathcal{S}

. The six language models have varied layer counts (see Table 6 ), which results in different maximum possible cluster numbers.

Figure 15: Distribution of model’s hidden layers grouped by k-means clustering within the ingestion algorithm $F$ for Llama3.1-8B. It portrays the percentages of cluster assignments across all instances.

Figure 16: Proof of concept for using hyperdimensional probe in multimodal settings. Figure A shows a complete probing procedure for a MNIST-based mathematical analogy. Figure B exhibits a VSA encodings describing a multimodal input using textual and image features.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

Apr 20, 2026

ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding

The explosive growth of large language models (LLMs) has created a heterogeneous and poorly documented ecosystem, making systematic model comparison increasingly important for provenance auditing, security analysis, and model selection. Existing representation methods struggle to address this setting efficiently. Approaches analyzing internal parameters are powerful when architectures are compatible, but face scalability barriers under structural heterogeneity, while methods relying on external outputs may conflate models with similar behaviors and are difficult to align in richer output spaces across different tokenizers. To bridge this gap, we propose ABLE (Attribution-Based Large-model Embedding), a framework that leverages the interpretability space to construct model representations. By aggregating gradient-based feature attributions via a tokenizer-agnostic word-level alignment, ABLE captures model-specific input-sensitivity patterns rather than only surface-level outputs. Beyond empirical utility, we provide a stability analysis showing that, under standard regularity assumptions for differentiable Transformer-style models, ABLE induces a Lipschitz-continuous parameter-to-embedding map with finite-sample convergence guarantees. Extensive experiments on 239 open-source LLMs demonstrate that our training-free approach achieves competitive or superior performance in relation prediction, model routing, and benchmark score prediction.

Zirui Wang, Yusen Hou, Shaofeng Liang +4

The Hong Kong University of Science and Technology (Guangzhou) · Deep Interdisciplinary Intelligence Lab (DI2Lab)

LLM Auditing Language Model Fingerprinting LLM Routing

Ask PDF

Mar 18, 2026

Do Language Models Encode Semantic Relations? Probing and Sparse Feature Analysis

Understanding whether large language models (LLMs) capture structured meaning requires examining how they represent concept relationships. In this work, we study three models of increasing scale: Pythia-70M, GPT-2, and Llama 3.1 8B, focusing on four semantic relations: synonymy, antonymy, hypernymy, and hyponymy. We combine linear probing with mechanistic interpretability techniques, including sparse autoencoders (SAE) and activation patching, to identify where these relations are encoded and how specific features contribute to their representation. Our results reveal a directional asymmetry in hierarchical relations: hypernymy is encoded redundantly and resists suppression, while hyponymy relies on compact features that are more easily disrupted by ablation. More broadly, relation signals are diffuse but exhibit stable profiles: they peak in the mid-layers and are stronger in post-residual/MLP pathways than in attention. Difficulty is consistent across models (antonymy easiest, synonymy hardest). Probe-level causality is capacity-dependent: on Llama 3.1, SAE-guided patching reliably shifts these signals, whereas on smaller models the shifts are weak or unstable. Our results clarify where and how reliably semantic relations are represented inside LLMs, and provide a reproducible framework for relating sparse features to probe-level causal evidence.

Andor Diera, Ansgar Scherp

Ulm University, Ulm, Germany

LLM Interpretability

Ask PDF

May 13, 2026

Polar probe linearly decodes semantic structures from LLMs

How do artificial neural networks bind concepts to form complex semantic structures? Here, we propose a simple neural code, whereby the existence and the type of relations between entities are represented by the distance and the direction between their embeddings, respectively. We test this hypothesis in a variety of Large Language Models (LLMs), each input with natural-language descriptions of minimalist tasks from five different domains: arithmetic, visual scenes, family trees, metro maps and social interactions. Results show that the true semantic structures can be linearly recovered with a Polar Probe targeting a subspace of LLMs' layer activations. Second, this code emerges mostly in middle layers and improves with LLM performance. Third, these Polar Probes successfully generalize to new entities and relation types, but degrades with the size of the semantic structure. Finally, the quality of the polar representation correlates with the LLM's ability to answer questions about the semantic structure. Together, these findings suggest that LLMs learn to build complex semantic structures by binding representations with a simple geometrical principle.

Pablo J. Diego-Simón, Pierre Orhan, Emmanuel Chemla +2

LSCP, ENS, PSL, EHESS, CNRS, Paris, France · Paris Brain Institute, Paris, France · Earth Species Project, Berkeley, California, USA +1

Representation Learning Linear Probing LLM Interpretability Neural Encoding Language Model Probing

Ask PDF

Abstract

LLM Interpretability

Explore similar work

ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding

Do Language Models Encode Semantic Relations? Probing and Sparse Feature Analysis

Polar probe linearly decodes semantic structures from LLMs