A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretability

Zhang Enyan; R. Thomas McCoy

cs.CLAug 29, 2026

A Unifying Perspective on Language Model Representations: From Filler-Role Structure to Mechanistic Interpretability

Authors: Zhang Enyan, R. Thomas McCoy

Organizations: Department of Computer Science Yale University New Haven, CT · Department of Linguistics & Wu Tsai Institute Yale University New Haven, CT

Abstract

A wide range of methods have been proposed for interpreting language models, delivering important insights into their inner workings. However, different methods and their resulting insights stand in relative isolation: what could the underlying structure of language models be, such that they give rise to all our interpretations? In this work, we propose using Tensor Product Representations (TPRs) as a unifying hypothesis. TPRs give a concrete proposal for how compositional structure could be represented in vector space --- as filler-role bindings. We show, both mathematically and empirically, that TPRs can unify several prior interpretability methods: additive analogies, linear probing, sparse autoencoders, and activation patching. Mathematically, we show that these methods can all be derived from TPRs. Empirically, we apply the derivations to a range of different models --- from small toy models to LLMs --- to construct instances of each of the above interpretability methods; these constructed variants perform comparably to their standard variants. We view this work as a step toward what interpretability will ideally provide: a unified account of the nature of neural networks, corroborated not just by individual observations but also by an explanation of the connections between them.

Figures & tables

Figure 1: Left: Partial observations of an elephant point to a variety of properties—e.g., the trunk is snake-like and the ears are fan-like. All these partial observations can together be explained by the elephant’s overall structure. Right: Similarly, different language model interpretability methods expose consistent but partial descriptions of the same underlying model. Here we ask what the model’s underlying structure could be such that it supports all of these observations.

Figure 2: A Tensor Product Encoder training step, where it is trained to approximate representations in a frozen neural network.

Figure 3: Additive analogy accuracy (%) evaluated with neural network representations (NN) and TPE-constructed analogy vectors (TPE). Analogies use encoder hidden states for recurrent networks, sentence embeddings for embedding models, and last token hidden state for LLMs. TPE analogies are constructed via Equation 4 . TPE analogies match or exceed the accuracy of NN analogies.

Figure 4: PCA projection of ModernBERT encodings for sentence triplets that differ only in subject. TPE-predicted embedding differences, shown in dashed arrows, closely align with those of sentence triplets.

Figure 5: Accuracy comparison between trained linear probes and probes constructed from TPEs. (a) Probes that classify the identity of a token at a specific position (1 through 6) in the sequence; (b) Probes that classify the subject, verb, and object of a sentence given sentence embeddings; (c) Probes that classify subject, verb, and object given a LLM’s period token representation at the last layer.

Model	SAE type	$R^{2}$	Quality (%)
ModBERT	Top- $k$	0.9803	72.52
	Supervised	0.9676	99.99
	TPE-constr.	0.9289	99.77
Gemma	Top- $k$	0.9889	92.63
	Supervised	0.9866	97.26
	TPE-constr.	0.9679	99.98

Table 1: SAE performance.

Figure 6: Comparison of restoration scores between standard activation patching (blue hues) and TPE-constructed patching (orange hues), on OLMo-2-13B, across layers and 14 token positions in the prompt. The two methods produce nearly identical results.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Input	Output
Copy	2 1 7 5 10 5	2 1 7 5 10 5
Reverse	2 1 7 5 10 5	5 10 5 7 1 2

Appendix

Table 2: Example input–output pairs for the sequence manipulation tasks: copy returns the input unchanged, and reverse outputs the reversed sequence.

	RNN		LSTM		GRU
Task	Token	Sequence	Token	Sequence	Token	Sequence
Copy	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000
Reverse	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000

Appendix

Table 3: Sequence-to-sequence model test accuracies with token- and sequence-level metrics.

Task	Token	Sequence	$R^{2}$	Token	Sequence	$R^{2}$	Token	Sequence	$R^{2}$
	RNN			LSTM			GRU
Copy	1.0000	1.0000	0.9853	1.0000	1.0000	0.9754	0.9998	0.9990	0.9343
Reverse	1.0000	1.0000	0.8325	1.0000	0.9998	0.9721	0.9999	0.9996	0.9311

Appendix

Table 4: Tensor Product Encoder substitution

R^{2}

and token and sequence-level accuracies on the test split.

Model	Position	Copy		Reverse
Model	Position	Trained	Constructed	Trained	Constructed
GRU	1	100.00	100.00	58.70	57.34
	2	100.00	100.00	85.84	76.68
	3	99.88	99.96	91.52	84.70
	4	99.12	99.08	98.40	97.32
	5	94.32	86.84	100.00	99.96

Appendix

Table 5: Probe accuracy (%) comparison between trained vs analytically constructed probes on the sequence manipulation task. The results provided here correspond to the data used in plotting figure 5 .

the carpenter will see the magician.

the tutor will help the engineer.

the violinist will see the dancer.

the umpire will help the librarian.

the intern will visit the scientist.

Appendix

Table 6: Example generated sentences.

professor	student	president	judge	senator	secretary	doctor	lawyer
scientist	banker	tourist	artist	author	actor	athlete	teacher
engineer	accountant	architect	chef	photographer	farmer	ambassador	astronaut
astronomer	blacksmith	baker	barber	biologist	butler	chemist	composer
cartoonist	coach	captain	carpenter	dancer	director	drummer	detective
explorer	economist	editor	governor	gardener	illustrator	intern	inventor
journalist	linguist	manager	magician	mayor	miner	mathematician	musician

Appendix

Table 7: List of the 77 occupation nouns used as <subject> and <object> in the SVO template.

Type	Model	$R^{2}$	MSE
Embedding Model	ModernBERT	0.9396	$4.27\times 10^{-5}$
	EmbeddingGemma	0.9155	$1.57\times 10^{-5}$
	Qwen3-Embedding-8B	0.9000	$9.51\times 10^{-6}$
LLM	Qwen3-8B	0.7052	$3.50\times 10^{-2}$
	OLMo-2-13B	0.6440	$3.28\times 10^{-2}$
	GPT-OSS-20B	0.7423	$5.16\times 10^{-1}$

Appendix

Table 8: SVO TPE metrics on the held-out test split.

Model	Role	Trained	Constructed
ModernBERT	Subject	99.26	89.61
	Verb	100.00	98.01
	Object	96.49	91.53
EmbedGemma	Subject	99.83	100.00
	Verb	100.00	99.83
	Object	99.87	99.80

Appendix

Table 9: SVO probe accuracy (%) for trained vs analytically constructed probes on SVO sentences.

		Test		Generalization
Task	Model	Substitution Acc.	$R^{2}$	Substitution Acc.	$R^{2}$
Copy	RNN	1.0000	0.9863	1.0000	0.9835
	LSTM	1.0000	0.9949	0.9634	0.9882
	GRU	0.9994	0.9600	0.8230	0.9005
Reverse	RNN	1.0000	0.8344	1.0000	0.8238
	LSTM	0.9994	0.9888	0.9464	0.9758

Appendix

Table 10: Digits filler-role holdout results. Test examples contain no held-out filler-role pairs; generalization examples contain at least one held-out filler-role pair.

Model	Test $R^{2}$	Generalization $R^{2}$
ModernBERT	0.9679	0.9589
EmbeddingGemma	0.9883	0.9602
Qwen3-Embedding-8B	0.9619	0.9350

Appendix

Table 11: SVO sentence-embedding filler-role holdout reconstruction results.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability
Aug 31, 2026Matvei Tarasov, Salman Ahmadi-Asl, Andre L. F. de Almeida +1LLM Compression LLM Inference Acceleration
Innopolis University, Innopolis 420500, Russia · Department of Teleinformatics Engineering, Federal University of Cear´a, Fortaleza 60455-760, Brazil · Systems Research Institute of Polish Academy of Science and Warsaw University of Technology, Poland
Scaling Inherently Interpretable Language Models
Aug 6, 2026Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail +7Language Model Steering LLM Interpretability
Guide Labs Team
Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability
Oct 14, 2026Anton Korznikov, Alexey Dontsov, Elena TutubalinaMechanistic Interpretability Neuron-Level Interpretability

Aug 31, 2026J/K move · Enter open · S save

Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability

Matvei Tarasov, Salman Ahmadi-Asl, Andre L. F. de Almeida, Andrzej Cichocki

Innopolis University, Innopolis 420500, Russia · Department of Teleinformatics Engineering, Federal University of Cear´a, Fortaleza 60455-760, Brazil · Systems Research Institute of Polish Academy of Science and Warsaw University of Technology, Poland

LLM Compression LLM Inference Acceleration Language Modeling

Open paper AskPDF arXiv

Large language models (LLMs) are built from structured high-dimensional objects such as token representations, weights, adaptation updates, caches, and activations, whose multilinear structure is underexploited by the conventional matrix-centric view. Tensor decompositions and tensor networks provide a principled algebraic language for this structure, yet the literature often treats them as isolated compression mechanisms. This survey organizes tensor methods for LLMs through two complementary views: a seven-stage lifecycle taxonomy covering tokenization, embeddings, pre-training, adaptation, compression, inference, and interpretability, and a component view covering embeddings, attention, and feed-forward networks. We provide unified notation and theoretical foundations, analyze tensorization strategies for individual Transformer components, and compare methods at each lifecycle stage while making differences in evaluation protocols and model scales explicit. We further connect tensor methods to neighboring efficiency techniques and probabilistic tensor networks. Finally, we synthesize open challenges and introduce

ρ_{\rm gap}

, a metric for the compression-realization gap between theoretical memory reduction and measured system-level speedup. By treating tensorization as a common structural principle, the survey provides a structured entry point to tensorized language models and clarifies when parameter savings can plausibly translate into memory efficiency, computational efficiency, or interpretability. The GitHub page dedicated to this paper is accessible at this https URL.

Abstract

LLM Interpretability

Explore similar work

Tensor Methods for Language Models: From Token Representation to Training, Adaptation, Inference, Compression, and Interpretability

Scaling Inherently Interpretable Language Models

Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability