StructLens: A Structural Lens for Language Models via Maximum Spanning Trees

Haruki Sakajo; Frederikus Hudi; Yusuke Sakai; Hidetaka Kamigaito; Taro Watanabe

cs.CLFeb 10, 2026

StructLens: A Structural Lens for Language Models via Maximum Spanning Trees

Authors: Haruki Sakajo, Frederikus Hudi, Yusuke Sakai, Hidetaka Kamigaito, Taro Watanabe

Organizations: Nara Institute of Science and Technology (NAIST)

Abstract

Language exhibits inherent structures, a property that explains both language acquisition and language change. Given this characteristic, we expect language models to manifest their own internal structures as well. While interpretability research has investigated how models compute representations mechanistically through attention patterns and Sparse AutoEncoders, the organization of the resulting representations is overlooked. To address this gap, we introduce StructLens, a framework to analyze representations through a holistic structural view. StructLens constructs maximum spanning trees based on the semantic representations in residual streams, inspired by tree representation in dependency parsing, and provides summaries of token relationships in representation space. We analyze how contiguous tokens are also nearby in representation space and find that middle layers show the strongest local-span organization. Moreover, analysis of pre-training checkpoints reveals that smaller local units become detectable earlier in pre-training, and larger units later. Our findings demonstrate that StructLens provides insights into how models organize token representations across layers and training. Our code is available at https://github.com/naist-nlp/structlens.

Figures & tables

Figure 1: Overview of StructLens . StructLens extracts hidden states and then computes inter-token similarity. Finally, it constructs Maximum Spanning Trees using the adjacency similarity matrices.

Figure 2: Contiguous and non-contiguous 4-node subtrees.

Figure 3: Illustration of trees. Colored nodes form contiguous 4-node subtrees.

Figure 4: Visualization of the layer-wise evolution of the contiguous 4-node subtrees and tokens in them for the original input order and the shuffled one for Llama3.1 8B. The x-axis denotes layer depth, and the y-axis denotes the contiguous subtree ratio and the contiguous token ratio.

Figure 5: Visualization of the layer-wise evolution of the 3-, 4-, and 5-node contiguous subtrees and tokens in them across multiple checkpoints of OLMo2 7B for MMLU. In each plot, the x-axis denotes the layer index, and the y-axis denotes the contiguous subtree and token ratio.

Panel of Figure 6 — Figure 6: Inter-layer similarity samples of Llama3.1 8B for each metric on MMLU. Bright color represents high similarity, while dark color represents low similarity.

Metric	Layers removed	ARC-Easy	MMLU	CMMLU	GSM8K	Multinews	VCSUM	Avg.
Llama3.1 8B Instruct
Dense	-	85.8	68.4	55.9	70.4	26.1	16.6	53.8
CosBase-BI	24 25 26 27	78.3	66.8	54.4	43.0	23.1	8.7	45.7
CKA-BI	24 25 26 31	75.3 †	66.9	54.2	5.8 †	18.2 †	1.9 †	37.0
CosStruct-BI	23 24 25 26	79.3	67.8 †	55.1 †	57.1 †	25.1 †	16.2 †	50.1
TreeEdit-BI	23 24 26 27	78.1	66.9	55.8 †	40.9 †	22.7	10.7 †	45.8

Table 1: Pruning results. Each value denotes the accuracy or the correctness of an answer in a task, with higher values indicating better performance. Values of each dataset result denoted by

\dagger

are statistically significant (

p<0.05

) compared to CosBase-BI.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Average L2 distance between tokens.

Table 2: HuggingFace ID and Hyperparameters.

Panel of Figure 8 — Figure 8: Contiguous subtree and token ratios for 3-, 4-, and 5-node subtrees of Llama3.1 8B.

Panel of Figure 9 — Figure 9: Contiguous subtree and token ratios for 3-, 4-, and 5-node subtrees of Qwen2.5 7B.

Figure 10: 5-node contiguous subtree ratio at Layer 18 and MMLU performance across checkpoints.

Figure 11: Visualization of the layer-wise evolution of the contiguous subtrees and tokens in them of multiple checkpoints of OLMo2 7B for MMLU. The x-axis denotes the layer index, and the y-axis denotes the contiguous subtree and token ratio.

Panel of Figure 12 — Figure 12: Inter-layer similarity samples of Qwen2.5 7B for each metric on MMLU. Bright color represents high similarity, while dark color represents low similarity.

Panel of Figure 13 — Figure 13: Inter-layer similarity samples by Edge-Edit of Llama3.1 8B and Llama3.1 70B for MMLU and CMMLU. Bright color represents high similarity, while dark color represents low similarity.

Panel of Figure 14 — Figure 14: Inter-layer similarity samples by Edge-Edit of Qwen2.5 7B and Qwen2.5 72B for MMLU and CMMLU. Bright color represents high similarity, while dark color represents low similarity.

Panel of Figure 15 — Figure 15: Inter-layer similarity samples of OLMo2 7B for each checkpoint on MMLU. Bright color represents high similarity, while dark color represents low similarity.

Method	k	Llama3.1 8B				Qwen2.5 7B
		MMLU		CMMLU		MMLU		CMMLU
		ARI $\uparrow$	Cond. $\downarrow$	ARI $\uparrow$	Cond. $\downarrow$	ARI $\uparrow$	Cond. $\downarrow$	ARI $\uparrow$	Cond. $\downarrow$
CKA	2	$.84_{\pm.171}$	$.70_{\pm.046}$	$.60_{\pm.256}$	$\mathbf{.60_{\pm.075}}$	$.92_{\pm.098}$	$.86_{\pm.019}$	$\mathbf{.88_{\pm.175}}$	$\mathbf{.43_{\pm.027}}$
	3	$\mathbf{1.0_{\pm.016}}$	$\mathbf{.64_{\pm.001}}$	$\mathbf{.67_{\pm.177}}$	$.61_{\pm.024}$	$.92_{\pm.077}$	$\mathbf{.81_{\pm.017}}$	$.53_{\pm.267}$	$.50_{\pm.057}$
	4	$.97_{\pm.039}$	$.74_{\pm.001}$	$.65_{\pm.209}$	$.70_{\pm.017}$	$\mathbf{.99_{\pm.060}}$	$.87_{\pm.007}$	$.62_{\pm.185}$	$.57_{\pm.028}$
Cos-Base	2	$1.0_{\pm.017}$	$\mathbf{.48_{\pm.001}}$	$\mathbf{.97_{\pm.051}}$	$\mathbf{.47_{\pm.010}}$	$.83_{\pm.220}$	$\mathbf{.64_{\pm.046}}$	$.82_{\pm.144}$	$\mathbf{.53_{\pm.022}}$

Appendix

Table 3: Adjusted Rand Index (ARI) and Conductance (Cond.) on Llama 3.1 8B and Qwen2.5 7B. Bold denotes the best performance within each method.

Table 4: Frequent subtree samples. The tree is represented as a strict S-expression. The number before “_” denotes the index of the token in the input. We replaced “(” and “)” in the input tokens with “[” and “]” to run subtree mining correctly.

Subtree

Llama3.1 8B

(72_,(86_.(414_.(471_ \CJK@punctchar \CJK@uniPunct 0"80"A6(570_ \CJK@punctchar \CJK@uniPunct 0"80"A6(788_ \CJK@punctchar \CJK@uniPunct 0"80"A6(862_ \CJK@punctchar \CJK@uniPunct 0"80"A6))))(505_.))))

Layers: 1, 5, 32

Absence interval: 27

(14_A(36_[A(41_[B(48_[C(54_[D(143_[D(1352_[D)))(131_[C)))))

Layers: 1, 2, 3, 4, 5, 6, 7, 8, 29

Appendix

Table 5: Frequent subtree patterns found in non-adjacent layers. This shows the top two patterns with the longest periods during which the structure was not used. The absence interval represents the maximum number of layers between when a structure observed at a layer disappears and when it reappears. For example, when a structure is observed in layers 3, 6, and 8, the absence interval is 3, corresponding to the number of layers from layer 3 to layer 6.

Panel of Figure 16 — Figure 16: Logit lens visualization on MMLU. We visualize the token predictions for each of the last eight tokens in the input. The lower rows represent predictions from the lower layers, while the upper rows show predictions from the higher layers. Color intensity represents prediction probability.

	Llama3.1 8B		Qwen2.5 7B
	Layers	Cond.	Layers	Cond.
Cluster 1	0, 1, 2, 3.	.39	0, 1, 2, 3, 4, 5, 6, 7.	.64
Cluster 2	4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17.	.76	8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20.	.47
Cluster 3	18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32.	.44	21, 22, 23, 24, 25, 26, 27, 28.	.54

Appendix

Table 6: Sample of clustering results of Edge-Edit for an instance of MMLU (

k=3

). Layer 0 indicates the input embeddings. Cond. denotes Conductance (see Appendix E ).

Figure 17: Visualization of layer importance. Lighter colors represent less importance, while darker colors represent greater importance.

Model	Metric	Layers removed	ARC-Easy	MMLU	CMMLU	Avg.
Llama-3.1-8B	Dense	-	84.6	65.3	52.9	67.6
	CosBase-BI	24 25 26 27	76.8	62.0	49.0	62.6
	CKA-BI	24 26 27 31	72.4 †	62.3	49.4	61.4
	CosStruct-BI	23 24 25 26	77.8	64.5 †	51.2 †	64.5
	TreeEdit-BI	23 24 26 27	76.8	64.6 †	51.9 †	64.4
	EdgeEdit-BI	23 24 25 26	77.8	64.5 †	51.2 †	64.5

Appendix

Table 7: Pruning results of multiple-choice QA tasks. Each value denotes the accuracy or the correctness of an answer in a particular task, with higher values indicating better performance. Values denoted by

\dagger

are statistically significant (

p<0.05

) compared to CosBase-BI.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

Non-Gaussianity Reveals Interpretable Directions in Language Models
Jun 10, 2026Sida Liu, Feijiang Han, Sarah Wiegreffe +1Independent Component Analysis LLM Interpretability
Independent Researcher · University of Maryland
Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming
Oct 13, 2026Woosang Lim, Haneul Yoo, Vincent-Daniel Yun +1LLM Pruning LLM Compression
Seoul National University, Republic of Korea · New York University, United States · University of Southern California, United States
Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs
May 12, 2026Jingzhou Jiang, Yi Yang, Kar Yan TamText Embeddings LLM Pruning
The Hong Kong University of Science and Technology

Jun 10, 2026J/K move · Enter open · S save

Non-Gaussianity Reveals Interpretable Directions in Language Models

Sida Liu, Feijiang Han, Sarah Wiegreffe, Furong Huang

Independent Researcher · University of Maryland

Independent Component Analysis LLM Interpretability Language Model Introspection Representation Geometry in Language Models

Open paper AskPDF arXiv

Finding human-interpretable directions in language-model activations is important for understanding and controlling model behavior. These directions are usually discovered through indirect learning objectives such as sparse reconstruction, but it remains unclear what existing statistical structure distinguishes interpretable directions and whether that structure can be used directly for feature discovery. We find that non-Gaussianity provides such a signature: projection distributions of natural-text residual-stream activations along sparse autoencoder (SAE) decoder directions are substantially more non-Gaussian than those along random directions, with most contexts forming a concentrated background and recurring patterns often occupying one tail. This raises a natural question: if interpretable directions tend to have non-Gaussian projection distributions, can we discover them by searching for non-Gaussian directions directly in activation space, without training another sparse dictionary? To test this hypothesis, we introduce ICA Lens, a practical workflow for efficient discovery, reliable annotation, and faithful evaluation of non-Gaussian directions in LLM residual-stream activations. Across GPT-2 Small, Gemma 2 2B, and Qwen 3.5 9B Base, we find that: (1) the recovered non-Gaussian directions capture coherent lexical, syntactic, and semantic patterns under both human and LLM-based evaluation; (2) non-Gaussianity is positively associated with both human-label confidence and automated interpretability; and (3) the resulting coordinates are competitive with SAEs across interpretability, sparse reconstruction, concept prediction, and targeted probe perturbation, while fitting in minutes per layer and using 7-43 times less memory. These results show that non-Gaussianity is a direct unsupervised signal for discovering interpretable and useful structure in activations.

Abstract

LLM Interpretability

Explore similar work

Non-Gaussianity Reveals Interpretable Directions in Language Models

Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming

Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs