Discovering Decoupled Functional Modules in Large Language Models

Yanke Yu; Jin Li; Ying Sun; Ping Li; Zhefeng Wang; Yi Zheng

cs.LGMar 18, 2026

Discovering Decoupled Functional Modules in Large Language Models

Authors: Yanke Yu, Jin Li, Ying Sun, Ping Li, Zhefeng Wang, Yi Zheng

Organizations: Thrust of AI, Hong Kong University of Science and Technology (Guangzhou) · Huawei Technologies Ltd.

Abstract

Understanding the internal functional organization of Large Language Models (LLMs) is crucial for improving their trustworthiness and performance. However, how LLMs organize different functions into modules remains highly unexplored. To bridge this gap, we formulate a functional module discovery problem and propose an Unsupervised LLM Cross-layer MOdule Discovery (ULCMOD) framework that simultaneously disentangles the large set of neurons in the entire LLM into modules while discovering the topics of input samples related to these modules. Our framework introduces a novel objective function and an efficient Iterative Decoupling (IterD) algorithm. Extensive experiments show that our method discovers high-quality, disentangled modules that capture more meaningful semantic information and achieve superior performance in various downstream tasks. Moreover, our qualitative analysis reveals that the discovered modules show semantic coherence, correspond to interpretable specializations, and a clear spatial and hierarchical organization within the LLM. Our work provides a novel tool for interpreting the functional modules of LLMs, filling a critical blank in LLM's interpretability research.

Figures & tables

Figure 1: Illustrations of brain modules vs. LLM modules.

Figure 2: Illustration of function modules. Each red block represents a function module, associated with a set of neurons and a set of representative samples employing it.

Figure 3: Overview of the IterD optimization preocess.

LLM	$K$	K-Means			Mini-Batch K-Means			Agglomerative			Spectral			Spectral Co-cluster			IterD ( Ours )
LLM	$K$	$\mathcal{L}(F)$	$\xi(F)$	$B(F)$	$\mathcal{L}(F)$	$\xi(F)$	$B(F)$	$\mathcal{L}(F)$	$\xi(F)$	$B(F)$	$\mathcal{L}(F)$	$\xi(F)$	$B(F)$	$\mathcal{L}(F)$	$\xi(F)$	$B(F)$	$\mathcal{L}(F)$	$\xi(F)$	$B(F)$
Qwen2.5 -1.5B -Instruct	5	19.2	0.409	46.9	23.4	0.406	57.6	10.1	0.311	32.4	9.6	0.335	28.7	24.0	0.380	63.2	31.4	0.465	67.5
	10	5.7	0.644	8.8	4.2	0.630	6.7	5.2	0.599	8.7	2.8	0.566	5.0	7.9	0.522	15.1	12.0	0.711	16.9
	15	0.9	0.829	1.1	1.9	0.752	2.6	1.9	0.655	2.9	0.7	0.750	0.9	3.8	0.583	6.4	6.6	0.892	7.5
	20	0.4	0.943	0.4	0.5	0.855	0.6	1.0	0.731	1.3	0.3	0.914	0.3	2.0	0.636	3.1	4.4	1.048	4.2
Qwen2.5 -3B -Instruct	5	28.0	0.466	60.1	26.9	0.490	54.9	16.1	0.439	36.6	0.0	0.001	7.7	34.6	0.404	85.6	56.9	0.533	106.8

Table 1: Comparison of Clustering Metrics for different methods, LLMs, and number of clusters (

K

). Values for

\mathcal{L}(F)

and

B(F)

are scaled by

\times 10^{6}

Figure 4: Average activation heatmap for discovered modules in Qwen2.5-7B-Instruct ( $K=10$ ). Each cell $(i,j)$ shows the average activation of neuron set $U_{j}$ on sample set $S_{i}$ .

LLM	Cls.	$K$	K-Means		Mini-Batch K-Means		Agglomerative		Spectral		Spectral Co-Clustering		IterD ( Ours )
LLM	Cls.	$K$	Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1	Acc	F1
Qwen2.5 -1.5B -Instruct	LR	5	0.4800	0.4758	0.5400	0.5404	0.5412	0.5409	0.5379	0.5300	0.4650	0.4602	0.5436	0.5428
		10	0.7186	0.7173	0.6929	0.6927	0.7057	0.7039	0.7150	0.7150	0.6793	0.6764	0.7257	0.7259
		15	0.7664	0.7662	0.7729	0.7724	0.7750	0.7750	0.7486	0.7474	0.7107	0.7089	0.8157	0.8158
		20	0.7907	0.7908	0.7843	0.7839	0.7850	0.7849	0.7864	0.7862	0.7643	0.7637	0.8200	0.8200
	SVC	5	0.5414	0.5417	0.5800	0.5829	0.5921	0.5933	0.5764	0.5695	0.5343	0.5339	0.5964	0.5987

Table 2: Comparison of Test Accuracy and Macro-F1 for different methods, LLMs, classifiers, and number of clusters (

K

). Features are derived from average function module activations.

Figure 5: Visualization of sample category similarity in Qwen2.5-7B-Instruct ( $K=10$ ). The value in cell $(i,j)$ represents the average cosine similarity between the feature vectors ( $\mathbf{x}_{s}$ ) of samples from category $i$ and category $j$ .

Figure 6: The studies of how neuron count is distributed in different layers inside each function module for Qwen2.5-3B-Instruct (

K=5

Panel of Figure 7 — Figure 7: Visualization of discovered function modules.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Layer-wise distribution of function modules in Qwen2.5-3B-Instruct (

K=10

). Each subfigure shows the neuron counts in each layer for the corresponding function module.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

S^3martCirc: Self-supervised Smart Circuit Discovery
Sep 1, 2026Wendy Zheng, Yinhan He, Liang Wu +1LLM Interpretability Mechanistic Interpretability
University of Virginia · Nokia
Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures
Apr 17, 2026Yutong Gao, Qinglin Meng, Yuan Zhou +1Transformer Interpretability LLM Interpretability
MOE Key Lab of Computational Linguistics, Peking University · Nanjing University of Science and Technology · Purdue University +1
Language Impairment as a Lens onto Large Language Models
Oct 20, 2026Muxuan He, Wang Bill Zhu, Yue Chen +5LLM Interpretability Circuits in Language Models
University of Southern California · University of California, Berkeley · Independent Researcher +1

Sep 1, 2026J/K move · Enter open · S save

S^3martCirc: Self-supervised Smart Circuit Discovery

Wendy Zheng, Yinhan He, Liang Wu, Jundong Li

University of Virginia · Nokia

LLM Interpretability Mechanistic Interpretability Circuits in Language Models

Open paper AskPDF arXiv

Large Language Models (LLMs) have demonstrated remarkable performance across diverse tasks, from text summarization to question answering. Despite these capabilities, their black-box nature obscures internal decision-making processes. Mechanistic interpretability (MI) aims to address this by reverse-engineering neural networks into human-understandable algorithms. Current MI approaches for LLMs typically follow a two-stage paradigm: first identifying important components (circuit discovery), where components are typically individual nodes such as an attention head or feedforward neuron, and second determining the role they play in a certain task (functional interpretation). However, this sequential approach overlooks a fundamental insight: a component's importance and its functional role are inherently codependent. Unifying these stages presents two key challenges: (1) functional roles are often tied to specific nodes or components, limiting generalization, and (2) their identification relies on subjective interpretation rather than quantifiable metrics. To address these challenges, we propose S^3martCirc (Self-supervised Smart Circuit Discovery), a unified framework that simultaneously discovers circuits and interprets functionality. S^3martCirc abstracts node behavior into two general computational roles that generalize across tasks and defines a quantitative metric for assigning them, enabling importance and functional role to be discovered jointly rather than in sequence. Extensive experiments show that our framework outperforms existing methods in circuit discovery.

Abstract

LLM Interpretability

Explore similar work

S^3martCirc: Self-supervised Smart Circuit Discovery

Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures

Language Impairment as a Lens onto Large Language Models