Uncovering Semantic Hierarchies in Text-Attributed Graphs via Variational EM-based LLM-GNN Synergy

Yunhui Liu; Xudong Jin; Qizhuo Xie; Chunhui Zhao; Xiao Luo; Tao Zheng; Bin Chong; Tieke He

cs.LGOct 17, 2026

Uncovering Semantic Hierarchies in Text-Attributed Graphs via Variational EM-based LLM-GNN Synergy

Authors: Yunhui Liu, Xudong Jin, Qizhuo Xie, Chunhui Zhao, Xiao Luo, Tao Zheng, Bin Chong, Tieke He

Organizations: State Key Laboratory for Novel Software Technology, Nanjing University · Department of Statistics, University of Wisconsin–Madison · National Engineering Laboratory for Big Data Analysis and Applications, Peking University

Abstract

While hierarchical structures are intrinsic to organizing human knowledge, current representation learning on text-attributed graphs predominantly operates on flat semantic spaces, overlooking the rich coarse-to-fine granularity inherent in real-world data. To bridge this gap, we propose SHiFT, which synergizes the reasoning power of LLMs with the structural representation capability of GNNs within an Expectation-Maximization (EM) paradigm. In the E-step, we harness LLMs as expert taxonomists to induce an explicit semantic tree, introducing a lightweight Mapping & Diagnosis strategy that reduces inference costs by surgically updating the hierarchy only when anomalies are detected. In the M-step, we align the GNN encoder with this induced hierarchy from complementary perspectives, including clustering partition, topological skeleton, and semantic concepts. Theoretical analysis interprets SHiFT as a generalized Variational EM algorithm. Extensive experiments demonstrate that SHiFT not only achieves state-of-the-art performance but also uncovers high-quality, human-readable taxonomies. Our code can be found at https://github.com/Cloudy1225/SHiFT.

Figures & tables

Figure 1: Existing paradigms vs. SHiFT .

Figure 2: The overview of SHiFT . (Left) The model is initialized via structure-aware pretraining. (Top, E-Step ) The LLM acts as an expert taxonomist to induce or incrementally update the semantic hierarchy

\mathcal{T}

based on current node embeddings. (Bottom, M-Step ) The GNN encoder is optimized to internalize the induced hierarchy through three complementary objectives: Taxonomy Partition Enhancement, Taxonomy Skeleton Alignment, and Taxonomy Concept Alignment.

Classic/Hyperbolic GNNs and PLMs
Method	Cora	Citeseer	ArXiv	WikiCS	Computer	Avg.
GCN SE [ 23 ]	82.30 ±0.19	70.55 ±0.32	71.39 ±0.28	79.86 ±0.19	71.44 ±1.19	75.11
SAGE SE [ 13 ]	82.27 ±0.37	69.56 ±0.43	71.21 ±0.18	79.67 ±0.25	81.43 ±0.27	76.83
GAT SE [ 59 ]	81.30 ±0.67	69.94 ±0.74	71.57 ±0.25	79.99 ±0.65	83.39 ±0.22	77.24
HGCN [ 2 ]	80.38 ±0.69	70.83 ±0.62	70.39 ±0.38	79.26 ±0.39	79.44 ±0.49	76.06
HGCL [ 27 ]	82.37 ±0.47	72.11 ±0.64	71.40 ±0.68	80.16 ±0.35	80.28 ±0.55	77.26

Table 1: Node classification accuracy (%). The best and second-best results are highlighted.

GNN-based Methods
Method	Cora		Citeseer		WikiCS
Method	NMI	ARI	NMI	ARI	NMI	ARI
DMoN [ 58 ]	49.00	43.70	39.10	38.40	23.50	9.00
MAGI [ 29 ]	52.90	46.40	41.10	39.60	46.70	38.10
LLM+GNN Methods
GCLR [ 57 ]	50.87	59.20	43.30	42.49	32.51	36.49

Table 2: Node clustering performance (%).

Table 5

Backbone: MLP (Structure-agnostic)
Input Features	Cora	Citeseer	WikiCS
$\mathcal{S}_{\text{raw}}$	76.05 ±0.41	67.22 ±0.49	75.79 ±0.18
$\mathcal{S}_{\text{raw}}+\mathcal{P}_{\text{w/o }\mathcal{G}}$	72.68 ±0.34	67.51 ±0.58	77.50 ±0.24
$\mathcal{S}_{\text{raw}}+\mathcal{P}_{\text{init}}$	80.59 ±0.18	71.05 ±0.55	78.06 ±0.37
$\mathcal{S}_{\text{raw}}+\mathcal{P}_{\text{final}}$	81.91 ±0.55	72.83 ±0.73	78.33 ±0.21
Backbone: GCN (Structure-aware)

Table 5: Classification accuracy using taxonomy-augmented features.

\mathcal{P}

is the path description.

Method	Cora	Citeseer	WikiCS	Avg Drop
SHiFT (Full)	84.83 ±0.58	73.96 ±0.92	82.40 ±0.48	-
w/o EM (One-pass)	83.27 ±0.37	72.62 ±0.75	81.48 ±0.54	$\downarrow$ 1.27
w/o TPA (Partition)	83.46 ±0.35	72.84 ±0.76	81.18 ±0.22	$\downarrow$ 1.24
w/o TSA (Skeleton)	83.34 ±0.27	72.31 ±0.94	81.38 ±0.23	$\downarrow$ 1.39
w/o TCA (Concept)	84.14 ±0.60	73.56 ±0.83	82.05 ±0.44	$\downarrow$ 0.48

Table 6: Ablation study on node classification accuracy.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Nodes	#Edges	#Classes	Domain
Cora	2,708	5,429	7	Citation
Citeseer	3,186	4,277	6	Citation
ArXiv	169,343	1,166,243	40	Citation
WikiCS	11,701	216,123	10	Web Link
Computer	87,229	721,081	10	E-Commerce

Appendix

Table 7: Dataset statistics including the number of nodes, edges, and classes, and their domains.

Method	Acc@L3	Acc@L2	Acc@L1	hF
MLP	64.32 $\pm$ 0.58	73.65 $\pm$ 0.65	81.32 $\pm$ 0.49	73.45 $\pm$ 0.42
GCN	70.91 $\pm$ 0.52	78.52 $\pm$ 0.33	83.84 $\pm$ 0.18	77.85 $\pm$ 0.39
TAPE	71.54 $\pm$ 0.41	79.43 $\pm$ 0.28	84.41 $\pm$ 0.31	78.96 $\pm$ 0.33
SHiFT	74.18 $\pm$ 0.44	81.27 $\pm$ 0.31	85.73 $\pm$ 0.25	81.34 $\pm$ 0.28

Appendix

Table 8: Hierarchical classification on Patent (%). Best results are in bold.

Taxonomy	Precision	Recall	Cophenetic $r$
$\mathcal{T}_{\text{w/o }\mathcal{G}}$ (text-only)	0.7969	0.5160	0.2852
$\mathcal{T}_{\text{init}}$ (cold-start)	0.8814	0.5863	0.5603
$\mathcal{T}_{\text{final}}$ (Ours)	0.9145	0.6399	0.6423

Appendix

Table 9: Agreement between induced taxonomies and the official CPC tree on Patent.

† SHiFT runs on a 32 GB V100; methods with $>$ 40 GB memory require 80 GB A100/H100 GPUs.
Metric	GCN ${}_{\text{SE}}$	SenBERT	RoBERTa	ENGINE	GCN ${}_{\text{LLM}}$	TAPE	GraphGPT	LLaGA	LLM ${}_{\text{IT}}$	SHiFT (Ours)
GPU Mem. (GB)	4.79	9.98	46.10	7.08	7.36	46.10	60.41	41.94	69.13	16.76 †
Total Time	51s	7m	41m	2.6h	1.4h	37.4h	7.8h	7.7h	36.3h	5.5h

Appendix

Table 10: Efficiency comparison on ArXiv. Baseline results are from LLMNodeBed running on an H100 (80 GB) ; SHiFT runs on a V100 (32 GB) . Time is reported in hours (h), minutes (m), or seconds (s).

Phase	LLM Calls	Reduction vs. Init.
Taxonomy Initialization	1,217	–
Incremental Update 1	285	$\downarrow$ 76.6%
Incremental Update 2	270	$\downarrow$ 77.8%

Appendix

Table 11: Number of LLM API calls during E-steps on ArXiv.

LLM Used	Cora	Citeseer	WikiCS
DeepSeek-V3 (Default)	84.83 ±0.58	73.96 ±0.92	82.40 ±0.48
Qwen3-32B	82.44 ±0.37	73.24 ±1.03	82.04 ±0.41
Qwen3-8B (Lightweight)	84.14 ±0.60	72.97 ±0.27	81.65 ±0.18
w/o LLM (GCN Backbone)	81.70 ±1.01	69.71 ±0.64	79.95 ±0.15

Appendix

Table 12: Node classification accuracy (%) with different LLM backbones.

$\lambda$	Cora	Citeseer
1.0 (equal)	84.21	73.18
1.25	84.52	73.67
1.5 (default)	84.83	73.96
2.0	84.45	73.40

Appendix

Table 13: Sensitivity of node classification accuracy (%) to the level weighting factor

\lambda

Depth	Cora	Citeseer	WikiCS
2	84.40 ±0.30	73.57 ±0.63	81.63 ±0.17
3 (Default)	84.83 ±0.58	73.96 ±0.92	82.40 ±0.48
4	84.20 ±0.20	72.70 ±0.82	81.85 ±0.65

Appendix

Table 14: Impact of maximum taxonomy depth (

L_{max}

) on node classification accuracy (%).

Panel of Figure 3 — Figure 3: Visualization of the pairwise cosine similarity matrix of node embeddings on the Cora dataset.

Cora Ground Truth	SHiFT Induced Concept ( $\mathcal{T}_{\text{final}}$ )
Neural Networks	Neural Network Modeling
Rule Learning	Decision Tree Learning
Reinforcement Learning	RL with Function Approximation
Probabilistic Methods	Bayesian Graphical Models
Theory	Agnostic Learning Theory
Genetic Algorithms	Evolutionary Algorithms

Appendix

Table 15: Mapping between Ground-Truth classes and SHiFT -induced concepts.

Text-Attributed Graph Learning

Text-attributed graph learning combines node text and graph structure to improve classification, retrieval, anomaly detection, and reasoning.

See all 62 papers

Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learning
Mar 9, 2026Yunhui Liu, Yongchao Liu, Yinfeng Chen +3Taxonomy Induction Hierarchical Representation Learning
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Ant Group, Hangzhou, China · Ant Group, Beijing, China
LLaTA: Unlocking Graph Structure Learning with Tree-Guided Large Language Models
Mar 27, 2025Zhihan Zhang, Xunkai Li, Hongchao Qin +2Graph Structure Learning Text-Attributed Graph Learning
Beijing Institute of Technology
PromptGNN-sim: Deep Fusion and Alignment of GNN and LLMs for Text-Attributed Graph Learning
Jun 29, 2026Zhifei Hu, Alexandra I. CristeaText-Attributed Graph Learning Multimodal Fusion
Durham University, Durham, United Kingdom

Mar 9, 2026J/K move · Enter open · S save

Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learning

Yunhui Liu, Yongchao Liu, Yinfeng Chen, Chuntao Hong+2

State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Ant Group, Hangzhou, China · Ant Group, Beijing, China

Taxonomy Induction Hierarchical Representation Learning Text-Attributed Graph Learning

Open paper AskPDF arXiv

Hierarchical knowledge structures are ubiquitous across real-world domains and play a vital role in organizing information from coarse to fine semantic levels. While such structures have been widely used in taxonomy systems, biomedical ontologies, and retrieval-augmented generation, their potential remains underexplored in the context of Text-Rich Networks (TRNs), where each node contains rich textual content and edges encode semantic relationships. Existing methods for learning on TRNs often focus on flat semantic modeling, overlooking the inherent hierarchical semantics embedded in textual documents. To this end, we propose TIER (Hierarchical Taxonomy-Informed REpresentation Learning on Text-Rich Networks), which first constructs an implicit hierarchical taxonomy and then integrates it into the learned node representations. Specifically, TIER employs similarity-guided contrastive learning to build a clustering-friendly embedding space, upon which it performs hierarchical K-Means followed by LLM-powered clustering refinement to enable semantically coherent taxonomy construction. Leveraging the resulting taxonomy, TIER introduces a cophenetic correlation coefficient-based regularization loss to align the learned embeddings with the hierarchical structure. By learning representations that respect both fine-grained and coarse-grained semantics, TIER enables more interpretable and structured modeling of real-world TRNs. We demonstrate that our approach significantly outperforms existing methods on multiple datasets across diverse domains, highlighting the importance of hierarchical knowledge learning for TRNs.

Abstract

Text-Attributed Graph Learning

Explore similar work

Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learning

LLaTA: Unlocking Graph Structure Learning with Tree-Guided Large Language Models

PromptGNN-sim: Deep Fusion and Alignment of GNN and LLMs for Text-Attributed Graph Learning