Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learning
Authors: Yunhui Liu, Yongchao Liu, Yinfeng Chen, Chuntao Hong, Tao Zheng, Tieke He
Organizations: State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Ant Group, Hangzhou, China · Ant Group, Beijing, China
Hierarchical knowledge structures are ubiquitous across real-world domains and play a vital role in organizing information from coarse to fine semantic levels. While such structures have been widely used in taxonomy systems, biomedical ontologies, and retrieval-augmented generation, their potential remains underexplored in the context of Text-Rich Networks (TRNs), where each node contains rich textual content and edges encode semantic relationships. Existing methods for learning on TRNs often focus on flat semantic modeling, overlooking the inherent hierarchical semantics embedded in textual documents. To this end, we propose TIER (Hierarchical Taxonomy-Informed REpresentation Learning on Text-Rich Networks), which first constructs an implicit hierarchical taxonomy and then integrates it into the learned node representations. Specifically, TIER employs similarity-guided contrastive learning to build a clustering-friendly embedding space, upon which it performs hierarchical K-Means followed by LLM-powered clustering refinement to enable semantically coherent taxonomy construction. Leveraging the resulting taxonomy, TIER introduces a cophenetic correlation coefficient-based regularization loss to align the learned embeddings with the hierarchical structure. By learning representations that respect both fine-grained and coarse-grained semantics, TIER enables more interpretable and structured modeling of real-world TRNs. We demonstrate that our approach significantly outperforms existing methods on multiple datasets across diverse domains, highlighting the importance of hierarchical knowledge learning for TRNs.
Figures & tables
Figure 1. An example taxonomy of computer science papers.
Figure 2. The framework of TIER.
Model
Cora
Citeseer
Pubmed
ArXiv
WikiCS
Books
Photo
Computer
Avg.
GCN SE ( Kipf and Welling, 2017 )
82.30±0.19
70.55±0.32
78.94±0.27
71.39±0.28
79.86±0.19
68.79±0.46
69.25±0.81
71.44±1.19
74.06
SAGE SE ( Hamilton et al., 2017 )
82.27±0.37
69.56±0.43
77.88±0.44
71.21±0.18
79.67±0.25
72.01±0.33
78.50±0.15
81.43±0.27
76.57
GAT SE ( Veličković et al., 2018 )
81.30±0.67
69.94±0.74
78.49±0.70
71.57±0.25
79.99±0.65
74.35±0.35
80.40±0.45
83.39±0.22
77.43
SenBERT ( Reimers and Gurevych, 2019 )
66.66±1.42
60.52±1.62
36.04±2.92
72.66±0.24
77.77±0.75
83.68±0.19
73.89±0.31
70.76±0.15
67.75
RoBERTa ( Liu et al., 2019 )
72.24±1.14
66.68±2.03
42.32±1.56
74.12±0.12
76.81±1.04
84.62±0.16
74.79±1.13
72.31±0.37
70.49
GIANT ( Chien et al., 2022 )
81.08±1.07
66.02±1.28
79.32±0.45
74.29±0.10
80.48±0.65
81.99±0.37
71.83±0.38
86.16±0.12
77.65
Table 1. Node classification accuracy on eight TRN datasets. The best and second-best results are highlighted.
Figure 3. Visualizations of the learned node representations (colored by ground-truth labels) and the pairwise distance matrix between finest-level cluster centroids on Citeseer, with and without taxonomy regularization. The constructed taxonomy tree has 3 hierarchical levels with 1, 6, and 62 cluster nodes from top to bottom, respectively. Darker colors indicate smaller distances. With taxonomy regularization, clearer block structures emerge, where darker diagonal blocks correspond to coarser-grained clusters, reflecting improved semantic hierarchy in learned representation space.
GCN SE
SenBERT
RoBERTa
ENGINE
GCN LLM
TAPE
GraphGPT
LLaGA
LLM IT
TIER
Mem.
4.79
9.98
46.10
7.08
7.36
46.10
60.41
41.94
69.13
6.78
Time
51.2s
7.4m
40.8m
2.6h
1.4h
37.4h
7.8h
7.7h
36.3h
16.8m
Table 2. Efficiency comparison in terms of training time and GPU memory (GB) on ArXiv.
Figure 4. RadialMap of the constructed taxonomy on Citeseer.
Variant
Cora
Citeseer
WikiCS
Photo
TIER
84.89±0.38
73.70±0.92
82.51±0.25
87.16±0.19
w/o SGCL
83.40±0.30
72.46±1.24
79.79±0.14
85.01±0.28
w/o LLM
83.47±0.71
72.95±1.07
80.33±0.29
86.53±0.35
w/o CCC
83.20±0.62
71.63±1.60
80.18±0.34
85.44±0.22
Table 3. Ablation study results for TIER.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Prompt 1: The prompt for cluster splitting.
Prompt 2: The prompt for cluster merging.
Prompt 3: The prompt for cluster labeling and summarization.
Prompt 4: The prompt for outlier reassignment.
Dataset
#Nodes
#Edges
#Tokens
#Classes
Domain
Cora
2,708
5,429
183.4
7
CS Citation
Citeseer
3,186
4,277
210.0
6
CS Citation
Pubmed
19,717
44,338
446.5
3
Bio Citation
ArXiv
169,343
1,166,243
239.8
40
CS Citation
WikiCS
11,701
216,123
629.9
10
Web Link
Books
41,551
358,574
337.0
12
E-Commerce
Appendix
Table 4. Dataset statistics including the number of nodes and edges, the average number of tokens in the textual descriptions associated with nodes, the number of classes, and the domains they belong to.
Cora
(1, 3, 64)
(1, 7, 64)
(1, 14, 64)
(1, 21, 64)
(1, 4, 16, 64)
(1, 7, 21, 64)
82.58±1.28
84.89±0.38
83.61±2.34
83.31±0.43
82.69±1.47
84.26±1.41
Citeseer
(1, 3, 64)
(1, 6, 64)
(1, 12, 64)
(1, 18, 64)
(1, 4, 16, 64)
(1, 6, 18, 64)
72.16±0.921
73.70±0.92
71.72±1.29
72.10±1.81
71.55±1.084
73.36±1.32
WikiCS
(1, 5, 128)
(1, 10, 128)
(1, 20, 128)
(1, 30, 128)
(1, 10, 32, 128)
(1, 10, 64, 128)
79.83±0.04
82.51±0.25
80.62±0.37
80.83±0.38
82.09±0.33
81.94±0.68
Photo
(1, 6, 256)
(1, 12, 256)
(1, 24, 256)
(1, 36, 256)
(1, 12, 32, 256)
(1, 12, 64, 256)
Appendix
Table 5. Performance with different taxonomy tree structures.
Figure 5. How the accuracy varies with different values of λ .
Cora
Citeseer
Model
Base
+Ours
Base
+Ours
GCN
32.50
34.69
50.01
53.88
GAE
29.98
32.43
63.33
64.53
BUDDY
26.40
30.16
59.48
61.30
Appendix
Table 6. Link prediction results (MRR, %) on Cora and Citeseer. “+Ours” denotes the backbone augmented with our taxonomy-informed regularization.
Figure 6. Visualizations of the learned node representations (colored by ground-truth labels) and the pairwise distance matrix between finest-level cluster centroids on Pumbed, with and without taxonomy regularization. The constructed taxonomy tree has 4 hierarchical levels with 1, 3, 16, and 63 cluster nodes from top to bottom, respectively. Darker colors indicate smaller distances. With taxonomy regularization, clearer block structures emerge, where darker diagonal blocks correspond to coarser-grained clusters, reflecting improved semantic hierarchy in learned representation space.
Figure 7. Visualizations of the learned node representations (colored by ground-truth labels) and the pairwise distance matrix between finest-level cluster centroids on WikiCS, with and without taxonomy regularization. The constructed taxonomy tree has 3 hierarchical levels with 1, 10, and 124 cluster nodes from top to bottom, respectively. Darker colors indicate smaller distances. With taxonomy regularization, clearer block structures emerge, where darker diagonal blocks correspond to coarser-grained clusters, reflecting improved semantic hierarchy in learned representation space.
Figure 8. Visualizations of the learned node representations (colored by ground-truth labels) and the pairwise distance matrix between finest-level cluster centroids on Photo, with and without taxonomy regularization. The constructed taxonomy tree has 4 hierarchical levels with 1, 12, 64, and 241 cluster nodes from top to bottom, respectively. Darker colors indicate smaller distances. With taxonomy regularization, clearer block structures emerge, where darker diagonal blocks correspond to coarser-grained clusters, reflecting improved semantic hierarchy in learned representation space.
Figure 9. RadialMap of the constructed taxonomy on Citeseer.
Figure 10. RadialMap of the constructed taxonomy on Pubmed.
Figure 11. RadialMap of the constructed taxonomy on WikiCS.
Figure 12. RadialMap of the constructed taxonomy on Photo.
State Key Laboratory for Novel Software Technology, Nanjing University · Department of Statistics, University of Wisconsin–Madison · National Engineering Laboratory for Big Data Analysis and Applications, Peking University
1MaiNLP, Center for Information and Language Processing, LMU Munich, Germany · 2Fraunhofer Institute for Systems and Innovation Research ISI, Germany · Department of Computer Science, IT University of Copenhagen, Denmark
Indian Institute of Technology Delhi, New Delhi, India · KDM Lab, Dhirubhai Ambani University, Gandhinagar, Gujarat, India · Indian Institute of Technology Delhi, Abu Dhabi, UAE.