Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learning

Yunhui Liu; Yongchao Liu; Yinfeng Chen; Chuntao Hong; Tao Zheng; Tieke He

cs.LGMar 9, 2026

Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learning

Authors: Yunhui Liu, Yongchao Liu, Yinfeng Chen, Chuntao Hong, Tao Zheng, Tieke He

Organizations: State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Ant Group, Hangzhou, China · Ant Group, Beijing, China

Abstract

Hierarchical knowledge structures are ubiquitous across real-world domains and play a vital role in organizing information from coarse to fine semantic levels. While such structures have been widely used in taxonomy systems, biomedical ontologies, and retrieval-augmented generation, their potential remains underexplored in the context of Text-Rich Networks (TRNs), where each node contains rich textual content and edges encode semantic relationships. Existing methods for learning on TRNs often focus on flat semantic modeling, overlooking the inherent hierarchical semantics embedded in textual documents. To this end, we propose TIER (Hierarchical Taxonomy-Informed REpresentation Learning on Text-Rich Networks), which first constructs an implicit hierarchical taxonomy and then integrates it into the learned node representations. Specifically, TIER employs similarity-guided contrastive learning to build a clustering-friendly embedding space, upon which it performs hierarchical K-Means followed by LLM-powered clustering refinement to enable semantically coherent taxonomy construction. Leveraging the resulting taxonomy, TIER introduces a cophenetic correlation coefficient-based regularization loss to align the learned embeddings with the hierarchical structure. By learning representations that respect both fine-grained and coarse-grained semantics, TIER enables more interpretable and structured modeling of real-world TRNs. We demonstrate that our approach significantly outperforms existing methods on multiple datasets across diverse domains, highlighting the importance of hierarchical knowledge learning for TRNs.

Figures & tables

Figure 1. An example taxonomy of computer science papers.

Model	Cora	Citeseer	Pubmed	ArXiv	WikiCS	Books	Photo	Computer	Avg.
GCN ${}_{\text{SE}}$ ( Kipf and Welling, 2017 )	82.30±0.19	70.55±0.32	78.94±0.27	71.39±0.28	79.86±0.19	68.79±0.46	69.25±0.81	71.44±1.19	74.06
SAGE ${}_{\text{SE}}$ ( Hamilton et al., 2017 )	82.27±0.37	69.56±0.43	77.88±0.44	71.21±0.18	79.67±0.25	72.01±0.33	78.50±0.15	81.43±0.27	76.57
GAT ${}_{\text{SE}}$ ( Veličković et al., 2018 )	81.30±0.67	69.94±0.74	78.49±0.70	71.57±0.25	79.99±0.65	74.35±0.35	80.40±0.45	83.39±0.22	77.43
SenBERT ( Reimers and Gurevych, 2019 )	66.66±1.42	60.52±1.62	36.04±2.92	72.66±0.24	77.77±0.75	83.68±0.19	73.89±0.31	70.76±0.15	67.75
RoBERTa ( Liu et al., 2019 )	72.24±1.14	66.68±2.03	42.32±1.56	74.12±0.12	76.81±1.04	84.62±0.16	74.79±1.13	72.31±0.37	70.49
GIANT ( Chien et al., 2022 )	81.08±1.07	66.02±1.28	79.32±0.45	74.29±0.10	80.48±0.65	81.99±0.37	71.83±0.38	86.16±0.12	77.65

Table 1. Node classification accuracy on eight TRN datasets. The best and second-best results are highlighted.

Panel of Figure 3 — Figure 3. Visualizations of the learned node representations (colored by ground-truth labels) and the pairwise distance matrix between finest-level cluster centroids on Citeseer, with and without taxonomy regularization. The constructed taxonomy tree has 3 hierarchical levels with 1, 6, and 62 cluster nodes from top to bottom, respectively. Darker colors indicate smaller distances. With taxonomy regularization, clearer block structures emerge, where darker diagonal blocks correspond to coarser-grained clusters, reflecting improved semantic hierarchy in learned representation space.

	GCN ${}_{\text{SE}}$	SenBERT	RoBERTa	ENGINE	GCN ${}_{\text{LLM}}$	TAPE	GraphGPT	LLaGA	LLM ${}_{\text{IT}}$	TIER
Mem.	4.79	9.98	46.10	7.08	7.36	46.10	60.41	41.94	69.13	6.78
Time	51.2s	7.4m	40.8m	2.6h	1.4h	37.4h	7.8h	7.7h	36.3h	16.8m

Table 2. Efficiency comparison in terms of training time and GPU memory (GB) on ArXiv.

Figure 4. RadialMap of the constructed taxonomy on Citeseer.

Variant	Cora	Citeseer	WikiCS	Photo
TIER	84.89±0.38	73.70±0.92	82.51±0.25	87.16±0.19
w/o SGCL	83.40±0.30	72.46±1.24	79.79±0.14	85.01±0.28
w/o LLM	83.47±0.71	72.95±1.07	80.33±0.29	86.53±0.35
w/o CCC	83.20±0.62	71.63±1.60	80.18±0.34	85.44±0.22

Table 3. Ablation study results for TIER.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Prompt 1: The prompt for cluster splitting.

Panel of Figure 9 — Prompt 2: The prompt for cluster merging.

Panel of Figure 10 — Prompt 3: The prompt for cluster labeling and summarization.

Panel of Figure 11 — Prompt 4: The prompt for outlier reassignment.

Dataset	#Nodes	#Edges	#Tokens	#Classes	Domain
Cora	2,708	5,429	183.4	7	CS Citation
Citeseer	3,186	4,277	210.0	6	CS Citation
Pubmed	19,717	44,338	446.5	3	Bio Citation
ArXiv	169,343	1,166,243	239.8	40	CS Citation
WikiCS	11,701	216,123	629.9	10	Web Link
Books	41,551	358,574	337.0	12	E-Commerce

Appendix

Table 4. Dataset statistics including the number of nodes and edges, the average number of tokens in the textual descriptions associated with nodes, the number of classes, and the domains they belong to.

Cora	(1, 3, 64)	(1, 7, 64)	(1, 14, 64)	(1, 21, 64)	(1, 4, 16, 64)	(1, 7, 21, 64)
Cora	82.58±1.28	84.89±0.38	83.61±2.34	83.31±0.43	82.69±1.47	84.26±1.41
Citeseer	(1, 3, 64)	(1, 6, 64)	(1, 12, 64)	(1, 18, 64)	(1, 4, 16, 64)	(1, 6, 18, 64)
Citeseer	72.16±0.921	73.70±0.92	71.72±1.29	72.10±1.81	71.55±1.084	73.36±1.32
WikiCS	(1, 5, 128)	(1, 10, 128)	(1, 20, 128)	(1, 30, 128)	(1, 10, 32, 128)	(1, 10, 64, 128)
WikiCS	79.83±0.04	82.51±0.25	80.62±0.37	80.83±0.38	82.09±0.33	81.94±0.68
Photo	(1, 6, 256)	(1, 12, 256)	(1, 24, 256)	(1, 36, 256)	(1, 12, 32, 256)	(1, 12, 64, 256)

Appendix

Table 5. Performance with different taxonomy tree structures.

Panel of Figure 5 — Figure 5. How the accuracy varies with different values of $\lambda$ .

	Cora		Citeseer
Model	Base	+Ours	Base	+Ours
GCN	32.50	34.69	50.01	53.88
GAE	29.98	32.43	63.33	64.53
BUDDY	26.40	30.16	59.48	61.30

Appendix

Table 6. Link prediction results (MRR, %) on Cora and Citeseer. “+Ours” denotes the backbone augmented with our taxonomy-informed regularization.

Panel of Figure 6 — Figure 6. Visualizations of the learned node representations (colored by ground-truth labels) and the pairwise distance matrix between finest-level cluster centroids on Pumbed, with and without taxonomy regularization. The constructed taxonomy tree has 4 hierarchical levels with 1, 3, 16, and 63 cluster nodes from top to bottom, respectively. Darker colors indicate smaller distances. With taxonomy regularization, clearer block structures emerge, where darker diagonal blocks correspond to coarser-grained clusters, reflecting improved semantic hierarchy in learned representation space.

Panel of Figure 7 — Figure 7. Visualizations of the learned node representations (colored by ground-truth labels) and the pairwise distance matrix between finest-level cluster centroids on WikiCS, with and without taxonomy regularization. The constructed taxonomy tree has 3 hierarchical levels with 1, 10, and 124 cluster nodes from top to bottom, respectively. Darker colors indicate smaller distances. With taxonomy regularization, clearer block structures emerge, where darker diagonal blocks correspond to coarser-grained clusters, reflecting improved semantic hierarchy in learned representation space.

Figure 9. RadialMap of the constructed taxonomy on Citeseer.

Figure 10. RadialMap of the constructed taxonomy on Pubmed.

Figure 11. RadialMap of the constructed taxonomy on WikiCS.

Figure 12. RadialMap of the constructed taxonomy on Photo.

Taxonomy Induction

See all 16 papers

Oct 17, 2026

Uncovering Semantic Hierarchies in Text-Attributed Graphs via Variational EM-based LLM-GNN Synergy

While hierarchical structures are intrinsic to organizing human knowledge, current representation learning on text-attributed graphs predominantly operates on flat semantic spaces, overlooking the rich coarse-to-fine granularity inherent in real-world data. To bridge this gap, we propose SHiFT, which synergizes the reasoning power of LLMs with the structural representation capability of GNNs within an Expectation-Maximization (EM) paradigm. In the E-step, we harness LLMs as expert taxonomists to induce an explicit semantic tree, introducing a lightweight Mapping & Diagnosis strategy that reduces inference costs by surgically updating the hierarchy only when anomalies are detected. In the M-step, we align the GNN encoder with this induced hierarchy from complementary perspectives, including clustering partition, topological skeleton, and semantic concepts. Theoretical analysis interprets SHiFT as a generalized Variational EM algorithm. Extensive experiments demonstrate that SHiFT not only achieves state-of-the-art performance but also uncovers high-quality, human-readable taxonomies. Our code can be found at https://github.com/Cloudy1225/SHiFT.

Yunhui Liu, Xudong Jin, Qizhuo Xie +5

State Key Laboratory for Novel Software Technology, Nanjing University · Department of Statistics, University of Wisconsin–Madison · National Engineering Laboratory for Big Data Analysis and Applications, Peking University

Text-Attributed Graph Learning Hierarchical Representation Learning Taxonomy Induction LLMs for Graph Learning

Ask PDF

Jun 25, 2026

Term-Centric Hierarchy Induction from Heterogeneous Corpora

Organizing knowledge from diverse text sources into interpretable hierarchies is crucial for tasks such as policy analysis, innovation monitoring, and exploratory domain mapping. Existing taxonomy induction methods typically rely on document-level representations that capture entire documents rather than the specific domain concepts relevant for knowledge organization, limiting their ability to generalize across heterogeneous sources. We propose a term-centric framework for inducing hierarchical taxonomies from heterogeneous corpora that scales to massive document collections. Our approach maps documents from diverse sources into a shared representation space using automatic term extraction, enabling robust cross-source alignment. Based on these representations, we construct interpretable hierarchies that integrate domain priors with datadriven clustering. Experiments on a novel English and German multi-source benchmark of over one million documents demonstrate that our method improves cross-source coherence and hierarchy quality over text- and summarybased baselines. A case study on German regional innovation analysis further demonstrates its practical utility for technology landscape mapping.

Elena Senger, Yuri Campbell, Jan-Peter Bergmann +2

1MaiNLP, Center for Information and Language Processing, LMU Munich, Germany · 2Fraunhofer Institute for Systems and Innovation Research ISI, Germany · Department of Computer Science, IT University of Copenhagen, Denmark

Clustering Hierarchical Clustering Taxonomy Induction

Ask PDF

Apr 30, 2026

Polaris: Coupled Orbital Polar Embeddings for Hierarchical Concept Learning

Real-world knowledge is often organized as hierarchies such as product taxonomies, medical ontologies, and label trees, yet learning hierarchical representations is challenging due to asymmetric structure and noisy semantics. We introduce Polaris, a polar hyperspherical embedding framework that separates semanticity from hierarchy using angular geometry and radius, enabling the learning of meaning and structure without interference. To map latent representation onto the sphere, we project it to the tangent space at the north pole, apply the exponential map, and learn unit-norm representations using spherical linear layers. Polaris then combines robust local constraints, global regularization that prevents geometric collapse, and uncertainty-aware asymmetric objectives that encourage directional containment. At inference time, Polaris uses structure-guided retrieval to efficiently narrow down candidate parents before final ranking. We evaluate Polaris on different settings of taxonomy expansion - spanning trees, multi-parent DAGs, and multimodal hierarchies, showing consistent improvements of up to ~19 points in top-K retrieval and up to ~60% reduction in mean rank over fourteen strong baselines.

Sahil Mishra, Srinitish Srinivasan, Sourish Dasgupta +1

Indian Institute of Technology Delhi, New Delhi, India · KDM Lab, Dhirubhai Ambani University, Gandhinagar, Gujarat, India · Indian Institute of Technology Delhi, Abu Dhabi, UAE.

Hierarchical Classification Hierarchical Representation Learning Hyperspherical Representation Learning Geometric Representation Learning Knowledge Representation

Ask PDF

Abstract

Taxonomy Induction

Explore similar work

Uncovering Semantic Hierarchies in Text-Attributed Graphs via Variational EM-based LLM-GNN Synergy

Term-Centric Hierarchy Induction from Heterogeneous Corpora

Polaris: Coupled Orbital Polar Embeddings for Hierarchical Concept Learning