Graph Topology Information Enhanced Heterogeneous Graph Representation Learning

He Zhao; Zhiwei Zeng; Yongwei Wang; Chunyan Miao

cs.LGApr 7, 2026

Graph Topology Information Enhanced Heterogeneous Graph Representation Learning

Authors: He Zhao, Zhiwei Zeng, Yongwei Wang, Chunyan Miao

Organizations: Nanyang Technological University, Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly, Singapore · Nanyang Technological University, Singapore

Abstract

Real-world heterogeneous graphs are inherently noisy and usually not in the optimal graph structures for downstream tasks, which often adversely affects the performance of GRL models in downstream tasks. Although Graph Structure Learning (GSL) methods have been proposed to learn graph structures and downstream tasks simultaneously, existing methods are predominantly designed for homogeneous graphs, while GSL for heterogeneous graphs remains largely unexplored. Two challenges arise in this context. Firstly, the quality of the input graph structure has a more profound impact on GNN-based heterogeneous GRL models compared to their homogeneous counterparts. Secondly, most existing homogenous GRL models encounter memory consumption issues when applied directly to heterogeneous graphs. In this paper, we propose a novel Graph Topology learning Enhanced Heterogeneous Graph Representation Learning framework (ToGRL).ToGRL learns high-quality graph structures and representations for downstream tasks by incorporating task-relevant latent topology information. Specifically, a novel GSL module is first proposed to extract downstream task-related topology information from a raw graph structure and project it into topology embeddings. These embeddings are utilized to construct a new graph with smooth graph signals. This two-stage approach to GSL separates the optimization of the adjacency matrix from node representation learning to reduce memory consumption. Following this, a representation learning module takes the new graph as input to learn embeddings for downstream tasks. ToGRL also leverages prompt tuning to better utilize the knowledge embedded in learned representations, thus enhancing adaptability to downstream tasks. Extensive experiments on five real-world datasets show that our ToGRL outperforms state-of-the-art methods by a large margin.

Figures & tables

Figure 1. An overview of our ToGRL framework. First, a graph structure learning module is used to mine latent graph topology together with downstream task information. The learned topology embeddings are utilized to construct a new graph. Then, a representation learning module is used to learn final node representations. Finally, a prompt tuning decoder is used to evaluate embeddings on downstream tasks.

Figure 2. Noise effect is enlarged by changing from two-hop relationships (in homogeneous models on the right) to one-hop relationships(in heterogeneous models on the left).

Figure 3. An illustration of smoothness on graphs. The arrows represent node signals. The two graphs has the same graph signals while having different graph structures. The left one is smoother than the right one. In ToGRL, we treat the topology embeddings as the node signals and aim to construct a smooth graph.

Datasets	Nodes	Edges	Node Types	Relations	Labels
ACM	8,994	12,961	3	4	3
IMDB	12,772	18,644	3	4	3
DBLP	18,405	67,946	3	4	4
ACM2	29,930	61,770	2	2	7
PubMed	63,109	125,167	4	10	8

Table 1. Summary of datasets statistics used in experiments.ACM, IMDB, DBLP and ACM2 are used for node classification tasks. ACM2 and PubMed are used for the link prediction task.

Dataset	ACM		IMDB		DBLP		ACM2
Metrics	Marco F1	Micro F1	Marco F1	Micro F1	Marco F1	Micro F1	Marco F1	Micro F1
Transh	$67.21\pm 0.20$	$67.11\pm 0.19$	$30.78\pm 0.23$	$53.61\pm 0.20$	$27.42\pm 0.41$	$34.05\pm 0.68$	$17.73\pm 0.61$	$50.78\pm 0.10$
Node2Vec	$81.57\pm 0.39$	$81.22\pm 0.41$	$48.59\pm 1.16$	$64.59\pm 0.41$	$93.26\pm 0.28$	$94.05\pm 0.06$	$60.01\pm 0.61$	$74.61\pm 0.46$
Metapath2Vec	$78.59\pm 0.30$	$78.11\pm 0.30$	$54.89\pm 0.72$	$67.16\pm 0.92$	$92.67\pm 1.01$	$93.52\pm 0.91$	$57.37\pm 1.11$	$72.64\pm 1.32$
SLAPS	$88.51\pm 0.51$	$88.52\pm 0.66$	$57.89\pm 0.24$	$61.35\pm 0.18$	$55.32\pm 1.20$	$56.70\pm 1.11$	$61.22\pm 0.73$	$72.85\pm 0.14$
HGSL	$93.21\pm 0.59$	$93.17\pm 0.59$	$43.34\pm 0.21$	$58.83\pm 0.25$	-	-	-	-

Table 2. Node classification results on the four real-world datasets. Bold fonts denote the best performance among all methods. ‘-’ denotes that the model cannot be run on the dataset on a 16GB GPU.

	ACM2	PubMed
Metapath2Vec	0.712	0.628
SLAPS	0.662	-
HAN	0.868	0.717
HGT	0.920	0.736
CKD	0.948	0.735
ToGRL	0.951	0.796

Table 4. The AUC scores on the Link Prediction task. Bold fonts denote the best performance among all methods.‘-’ denotes that the model cannot be run on the dataset on a single 16GB GPU card.

Figure 4. The performance of ToGRL on ACM and IMDB datasets with different $k$ values.

	SLAPS	HGSL	GTN	SimpleHGN	ToGRL_st	ToGRL_re
IMDB	9.4	11.7	2.7	2.1	2.7	1.7
ACM	5.2	8.6	1.6	2.0	2.1	1.6

Table 5. Memory consumption results (GB) for different models on IMDB and ACM datasets on node classification. The ToGRL_st represents the structure learning part in our framework and the ToGRL_re represents the representation learning part. The model settings are the same as those reported in Table 2 .

Figure 5. Visualization of node embeddings on ACM dataset. Compared with baseline models, our ToGRL can successfully learn more separated node embeddings which further proves that our structure learning module can extract downstream tasks related topology information effectively and facilitate the downstream task eventually.

Graph Representation Learning

Graph representation learning develops methods for encoding nodes, edges, and whole graphs for prediction, comparison, and transfer.

See all 305 papers

Jul 30, 2026

TopoFormer: Topology Meets Attention for Graph Learning

We introduce Topoformer, a lightweight and scalable framework for graph representation learning that encodes topological structure into attention-friendly sequences. At the core of our method is Topo-Scan, a novel module that decomposes a graph into a short, ordered sequence of topological tokens by slicing over node or edge filtrations. These sequences capture multi-scale structural patterns, from local motifs to global organization, and are processed by a Transformer to produce expressive graph-level embeddings. Unlike traditional persistent homology pipelines, Topo-Scan is parallelizable, avoids costly diagram computations, and integrates seamlessly with standard deep learning architectures. We provide theoretical guarantees on the stability of our topological encodings and demonstrate state-of-the-art performance across graph classification and molecular property prediction benchmarks. Our results show that Topoformer matches or exceeds strong GNN and topology-based baselines while offering predictable and efficient compute. This work opens a new path for parallelizable and unifying approaches to graph representation learning that integrate topological inductive biases into attention frameworks.

Md Joshem Uddin, Astrit Tola, Cuneyt Gurcan Akcora +1

Department of Mathematical Science The University of Texas at Dallas Richardson, TX 75080, USA · Department of Mathematics Florida State University Tallahassee, FL 32306, USA · AI Initiative University of Central Florida Orlando, FL 32816, USA

Graph Representation Learning Graph Classification Graph Transformer Topological Representation Learning Molecular Representation Learning

Ask PDF

Sep 4, 2026

Dynamic Heterogeneous Graph Representation Learning: A Survey

Graph representation learning (GRL) serves as a canonical paradigm for modeling complex networks. However, real-world AI systems inherently manifest as evolving heterogeneous entities with complex interactions, posing significant challenges to static or homogeneous modeling. To address these complexities, representation learning for Dynamic Heterogeneous Graphs (DHGs) has emerged as a vital approach for learning low-dimensional representations that simultaneously preserve structural semantics and temporal dynamics. This survey presents the first systematic review of DHG representation learning methods. We first introduce a unified formal definition that encompasses both discrete-time and continuous-time DHGs from the perspective of temporal granularity. Building upon this formulation, we propose a novel algorithm-centric taxonomy that categorizes existing literature, including early embedding-based approaches, graph neural network (GNN)-based models, and relatively recent Transformer-based DHG methods, while explicitly highlighting their intrinsic modeling biases with respect to dynamic granularity. Furthermore, we summarize representative applications of DHG representation learning, along with commonly used datasets and benchmarks. Finally, we discuss promising research directions that guide future advances in this rapidly evolving field.

Huan Liu, Pengfei Jiao, Jie Yin +2

School of Cyberspace, Hangzhou Dianzi University, China · Discipline of Business Analytics, The University of Sydney, Australia · Zhejiang Provincial Key Laboratory for Sensitive Data Security Protection and Confidentiality Management, Hangzhou, China

Graph Representation Learning Dynamic Graph Learning Heterogeneous GNNs Temporal GNNs

Ask PDF

May 23, 2025

HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning

Machine learning on heterogeneous graphs has advanced rapidly, driven by the diverse entities, relations, and semantics inherent in real-world data. However, most existing studies assume static graphs, whereas real-world graph data are often continuously updated. Continual learning in this setting is particularly challenging because historical knowledge is encoded not only in target-node representations, but also in their heterogeneous structural contexts and relation-dependent semantic responses. To this end, we introduce HERO, a heterogeneous continual graph learning framework that preserves historical knowledge at both structural and semantic levels while adapting to incoming tasks. We analyze experience replay as an approximation to the unavailable historical gradient and decompose its error into target-selection and context-reconstruction terms. This analysis shows that replaying target nodes alone is generally insufficient for relation-sensitive HGNNs, as removing typed neighborhood context can alter both representations and historical gradients. Motivated by this, HERO employs DiSCo to select representative target nodes and reconstruct compact heterogeneous contexts through relation-aware multi-type neighbor expansion. To further preserve knowledge that cannot be captured by replayed subgraphs alone, HERO aligns task-conditioned predictions and heterogeneous semantic responses through multi-level knowledge distillation. A lightweight look-ahead adaptation step is additionally used to improve plasticity on incoming tasks. Experiments on four datasets with three HGNN backbones show that HERO achieves the best or tied-best average performance among state-of-the-art baselines, while maintaining competitive forgetting. Our code is available at https://github.com/gqBond/HCGL.

Guiquan Sun, Xikun Zhang, Jingchao Ni +1

School of Computing, University of Connecticut · RMIT University, Melbourne · Department of Computer Science, University of Houston

Heterogeneous GNNs Replay-Based Continual Learning

Ask PDF

Abstract

Graph Representation Learning

Explore similar work

TopoFormer: Topology Meets Attention for Graph Learning

Dynamic Heterogeneous Graph Representation Learning: A Survey

HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning