HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning

Guiquan Sun; Xikun Zhang; Jingchao Ni; Dongjin Song

cs.LGMay 23, 2025

HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning

Authors: Guiquan Sun, Xikun Zhang, Jingchao Ni, Dongjin Song

Organizations: School of Computing, University of Connecticut · RMIT University, Melbourne · Department of Computer Science, University of Houston

Abstract

Machine learning on heterogeneous graphs has advanced rapidly, driven by the diverse entities, relations, and semantics inherent in real-world data. However, most existing studies assume static graphs, whereas real-world graph data are often continuously updated. Continual learning in this setting is particularly challenging because historical knowledge is encoded not only in target-node representations, but also in their heterogeneous structural contexts and relation-dependent semantic responses. To this end, we introduce HERO, a heterogeneous continual graph learning framework that preserves historical knowledge at both structural and semantic levels while adapting to incoming tasks. We analyze experience replay as an approximation to the unavailable historical gradient and decompose its error into target-selection and context-reconstruction terms. This analysis shows that replaying target nodes alone is generally insufficient for relation-sensitive HGNNs, as removing typed neighborhood context can alter both representations and historical gradients. Motivated by this, HERO employs DiSCo to select representative target nodes and reconstruct compact heterogeneous contexts through relation-aware multi-type neighbor expansion. To further preserve knowledge that cannot be captured by replayed subgraphs alone, HERO aligns task-conditioned predictions and heterogeneous semantic responses through multi-level knowledge distillation. A lightweight look-ahead adaptation step is additionally used to improve plasticity on incoming tasks. Experiments on four datasets with three HGNN backbones show that HERO achieves the best or tied-best average performance among state-of-the-art baselines, while maintaining competitive forgetting. Our code is available at https://github.com/gqBond/HCGL.

Figures & tables

Figure 1: Overview of HERO for heterogeneous continual graph learning. For each incoming task, DiSCo constructs a compact replay subgraph by selecting representative target nodes and retaining their relation-aware multi-type neighborhoods, thereby preserving historical heterogeneous contexts. HKD complements replay by aligning task-conditioned predictions and heterogeneous semantic responses with historical teachers. A look-ahead meta-adaptation step further improves new-task plasticity under the joint current-task, replay, and distillation objectives.

Methods	DBLP		IMDB		Freebase		Yelp
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
Finetune	82.8 $\pm$ 4.6	26.6 $\pm$ 9.1	68.8 $\pm$ 2.2	26.0 $\pm$ 2.8	53.8 $\pm$ 4.7	25.7 $\pm$ 8.8	24.5 $\pm$ 0.5	44.8 $\pm$ 1.6
Joint Train	95.2 $\pm$ 0.6	1.7 $\pm$ 1.0	80.4 $\pm$ 0.7	3.9 $\pm$ 0.8	65.6 $\pm$ 0.9	2.8 $\pm$ 2.0	63.0 $\pm$ 2.3	0.0 $\pm$ 1.6
EWC	85.4 $\pm$ 5.5	21.5 $\pm$ 10.6	69.6 $\pm$ 1.2	25.8 $\pm$ 2.4	58.3 $\pm$ 2.4	18.4 $\pm$ 4.2	25.3 $\pm$ 2.1	44.8 $\pm$ 0.9
MAS	91.1 $\pm$ 0.8	8.3 $\pm$ 1.5	72.0 $\pm$ 2.3	20.5 $\pm$ 3.9	59.0 $\pm$ 1.0	14.8 $\pm$ 0.9	36.7 $\pm$ 10.9	27.6 $\pm$ 16.2
TWP	90.5 $\pm$ 1.3	10.1 $\pm$ 2.6	74.8 $\pm$ 2.7	14.5 $\pm$ 4.4	60.9 $\pm$ 4.3	10.8 $\pm$ 6.9	32.0 $\pm$ 12.1	36.4 $\pm$ 15.1

Table 1: Performance comparison on four benchmark datasets using HAN as the backbone. Results on Yelp are reported under the 5 tasks/3-way setting. The symbol

\uparrow

(

\downarrow

) indicates that higher (lower) is better. The best results are highlighted in bold , while the second-best results are underlined .

Figure 2: Accuracy matrix of selected continual graph learning methods on the Yelp dataset.

Figure 3: Performance comparison on Yelp under the 8-task/2-way setting.

Figure 4: Effect of the shot number used by G-MM across four datasets.

Methods	DBLP		IMDB		Freebase		Yelp
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
HERO	93.6 $\pm$ 0.8	4.3 $\pm$ 1.6	78.4 $\pm$ 1.7	7.3 $\pm$ 3.0	60.9 $\pm$ 1.5	11.1 $\pm$ 1.6	55.9 $\pm$ 0.6	7.5 $\pm$ 1.2
w/o ER	89.7 $\pm$ 0.6	12.0 $\pm$ 0.7	75.1 $\pm$ 0.9	12.8 $\pm$ 1.7	58.6 $\pm$ 1.2	16.3 $\pm$ 2.2	53.9 $\pm$ 2.4	6.8 $\pm$ 2.4
w/o HKD	89.6 $\pm$ 1.4	12.1 $\pm$ 2.6	72.2 $\pm$ 1.5	19.3 $\pm$ 2.3	59.4 $\pm$ 1.1	14.7 $\pm$ 1.8	44.9 $\pm$ 14.3	19.3 $\pm$ 17.0
w/o G-MM	92.6 $\pm$ 1.7	5.2 $\pm$ 0.9	75.0 $\pm$ 2.7	13.3 $\pm$ 4.5	59.7 $\pm$ 0.9	12.0 $\pm$ 2.2	55.6 $\pm$ 1.9	8.0 $\pm$ 2.0
w/o DiSCo	92.4 $\pm$ 1.4	6.0 $\pm$ 3.1	75.4 $\pm$ 2.2	12.1 $\pm$ 5.0	56.2 $\pm$ 0.4	18.0 $\pm$ 0.5	45.7 $\pm$ 14.0	23.7 $\pm$ 15.6

Table 2: Ablation results of HERO. We remove Experience Replay (ER), Heterogeneity-aware Knowledge Distillation (HKD), and the Gradient-based Meta-learning Module (G-MM) individually. And “w/o DiSCo” denotes replacing the proposed Diversity Sampling with semantic Consistency with Coverage Maximization applied to all node types.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Node Classification	#Nodes	#Node Types	#Edges	#Edge Types	#Classes
DBLP ( Lv et al., 2021 )	26,128	4	239,566	6	4
IMDB ( Lv et al., 2021 )	21,420	4	86,642	6	5
Freebase ( Lv et al., 2021 )	180,098	8	1,057,688	36	7
Yelp ( Yang et al., 2020 )	82,465	4	32,548,358	7	16

Appendix

Table 3: Key characteristics of datasets

Figure 5: Illustration of the setting used in this paper.

Method	Avg. GPU/task (MB) $\downarrow$	Peak GPU (MB) $\downarrow$
Replay-based methods
HERO (Ours)	7,046	8,107
DMSG	7,126	8,131
ER-GNN	7,065	8,138
FTF-ER	7,097	8,122
MetaCLGraph	7,102	8,125

Appendix

Table 4: GPU memory usage on Freebase. The first group compares HERO with replay-based methods, while the second group provides diagnostic controls for the additional overhead of HKD. Lower is better.

	Yelp 5-task		Yelp 8-task
Teachers	AP $\uparrow$	AF $\downarrow$	AP $\uparrow$	AF $\downarrow$
$k=1$	53.51	13.41	55.55	11.31
$\mathbf{k=3}$	$\mathbf{56.83}$	$\mathbf{9.29}$	$\mathbf{57.43}$	$\mathbf{7.98}$
$k=T-1$	57.44	8.15	57.81	7.43

Appendix

Table 5: Effect of the teacher-window size on Yelp.

k=T-1

retains all historical teachers, while fixed

k

bounds teacher storage to

O(k)

Method	DBLP	IMDB	Freebase	Yelp
TWP	0.157 (x0.42)	0.032 (x0.78)	0.068 (x0.59)	3.020 (x0.50)
FTF-ER	1.619 (x4.39)	0.238 (x5.78)	0.351 (x3.06)	8.424 (x1.39)
ER-GNN	0.411 (x1.11)	0.053 (x1.29)	0.078 (x0.68)	6.652 (x1.10)
MetaCLGraph	1.770 (x4.80)	0.310 (x7.53)	0.614 (x5.34)	9.752 (x1.61)
HERO	0.369 (x1.00)	0.041 (x1.00)	0.115 (x1.00)	6.043 (x1.00)

Appendix

Table 6: Training time (s) comparison on four datasets. Lower is better. Numbers in parentheses denote the ratio relative to HERO.

Methods	DBLP		IMDB		Freebase
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
Finetune	86.2 $\pm$ 1.5	12.7 $\pm$ 2.2	75.1 $\pm$ 7.5	17.0 $\pm$ 9.4	67.8 $\pm$ 1.8	16.5 $\pm$ 2.5
Joint Train	93.9 $\pm$ 0.6	1.7 $\pm$ 1.0	80.0 $\pm$ 0.7	2.0 $\pm$ 0.9	72.5 $\pm$ 0.6	8.5 $\pm$ 1.2
EWC ( Kirkpatrick et al., 2017 )	89.1 $\pm$ 1.2	10.7 $\pm$ 1.8	77.3 $\pm$ 0.4	12.5 $\pm$ 0.2	69.6 $\pm$ 3.0	14.7 $\pm$ 2.9
MAS ( Aljundi et al., 2018 )	89.8 $\pm$ 2.5	9.0 $\pm$ 3.0	77.5 $\pm$ 1.2	11.8 $\pm$ 2.6	70.4 $\pm$ 3.1	14.8 $\pm$ 2.0
TWP ( Liu et al., 2021 )	89.1 $\pm$ 2.1	10.3 $\pm$ 1.1	77.7 $\pm$ 0.6	11.1 $\pm$ 1.6	69.6 $\pm$ 2.0	15.1 $\pm$ 0.9

Appendix

Table 7: Performance comparison on three benchmark datasets using HGT as the backbone. The symbol

\uparrow

(

\downarrow

) indicates that higher (lower) is better. The best results are highlighted in bold , while the second-best results are underlined .

Methods	DBLP		IMDB
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
Finetune	79.1 $\pm$ 5.1	34.8 $\pm$ 10.1	71.3 $\pm$ 0.6	21.7 $\pm$ 3.0
Joint Train	94.1 $\pm$ 0.1	1.9 $\pm$ 1.0	77.4 $\pm$ 0.7	4.7 $\pm$ 1.5
EWC ( Kirkpatrick et al., 2017 )	91.0 $\pm$ 2.3	10.9 $\pm$ 4.5	73.4 $\pm$ 1.6	16.7 $\pm$ 2.5
MAS ( Aljundi et al., 2018 )	92.1 $\pm$ 2.1	7.8 $\pm$ 5.3	74.3 $\pm$ 2.0	14.2 $\pm$ 3.7
TWP ( Liu et al., 2021 )	91.6 $\pm$ 2.1	7.4 $\pm$ 5.0	74.0 $\pm$ 0.8	13.7 $\pm$ 3.3

Appendix

Table 8: Performance comparison on DBLP and IMDB using MAGNN as the backbone. MAGNN runs out of memory on Freebase and is therefore not reported. The symbol

\uparrow

(

\downarrow

) indicates that higher (lower) is better. The best results are highlighted in bold , while the second-best results are underlined .

Methods	DBLP		IMDB		Freebase
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
Finetune	83.5 $\pm$ 0.9	24.3 $\pm$ 2.1	56.7 $\pm$ 0.7	49.8 $\pm$ 1.3	49.2 $\pm$ 2.3	31.8 $\pm$ 5.7
EWC ( Kirkpatrick et al., 2017 )	90.7 $\pm$ 1.1	10.5 $\pm$ 2.5	67.8 $\pm$ 2.5	24.0 $\pm$ 2.6	54.0 $\pm$ 11.0	24.2 $\pm$ 20.1
MAS ( Aljundi et al., 2018 )	92.6 $\pm$ 2.5	4.8 $\pm$ 5.3	72.0 $\pm$ 4.9	13.0 $\pm$ 10.7	47.1 $\pm$ 0.6	34.6 $\pm$ 2.3
TWP ( Liu et al., 2021 )	93.6 $\pm$ 1.1	3.5 $\pm$ 1.5	73.4 $\pm$ 0.5	11.2 $\pm$ 4.9	52.0 $\pm$ 6.8	26.4 $\pm$ 9.9
ER-GNN ( Zhou and Cao, 2021 )	93.2 $\pm$ 2.7	6.1 $\pm$ 5.4	69.7 $\pm$ 2.7	13.6 $\pm$ 3.2	56.0 $\pm$ 3.9	17.7 $\pm$ 7.5

Appendix

Table 9: Performance comparison under reordered task sequences on three benchmark datasets using HAN as the backbone. The symbol

\uparrow

(

\downarrow

) indicates that higher (lower) is better. The best results are highlighted in bold , while the second-best results are underlined .

Figure 6: Ablation Study on Heterogeneity-aware Knowledge Distillation Module on all datasets.

Methods	DBLP	IMDB	Freebase	Yelp
Other
Joint Train	0.69	2.47	11.59	11.58
TWP	1.03	2.27	10.44	5.58
Memory Replay
ER-GNN	0.22	2.62	8.44	8.46
FTF-ER	0.83	3.43	10.59	7.19

Appendix

Table 10: Forgetting-aware Gap (FaG, %) on four benchmark datasets, averaged over five runs. Lower values indicate better performance. The best results are highlighted in bold , while the second-best results are underlined .

Figure 7: Effect of the distillation loss weight $\lambda_{kd}$ across four datasets.

Figure 8: Effect of the buffer size across four datasets.

Heterogeneous GNNs

Heterogeneous graph neural networks (GNNs) learn from graphs with multiple node and relation types.

See all 77 papers

UNIT: Unleash Large Language Models Potential for Graph Continual Learning
Jul 11, 2026Tairan Huang, Yili Wang, Beibei Hu +4Graph Representation Learning Continual Learning
Central South University Changsha, China · Hongkong University of Science and Technology (Guangzhou) Guangzhou, China · Hunan Institute of Engineering Changsha, Hunan
HCPN-GCN: Scaling Hierarchical Prototype Networks with Cone Geometry for Continual Graph Learning
Sep 25, 2026Sammuel R. Silva, Vander L. S. Freitas, Gladston Moreira +2Incremental Learning Hierarchical Representation Learning
Universidade Federal de Ouro Preto, Ouro Preto 35402-163, Brasil
CoPE-CGL: Community-aware Prototype Extraction for Replay-Free Continual Graph Learning
Oct 26, 2026Bo-Young Lim, Seokhwan Kim, Kisung Lee +1Incremental Learning
Graduate School of Data Science Seoul National University of Science and Technology Seoul, South Korea · Division of Computer Science and Engineering Louisiana State University Baton Rouge, Louisiana, USA

Jul 11, 2026J/K move · Enter open · S save

UNIT: Unleash Large Language Models Potential for Graph Continual Learning

Tairan Huang, Yili Wang, Beibei Hu, Yiting Shi+3

Central South University Changsha, China · Hongkong University of Science and Technology (Guangzhou) Guangzhou, China · Hunan Institute of Engineering Changsha, Hunan

Graph Representation Learning Continual Learning Task-Free Continual Learning LLMs for Graph Learning

Open paper AskPDF arXiv

In real-world multimodal web scenarios, graph-structured data often arrives in a streaming manner, making graph continual learning a crucial paradigm for continuously modeling such evolving structures. However, existing graph continual learning methods still face two fundamental challenges. 1) semantic-structural separation, where the graph-based methods excel at modeling topological relationships but neglect deep semantics. 2) imbalanced knowledge transfer, where existing models fail to effectively leverage general knowledge gained from early tasks to benefit subsequent new tasks. To address above issues, we propose a novel framework, UNleash Large Language Models PotentIal for Graph ConTinual Learning (UNIT). By fine-tuning large language model only on the first task, we bridge the distributional gap between the pre-trained LLM corpus and the target task dataset to enhance the adaptability of LLMs for graph-structured tasks. Meanwhile, we propose an uncertain-aware anchor generation mechanism to effectively preserve representative knowledge across tasks, avoiding the neglect of universal knowledge learned from previous tasks. Additionally, we introduce structural confluence modeling to explicitly integrates graph topology information into semantic information, enhancing the collaborative capabilities between semantic understanding and structural modeling. Extensive experiments demonstrate that our proposed method achieves state-of-the-art performance in the graph continual learning task.

Abstract

Heterogeneous GNNs

Explore similar work

UNIT: Unleash Large Language Models Potential for Graph Continual Learning

HCPN-GCN: Scaling Hierarchical Prototype Networks with Cone Geometry for Continual Graph Learning

CoPE-CGL: Community-aware Prototype Extraction for Replay-Free Continual Graph Learning