HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning

Guiquan Sun; Xikun Zhang; Jingchao Ni; Dongjin Song

cs.LGMay 23, 2025

HERO: Preserving Structure and Semantics in Heterogeneous Continual Graph Learning

Authors: Guiquan Sun, Xikun Zhang, Jingchao Ni, Dongjin Song

Organizations: School of Computing, University of Connecticut · RMIT University, Melbourne · Department of Computer Science, University of Houston

Abstract

Machine learning on heterogeneous graphs has advanced rapidly, driven by the diverse entities, relations, and semantics inherent in real-world data. However, most existing studies assume static graphs, whereas real-world graph data are often continuously updated. Continual learning in this setting is particularly challenging because historical knowledge is encoded not only in target-node representations, but also in their heterogeneous structural contexts and relation-dependent semantic responses. To this end, we introduce HERO, a heterogeneous continual graph learning framework that preserves historical knowledge at both structural and semantic levels while adapting to incoming tasks. We analyze experience replay as an approximation to the unavailable historical gradient and decompose its error into target-selection and context-reconstruction terms. This analysis shows that replaying target nodes alone is generally insufficient for relation-sensitive HGNNs, as removing typed neighborhood context can alter both representations and historical gradients. Motivated by this, HERO employs DiSCo to select representative target nodes and reconstruct compact heterogeneous contexts through relation-aware multi-type neighbor expansion. To further preserve knowledge that cannot be captured by replayed subgraphs alone, HERO aligns task-conditioned predictions and heterogeneous semantic responses through multi-level knowledge distillation. A lightweight look-ahead adaptation step is additionally used to improve plasticity on incoming tasks. Experiments on four datasets with three HGNN backbones show that HERO achieves the best or tied-best average performance among state-of-the-art baselines, while maintaining competitive forgetting. Our code is available at https://github.com/gqBond/HCGL.

Figures & tables

Figure 1: Overview of HERO for heterogeneous continual graph learning. For each incoming task, DiSCo constructs a compact replay subgraph by selecting representative target nodes and retaining their relation-aware multi-type neighborhoods, thereby preserving historical heterogeneous contexts. HKD complements replay by aligning task-conditioned predictions and heterogeneous semantic responses with historical teachers. A look-ahead meta-adaptation step further improves new-task plasticity under the joint current-task, replay, and distillation objectives.

Methods	DBLP		IMDB		Freebase		Yelp
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
Finetune	82.8 $\pm$ 4.6	26.6 $\pm$ 9.1	68.8 $\pm$ 2.2	26.0 $\pm$ 2.8	53.8 $\pm$ 4.7	25.7 $\pm$ 8.8	24.5 $\pm$ 0.5	44.8 $\pm$ 1.6
Joint Train	95.2 $\pm$ 0.6	1.7 $\pm$ 1.0	80.4 $\pm$ 0.7	3.9 $\pm$ 0.8	65.6 $\pm$ 0.9	2.8 $\pm$ 2.0	63.0 $\pm$ 2.3	0.0 $\pm$ 1.6
EWC	85.4 $\pm$ 5.5	21.5 $\pm$ 10.6	69.6 $\pm$ 1.2	25.8 $\pm$ 2.4	58.3 $\pm$ 2.4	18.4 $\pm$ 4.2	25.3 $\pm$ 2.1	44.8 $\pm$ 0.9
MAS	91.1 $\pm$ 0.8	8.3 $\pm$ 1.5	72.0 $\pm$ 2.3	20.5 $\pm$ 3.9	59.0 $\pm$ 1.0	14.8 $\pm$ 0.9	36.7 $\pm$ 10.9	27.6 $\pm$ 16.2
TWP	90.5 $\pm$ 1.3	10.1 $\pm$ 2.6	74.8 $\pm$ 2.7	14.5 $\pm$ 4.4	60.9 $\pm$ 4.3	10.8 $\pm$ 6.9	32.0 $\pm$ 12.1	36.4 $\pm$ 15.1

Table 1: Performance comparison on four benchmark datasets using HAN as the backbone. Results on Yelp are reported under the 5 tasks/3-way setting. The symbol

\uparrow

(

\downarrow

) indicates that higher (lower) is better. The best results are highlighted in bold , while the second-best results are underlined .

Figure 2: Accuracy matrix of selected continual graph learning methods on the Yelp dataset.

Figure 3: Performance comparison on Yelp under the 8-task/2-way setting.

Figure 4: Effect of the shot number used by G-MM across four datasets.

Methods	DBLP		IMDB		Freebase		Yelp
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
HERO	93.6 $\pm$ 0.8	4.3 $\pm$ 1.6	78.4 $\pm$ 1.7	7.3 $\pm$ 3.0	60.9 $\pm$ 1.5	11.1 $\pm$ 1.6	55.9 $\pm$ 0.6	7.5 $\pm$ 1.2
w/o ER	89.7 $\pm$ 0.6	12.0 $\pm$ 0.7	75.1 $\pm$ 0.9	12.8 $\pm$ 1.7	58.6 $\pm$ 1.2	16.3 $\pm$ 2.2	53.9 $\pm$ 2.4	6.8 $\pm$ 2.4
w/o HKD	89.6 $\pm$ 1.4	12.1 $\pm$ 2.6	72.2 $\pm$ 1.5	19.3 $\pm$ 2.3	59.4 $\pm$ 1.1	14.7 $\pm$ 1.8	44.9 $\pm$ 14.3	19.3 $\pm$ 17.0
w/o G-MM	92.6 $\pm$ 1.7	5.2 $\pm$ 0.9	75.0 $\pm$ 2.7	13.3 $\pm$ 4.5	59.7 $\pm$ 0.9	12.0 $\pm$ 2.2	55.6 $\pm$ 1.9	8.0 $\pm$ 2.0
w/o DiSCo	92.4 $\pm$ 1.4	6.0 $\pm$ 3.1	75.4 $\pm$ 2.2	12.1 $\pm$ 5.0	56.2 $\pm$ 0.4	18.0 $\pm$ 0.5	45.7 $\pm$ 14.0	23.7 $\pm$ 15.6

Table 2: Ablation results of HERO. We remove Experience Replay (ER), Heterogeneity-aware Knowledge Distillation (HKD), and the Gradient-based Meta-learning Module (G-MM) individually. And “w/o DiSCo” denotes replacing the proposed Diversity Sampling with semantic Consistency with Coverage Maximization applied to all node types.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Node Classification	#Nodes	#Node Types	#Edges	#Edge Types	#Classes
DBLP ( Lv et al., 2021 )	26,128	4	239,566	6	4
IMDB ( Lv et al., 2021 )	21,420	4	86,642	6	5
Freebase ( Lv et al., 2021 )	180,098	8	1,057,688	36	7
Yelp ( Yang et al., 2020 )	82,465	4	32,548,358	7	16

Appendix

Table 3: Key characteristics of datasets

Figure 5: Illustration of the setting used in this paper.

Method	Avg. GPU/task (MB) $\downarrow$	Peak GPU (MB) $\downarrow$
Replay-based methods
HERO (Ours)	7,046	8,107
DMSG	7,126	8,131
ER-GNN	7,065	8,138
FTF-ER	7,097	8,122
MetaCLGraph	7,102	8,125

Appendix

Table 4: GPU memory usage on Freebase. The first group compares HERO with replay-based methods, while the second group provides diagnostic controls for the additional overhead of HKD. Lower is better.

	Yelp 5-task		Yelp 8-task
Teachers	AP $\uparrow$	AF $\downarrow$	AP $\uparrow$	AF $\downarrow$
$k=1$	53.51	13.41	55.55	11.31
$\mathbf{k=3}$	$\mathbf{56.83}$	$\mathbf{9.29}$	$\mathbf{57.43}$	$\mathbf{7.98}$
$k=T-1$	57.44	8.15	57.81	7.43

Appendix

Table 5: Effect of the teacher-window size on Yelp.

k=T-1

retains all historical teachers, while fixed

k

bounds teacher storage to

O(k)

Method	DBLP	IMDB	Freebase	Yelp
TWP	0.157 (x0.42)	0.032 (x0.78)	0.068 (x0.59)	3.020 (x0.50)
FTF-ER	1.619 (x4.39)	0.238 (x5.78)	0.351 (x3.06)	8.424 (x1.39)
ER-GNN	0.411 (x1.11)	0.053 (x1.29)	0.078 (x0.68)	6.652 (x1.10)
MetaCLGraph	1.770 (x4.80)	0.310 (x7.53)	0.614 (x5.34)	9.752 (x1.61)
HERO	0.369 (x1.00)	0.041 (x1.00)	0.115 (x1.00)	6.043 (x1.00)

Appendix

Table 6: Training time (s) comparison on four datasets. Lower is better. Numbers in parentheses denote the ratio relative to HERO.

Methods	DBLP		IMDB		Freebase
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
Finetune	86.2 $\pm$ 1.5	12.7 $\pm$ 2.2	75.1 $\pm$ 7.5	17.0 $\pm$ 9.4	67.8 $\pm$ 1.8	16.5 $\pm$ 2.5
Joint Train	93.9 $\pm$ 0.6	1.7 $\pm$ 1.0	80.0 $\pm$ 0.7	2.0 $\pm$ 0.9	72.5 $\pm$ 0.6	8.5 $\pm$ 1.2
EWC ( Kirkpatrick et al., 2017 )	89.1 $\pm$ 1.2	10.7 $\pm$ 1.8	77.3 $\pm$ 0.4	12.5 $\pm$ 0.2	69.6 $\pm$ 3.0	14.7 $\pm$ 2.9
MAS ( Aljundi et al., 2018 )	89.8 $\pm$ 2.5	9.0 $\pm$ 3.0	77.5 $\pm$ 1.2	11.8 $\pm$ 2.6	70.4 $\pm$ 3.1	14.8 $\pm$ 2.0
TWP ( Liu et al., 2021 )	89.1 $\pm$ 2.1	10.3 $\pm$ 1.1	77.7 $\pm$ 0.6	11.1 $\pm$ 1.6	69.6 $\pm$ 2.0	15.1 $\pm$ 0.9

Appendix

Table 7: Performance comparison on three benchmark datasets using HGT as the backbone. The symbol

\uparrow

(

\downarrow

) indicates that higher (lower) is better. The best results are highlighted in bold , while the second-best results are underlined .

Methods	DBLP		IMDB
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
Finetune	79.1 $\pm$ 5.1	34.8 $\pm$ 10.1	71.3 $\pm$ 0.6	21.7 $\pm$ 3.0
Joint Train	94.1 $\pm$ 0.1	1.9 $\pm$ 1.0	77.4 $\pm$ 0.7	4.7 $\pm$ 1.5
EWC ( Kirkpatrick et al., 2017 )	91.0 $\pm$ 2.3	10.9 $\pm$ 4.5	73.4 $\pm$ 1.6	16.7 $\pm$ 2.5
MAS ( Aljundi et al., 2018 )	92.1 $\pm$ 2.1	7.8 $\pm$ 5.3	74.3 $\pm$ 2.0	14.2 $\pm$ 3.7
TWP ( Liu et al., 2021 )	91.6 $\pm$ 2.1	7.4 $\pm$ 5.0	74.0 $\pm$ 0.8	13.7 $\pm$ 3.3

Appendix

Table 8: Performance comparison on DBLP and IMDB using MAGNN as the backbone. MAGNN runs out of memory on Freebase and is therefore not reported. The symbol

\uparrow

(

\downarrow

) indicates that higher (lower) is better. The best results are highlighted in bold , while the second-best results are underlined .

Methods	DBLP		IMDB		Freebase
Methods	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$	AP / % $\uparrow$	AF / % $\downarrow$
Finetune	83.5 $\pm$ 0.9	24.3 $\pm$ 2.1	56.7 $\pm$ 0.7	49.8 $\pm$ 1.3	49.2 $\pm$ 2.3	31.8 $\pm$ 5.7
EWC ( Kirkpatrick et al., 2017 )	90.7 $\pm$ 1.1	10.5 $\pm$ 2.5	67.8 $\pm$ 2.5	24.0 $\pm$ 2.6	54.0 $\pm$ 11.0	24.2 $\pm$ 20.1
MAS ( Aljundi et al., 2018 )	92.6 $\pm$ 2.5	4.8 $\pm$ 5.3	72.0 $\pm$ 4.9	13.0 $\pm$ 10.7	47.1 $\pm$ 0.6	34.6 $\pm$ 2.3
TWP ( Liu et al., 2021 )	93.6 $\pm$ 1.1	3.5 $\pm$ 1.5	73.4 $\pm$ 0.5	11.2 $\pm$ 4.9	52.0 $\pm$ 6.8	26.4 $\pm$ 9.9
ER-GNN ( Zhou and Cao, 2021 )	93.2 $\pm$ 2.7	6.1 $\pm$ 5.4	69.7 $\pm$ 2.7	13.6 $\pm$ 3.2	56.0 $\pm$ 3.9	17.7 $\pm$ 7.5

Appendix

Table 9: Performance comparison under reordered task sequences on three benchmark datasets using HAN as the backbone. The symbol

\uparrow

(

\downarrow

) indicates that higher (lower) is better. The best results are highlighted in bold , while the second-best results are underlined .

Figure 6: Ablation Study on Heterogeneity-aware Knowledge Distillation Module on all datasets.

Methods	DBLP	IMDB	Freebase	Yelp
Other
Joint Train	0.69	2.47	11.59	11.58
TWP	1.03	2.27	10.44	5.58
Memory Replay
ER-GNN	0.22	2.62	8.44	8.46
FTF-ER	0.83	3.43	10.59	7.19

Appendix

Table 10: Forgetting-aware Gap (FaG, %) on four benchmark datasets, averaged over five runs. Lower values indicate better performance. The best results are highlighted in bold , while the second-best results are underlined .

Figure 7: Effect of the distillation loss weight $\lambda_{kd}$ across four datasets.

Figure 8: Effect of the buffer size across four datasets.

Heterogeneous GNNs

Heterogeneous graph neural networks (GNNs) learn from graphs with multiple node and relation types.

See all 77 papers

Jul 11, 2026

UNIT: Unleash Large Language Models Potential for Graph Continual Learning

In real-world multimodal web scenarios, graph-structured data often arrives in a streaming manner, making graph continual learning a crucial paradigm for continuously modeling such evolving structures. However, existing graph continual learning methods still face two fundamental challenges. 1) semantic-structural separation, where the graph-based methods excel at modeling topological relationships but neglect deep semantics. 2) imbalanced knowledge transfer, where existing models fail to effectively leverage general knowledge gained from early tasks to benefit subsequent new tasks. To address above issues, we propose a novel framework, UNleash Large Language Models PotentIal for Graph ConTinual Learning (UNIT). By fine-tuning large language model only on the first task, we bridge the distributional gap between the pre-trained LLM corpus and the target task dataset to enhance the adaptability of LLMs for graph-structured tasks. Meanwhile, we propose an uncertain-aware anchor generation mechanism to effectively preserve representative knowledge across tasks, avoiding the neglect of universal knowledge learned from previous tasks. Additionally, we introduce structural confluence modeling to explicitly integrates graph topology information into semantic information, enhancing the collaborative capabilities between semantic understanding and structural modeling. Extensive experiments demonstrate that our proposed method achieves state-of-the-art performance in the graph continual learning task.

Tairan Huang, Yili Wang, Beibei Hu +4

Central South University Changsha, China · Hongkong University of Science and Technology (Guangzhou) Guangzhou, China · Hunan Institute of Engineering Changsha, Hunan

Graph Representation Learning Continual Learning Task-Free Continual Learning LLMs for Graph Learning

Ask PDF

Sep 25, 2026

HCPN-GCN: Scaling Hierarchical Prototype Networks with Cone Geometry for Continual Graph Learning

Continual Graph Learning (CGL) aims to incrementally learn from graph-structured data while preserving knowledge acquired from previous tasks. A major challenge in this setting is catastrophic forgetting, where learning new tasks degrades performance on previously learned ones. Hierarchical Prototype Networks (HPNs) address this problem through a prototype-based memory mechanism that avoids storing historical data, but their reliance on linear feature extractors limits their ability to exploit graph topology, while point-based prototypes often lead to inefficient prototype growth on structurally diverse graphs. In this work, we propose HCPN-GCN, a graph-aware extension of HPN that replaces the original linear feature extractors with Graph Convolutional Networks (GCNs) and introduces cone-based prototypes with a diversity regularization objective. The proposed design produces richer graph-aware representations while compactly modeling the embedding space, reducing prototype proliferation without sacrificing discriminability. Experimental results on six continual graph learning benchmarks demonstrate that HCPN-GCN consistently improves average classification accuracy over the original HPN and representative continual learning baselines while maintaining near-zero forgetting. Furthermore, our analysis shows that the proposed model learns substantially richer class-level prototype hierarchies using approximately

30\times

fewer atomic prototypes than the original HPN, providing a more compact and effective memory representation for continual graph learning.

Sammuel R. Silva, Vander L. S. Freitas, Gladston Moreira +2

Universidade Federal de Ouro Preto, Ouro Preto 35402-163, Brasil

Incremental Learning Hierarchical Representation Learning Graph Neural Networks

Ask PDF

Oct 26, 2026

CoPE-CGL: Community-aware Prototype Extraction for Replay-Free Continual Graph Learning

Continual graph learning (CGL) maintains models as graphs evolve without retraining from scratch, while mitigating catastrophic forgetting. Existing methods typically target either transductive or inductive settings and often store past subgraphs or require task identifiers at inference. We propose Community-aware Prototype Extraction for Continual Graph Learning (CoPE-CGL), a replay-free framework that uses a parameter-free structural prior, precomputed by multi-hop Laplacian smoothing, to anchor learning, memory, and inference. CoPE-CGL combines topology-infused bottleneck adaptation, community-level prototypes, and dynamic routing by majority vote over node-to-prototype matches. The default protocol assumes each test graph comes from one learned task; a node-level variant handles mixed-task graphs. Each task adds at most a constant amount of storage, with no past graph data retained. At equal storage and without task identifiers, CoPE-CGL matches the strongest replay baseline even when it receives true task identifiers: average performance (AP) is 95.17% and 95.72% versus 94.28% and 95.82% on Arxiv and CoraFull, respectively, with 5.9-14.0 times faster runtime and zero forgetting. Across four transductive and three inductive benchmarks and mixed streams, CoPE-CGL exceeds the second-best of up to eleven baselines by up to 1.41, 6.32, and 9.33 AP percentage points, respectively. It runs 1.3-7.9 times faster end-to-end than the most accurate competitor on each benchmark and scales to the largest benchmark graph. On graphs mixing two tasks, its node-level variant achieves the best AP among CGL methods. The design targets homophilic graphs with community structure.

Bo-Young Lim, Seokhwan Kim, Kisung Lee +1

Graduate School of Data Science Seoul National University of Science and Technology Seoul, South Korea · Division of Computer Science and Engineering Louisiana State University Baton Rouge, Louisiana, USA

Incremental Learning

Ask PDF

Abstract

Heterogeneous GNNs

Explore similar work

UNIT: Unleash Large Language Models Potential for Graph Continual Learning

HCPN-GCN: Scaling Hierarchical Prototype Networks with Cone Geometry for Continual Graph Learning

CoPE-CGL: Community-aware Prototype Extraction for Replay-Free Continual Graph Learning