Relation-Aware Graph Foundation Model

Jianxiang Yu; Jiapeng Zhu; Yibo Zhao; Hao Qian; Ziqi Liu; Zhiqiang Zhang; Xiang Li

cs.LGMay 17, 2025

Relation-Aware Graph Foundation Model

Authors: Jianxiang Yu, Jiapeng Zhu, Yibo Zhao, Hao Qian, Ziqi Liu, Zhiqiang Zhang, Xiang Li

Organizations: School of Data Science and Engineering, East China Normal University · Ant Group

Abstract

In recent years, large language models (LLMs) have demonstrated remarkable capability to generalize across diverse natural language processing tasks, inspiring the development of graph foundation models (GFMs) for large-scale pre-training. However, unlike language models with explicit token units, graphs lack a well-defined unit for generalization, making it challenging to design effective pre-training strategies. In this work, we propose REEF, a novel GFM framework that leverages relation tokens as the fundamental units. We construct a vocabulary of relation tokens to encode relational information within graphs. To accommodate diverse relations, we introduce two hypernetworks that adaptively generate the parameters of aggregators and classifiers in graph neural networks based on relation tokens. In addition, we design another hypernetwork to construct dataset-specific projectors and incorporate a dataset-level feature bias into the initial node representations, enhancing flexibility across different datasets with the same relation. Extensive experiments demonstrate that REEF consistently outperforms existing methods in both pre-training and transfer learning, highlighting its potential as a general-purpose graph foundation model.

Figures & tables

Figure 1: Comparison of transfer unit at the node, dataset, and relation levels.

Figure 2: Relation vocabulary inspired by LLMs.

Figure 3: The overall framework of REEF.

Domain	Citation		WebKB		Amazon	Average
Dataset	Pubmed	Citeseer	Wisconsin	Texas	Photo	Acc $\uparrow$	$\Delta\downarrow$
GCN (ind)	88.42 ( $\Delta_{0.00}$ )	76.50 ( $\Delta_{0.05}$ )	51.76 ( $\Delta_{24.71}$ )	55.14 ( $\Delta_{25.81}$ )	93.02 ( $\Delta_{0.00}$ )	72.97	10.11
GAT (ind)	86.33 ( $\Delta_{2.09}$ )	76.55 ( $\Delta_{0.00}$ )	49.41 ( $\Delta_{27.06}$ )	52.16 ( $\Delta_{28.79}$ )	92.73 ( $\Delta_{0.29}$ )	71.44	11.65
GCN (joint)	61.21 ( $\Delta_{27.21}$ )	64.86 ( $\Delta_{11.69}$ )	61.21 ( $\Delta_{15.26}$ )	57.14 ( $\Delta_{23.81}$ )	30.45 ( $\Delta_{62.57}$ )	54.97	28.11
GAT (joint)	52.93 ( $\Delta_{35.49}$ )	57.81 ( $\Delta_{18.74}$ )	55.81 ( $\Delta_{20.66}$ )	52.38 ( $\Delta_{28.57}$ )	28.52 ( $\Delta_{64.50}$ )	49.49	33.59
RGCN (joint)	80.30 ( $\Delta_{8.12}$ )	71.77 ( $\Delta_{4.78}$ )	72.09 ( $\Delta_{4.38}$ )	80.95 ( $\Delta_{0.00}$ )	79.03 ( $\Delta_{13.99}$ )	76.83	6.25

Table 1: Comparison of model performance on various pretraining datasets. The table reports accuracy (Acc

\%

) and the performance gap

(\Delta=\text{Top}-\text{Score})

to the best result on each dataset. The average “Acc” is the mean accuracy calculated across the five datasets presented in the table. The best result is bold and underlined, while the runner-up is underlined.

Method	Cora (Citation)			Cornell (WebKB)			Computers (Amazon)
Method	Acc	AUC	F1	Acc	AUC	F1	Acc	AUC	F1
GCN	$0.3012_{\pm.06}$	$0.6444_{\pm.04}$	$0.2591_{\pm.04}$	$0.3263_{\pm.04}$	$0.5666_{\pm.01}$	$0.3151_{\pm.03}$	$0.2602_{\pm.07}$	$0.6773_{\pm.02}$	$0.2428_{\pm.04}$
GAT	$0.3646_{\pm.04}$	$0.6769_{\pm.03}$	$0.3108_{\pm.04}$	$0.3275_{\pm.14}$	$0.5306_{\pm.03}$	$0.1497_{\pm.04}$	$0.3482_{\pm.07}$	$0.6878_{\pm.05}$	$0.2397_{\pm.05}$
GraphCL	$0.2507_{\pm.06}$	$0.6350_{\pm.03}$	$0.2240_{\pm.03}$	$0.4175_{\pm.04}$	$0.6350_{\pm.02}$	$0.3500_{\pm.04}$	$0.2856_{\pm.04}$	$0.6467_{\pm.03}$	$0.1653_{\pm.06}$
SimGRACE	$0.2492_{\pm.02}$	$0.5765_{\pm.03}$	$0.1567_{\pm.04}$	$0.3438_{\pm.13}$	$0.5954_{\pm.09}$	$0.2168_{\pm.09}$	$0.2666_{\pm.10}$	$0.6286_{\pm.01}$	$0.1603_{\pm.03}$
GCOPE+CL	$0.3368_{\pm.02}$	$0.6971_{\pm.04}$	$0.2967_{\pm.03}$	$0.3975_{\pm.10}$	$0.6694_{\pm.04}$	$0.3120_{\pm.04}$	$0.3439_{\pm.03}$	$0.7023_{\pm.01}$	$0.2976_{\pm.03}$

Table 2: Performance comparison of different methods under transfer learning (1-shot). “GCOPE+CL” and “GCOPE+Sim” denote the variants of GCOPE that utilize the contrastive learning frameworks of GraphCL and SimGRACE, respectively. The best results are highlighted in bold. “Improve.” represents the improvement of REEF over the best baseline on the corresponding metric.

Domain	#Datasets	GAT	GraphAny	TS-Mean	REEF
Citation Networks	4	64.23	70.14	63.39	71.08
Co-authorship Networks	4	77.63	81.82	81.03	83.00
Social Networks	6	69.00	69.03	68.31	70.25
Co-purchase Graphs	3	64.12	71.58	71.27	71.00
Heterophilic Benchmarks	6	72.85	69.34	68.53	76.77
Air-traffic Graphs	3	39.14	40.38	39.15	42.64

Table 3: Performance comparison across different graph domains. Best results are highlighted in bold. Detailed results for each individual dataset are provided in Table 15 of Appendix F .

Method	FT scale	BBBP	HIV
OFA Liu et al. (2023a)	N/A	–	35.67
MoMu Su et al. (2022)	N/A	49.81	50.26
Galactica Taylor et al. (2022)	N/A	53.94	33.85
GIMLET Zhao et al. (2023a)	400M QA	59.39	66.24
GOFA Kong et al. (2024)	100k QA	54.91	53.02
REEF	None	54.80	63.17

Table 4: Zero-shot transfer performance on molecular graph benchmarks without molecular-domain pretraining or fine-tuning.

Method	Cora	History	Ratings
GraphMAE	72.49	39.15	31.68
LLaGA	60.70	36.45	23.45
OFA	52.49	39.36	29.08
OFA-FS	42.10	17.50	20.50
Prodigy (M)	23.40	12.71	20.16
Prodigy	40.59	19.47	20.84

Table 5: 3-shot transfer learning results using LM-based node features. † denotes that no E-commerce graph data is used during pretraining, making History and Ratings cross-domain transfer datasets. The best and second-best results are highlighted in bold and underline , respectively.

Method	Pre-training datasets							Transfer datasets			Avg.
Method	FB15K237	WN18RR	Pubmed	Citeseer	Wisconsin	Texas	Photo	Cora	Cornell	Computers	Pre.	Trans.	All
REEF-LM	91.33	94.06	86.05	70.42	64.71	59.46	93.53	46.58	44.72	43.46	79.94	44.92	69.43
REEF-FB	89.90	92.18	86.23	70.27	70.59	54.05	92.16	40.07	44.51	42.09	79.34	42.22	68.21
REEF-FP	90.72	94.29	85.04	73.12	56.86	59.46	93.33	40.22	43.69	46.50	78.97	45.10	68.32
REEF-AGU	91.14	93.78	86.41	71.62	68.63	64.86	92.22	40.81	43.69	46.68	81.24	43.73	69.98
REEF	91.04	94.70	86.38	72.37	76.47	70.27	93.01	48.66	48.65	47.72	83.46	48.34	72.93

Table 6: Ablation Study on REEF. “Pre.” refers to pretraining, “Trans.” refers to transfer learning, and “All” denotes the overall average performance. We highlight the best score in bold.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Domain	#Nodes	#Edges	#Classes	Task	Type
FB15K237	Knowledge	14,541	310,116	237	Link	$\mathcal{G}_{\text{source}}$
WN18RR	Knowledge	40,943	93,003	11	Link	$\mathcal{G}_{\text{source}}$
PubMed	Citation	19,717	44,338	3	Node	$\mathcal{G}_{\text{source}}$
Citeseer	Citation	3,237	9,104	6	Node	$\mathcal{G}_{\text{source}}$
Cora	Citation	2,708	10,556	7	Node	$\mathcal{G}_{\text{target}}$
Wisconsin	WebKB	251	515	5	Node	$\mathcal{G}_{\text{source}}$

Appendix

Table 7: Statistics of datasets.

Dataset	Linear	GCN	GAT	GIN	DGI	BGRL	GraphMAE	GIANT	GFT	REEF
FB15K237	87.39	82.22	88.93	83.21	81.34	80.66	85.30	87.45	89.72	91.04
WN18RR	78.50	73.79	80.16	74.02	75.75	75.44	78.99	84.36	91.91	94.70

Appendix

Table 8: Accuracy results on the link prediction task.

Figure 4: Performance comparison across different dataset scales in the citation domain.

Photo+FB15K237+WN18RR (REEF)	48.65 $\pm$ 7.76	70.16 $\pm$ 2.82	36.49 $\pm$ 6.77	47.72 $\pm$ 5.61	84.80 $\pm$ 3.22	44.03 $\pm$ 2.49
Pre-training datasets	Cornell (#Labels = 5)			Computers (#Labels = 10)
Pre-training datasets	Acc	AUC	F1	Acc	AUC	F1
Pubmed+Citeseer+Wisconsin+Texas+	48.65 $\pm$ 7.76	70.16 $\pm$ 2.82	36.49 $\pm$ 6.77	47.72 $\pm$ 5.61	84.80 $\pm$ 3.22	44.03 $\pm$ 2.49
Pubmed+Citeseer+FB15K237+WN18RR	34.58 $\pm$ 12.87	57.45 $\pm$ 4.57	20.29 $\pm$ 6.34	26.40 $\pm$ 11.19	47.65 $\pm$ 6.20	6.10 $\pm$ 1.35

Appendix

Table 9: Transfer Learning Performance on Out-of-Domain Relations for Cornell and Computers Datasets.

Figure 5: Performance of different datasets with different hidden dimensions. Model size is indicated in parentheses.

Figure 6: T-SNE visualization of representations after pretraining.

Figure 7: Test accuracy of different datasets during pre-training.

Aggregator	Pre.	Trans.	All
Shared	67.46	29.80	56.16
Concat	80.60	39.39	68.23
Relation-ID	83.24	33.51	68.32
Cross-Attention	65.10	34.29	55.86
REEF (Hypernetwork)	83.46	48.34	72.93

Appendix

Table 10: Comparison of different relation-conditioned aggregator designs under the same pre-training and transfer protocol. Only the aggregator is replaced; the classifier hypernetwork, dataset projector, and feature bias remain unchanged. “Pre.” refers to the pretraining average, “Trans.” to the transfer average, and “All” to the overall average. The best score is highlighted in bold .

Activation	FB15K237	WN18RR	Wisconsin	Texas	Citeseer	Pubmed	Photo	Avg.
Linear	91.04	94.70	76.47	70.27	72.37	86.38	93.01	83.46
ReLU	90.47	92.38	72.55	67.57	78.10	89.00	94.34	83.49
GELU	90.80	93.43	70.59	75.68	74.20	88.50	94.66	83.98

Appendix

Table 11: Effect of the activation function used in the generated message-passing layers. All variants share the same pre-training protocol and differ only in the activation applied within the aggregator. The best average is highlighted in bold.

Method	Paradigm	Trainable params	Training resources
GCOPE ( Zhao et al., 2024a )	GNN-based	93.3K	34 min, 1 $\times$ A800 80GB; 23.40 MiB peak
GOFA ( Kong et al., 2024 )	LLM-based	1.01B pretrain / 852.7M instruction tuning	Pre-training: 4 days, 4 $\times$ A100 80GB
REEF	Hypernetwork + GNN-based	177.4M	8 h, 1 $\times$ A800 80GB; 48.21 GiB peak

Appendix

Table 12: Trainable parameters and total training resources of REEF compared with a representative GNN-based (GCOPE) and an LLM-based (GOFA) graph foundation model. GOFA numbers are as reported by its authors.

Dataset	H2GCN	FAGCN	GPR-GNN	LINKX	ACM-GCN	REEF
Cornell	72.97	45.95	51.35	78.38	67.57	78.38
Texas	78.38	64.86	59.46	75.68	78.38	81.08
Wisconsin	82.35	66.67	70.59	62.75	70.59	84.31
Chameleon	57.46	62.06	61.18	72.59	60.75	56.80
Actor	37.96	36.32	34.74	29.67	37.24	33.82
BlogCatalog	80.65	86.52	89.32	60.91	74.98	76.93

Appendix

Table 13: Node classification accuracy on eight heterophilous datasets, comparing REEF with GNNs specifically designed for heterophily. The best score in each row is highlighted in bold.

Dataset	Micro-F1	Macro-F1	AUROC
ACM	84.94	85.03	94.46
DBLP	81.09	80.46	93.17

Appendix

Table 14: Node classification performance of REEF on standard heterogeneous graph benchmarks (ACM and DBLP) under the original HGB splits. Each canonical edge type is treated as an individual relation token, and node-type-specific projections map heterogeneous node features into the shared input space.

Dataset	GAT	GraphAny	TS-Mean	REEF
Actor	32.59	29.51	28.09	33.82
AirBrazil	35.38	36.15	39.23	44.44
AirEU	39.00	41.13	35.88	41.87
AirUS	43.03	43.86	42.34	41.62
AmzComp	70.94	82.00	81.37	77.71
AmzPhoto	80.78	90.18	90.18	93.83

Appendix

Table 15: Performance comparison across 26 common datasets. Best results are highlighted in bold.

Domain	Relation Type	Description
Citation	Citation	A citation in a network represents the act of one scientific paper referencing another, embodying the flow and accumulation of knowledge. It reflects the topical relevance between papers and researchers’ acknowledgment of prior work, serving as a means to validate and support their own research while highlighting the collaborative nature of scientific inquiry.
	Paper classification	Paper classification in graph learning focuses on predicting the categories or topics of individual papers within a graph.
WebKB	Hyperlinks	A hyperlink represents a relationship or connection between nodes (e.g., entities such as documents, web pages, or words). It encapsulates semantic associations, enables information flow, and reflects the nature of interactions, which may vary in type, intensity, or direction.
	Webpage classification	Webpage classification involves predicting the categories or types of individual webpages in a graph, often determined by the content, purpose, or role of the webpage within a network of hyperlinks.
Amazon	Co-purchase	The “bought together” relationship between goods represents the co-purchasing behavior of customers, indicating a semantic association between products based on shared purchasing patterns. This relationship reflects the likelihood of two products being bought in conjunction, capturing their complementary nature or relevance in a shopping context.
	Product classification	Product classification is a task in graph learning that predicts the categories or labels of product nodes, focusing on their classification based on attributes like type, function, or consumer category in a co-purchasing or recommendation graph.

Appendix

Table 16: Relations and Descriptions for Different Domains. “Relation type” of the knowledge graph is the corresponding dataset.

Explore similar work

Learning Graph Foundation Models on Riemannian Graph-of-Graphs
May 11, 2026Haokun Liu, Zezhong Ding, Xike XieGraph Foundation Model Graph Representation Learning
School of Biomedical Engineering, University of Science and Technology of China (USTC), Suzhou, Jiangsu, China · Data Darkness Lab, Suzhou Institute for Advanced Research, USTC, Suzhou, Jiangsu, China · School of Artificial Intelligence and Data Science, USTC, Hefei, Anhui, China
Surprisingly Simple and Effective Multi-Domain Graph Foundation Model through Graph-to-Table Alignment
Jul 13, 2026Chunyu Hu, Tianyin Liao, Ge Lan +4Graph Foundation Model Tabular Foundation Models
Nankai University · Tsinghua University · Beihang University
Empowering Heterogeneous Graph Foundation Models via Decoupled Relation Alignment
May 1, 2026Ziyu Zheng, Yaming Yang, Zhe Wang +2Graph Foundation Model Subspace
School of Computer Science and Technology, Xidian University, Xi’an, China 710071

May 11, 2026, cs.LGJ/K move · Enter open · S save

Learning Graph Foundation Models on Riemannian Graph-of-Graphs

Haokun Liu, Zezhong Ding, Xike Xie

School of Biomedical Engineering, University of Science and Technology of China (USTC), Suzhou, Jiangsu, China · Data Darkness Lab, Suzhou Institute for Advanced Research, USTC, Suzhou, Jiangsu, China · School of Artificial Intelligence and Data Science, USTC, Hefei, Anhui, China

Graph Foundation Model Graph Representation Learning Graph Representations Wireless Foundation Models Foundation Model Receptive Fields

Open paper AskPDF arXiv

Graph foundation models (GFMs), pretrained on massive graph data, have transformed graph machine learning by supporting general-purpose reasoning across diverse graph tasks and domains. Existing GFMs pretrained with fixed-hop subgraph sampling impose a fixed receptive field, causing scale mismatch on diverse tasks, which often require heterogeneous and unknown structural contexts beyond a fixed sampling scale. We propose R-GFM, a Riemannian Graph-of-Graphs (GoG) based foundation model, that treats structural scale as a first-class citizen in modeling. R-GFM constructs a multi-scale GoG over-sampled subgraphs at different hop distances and learns geometry-adaptive representations from Riemannian manifolds. Theoretical analysis shows that R-GFM reduces structural domain generalization error compared to fixed-scale GFMs. Experiments on various datasets demonstrate that R-GFM achieves state-of-the-art performance, with up to a 49% relative improvement on downstream tasks. Our code is available at https://github.com/USTC-DataDarknessLab/R-GFM.

Abstract

Explore similar work

Learning Graph Foundation Models on Riemannian Graph-of-Graphs

Surprisingly Simple and Effective Multi-Domain Graph Foundation Model through Graph-to-Table Alignment

Empowering Heterogeneous Graph Foundation Models via Decoupled Relation Alignment