More Domains, No Gains? FLO: Fast-Slow Reciprocal Learning for Multi-Domain Graph Pre-training

Yupeng Tong; Lianze Shan; Jitao Zhao; Di Jin; Dongxiao He

cs.LGOct 10, 2026

More Domains, No Gains? FLO: Fast-Slow Reciprocal Learning for Multi-Domain Graph Pre-training

Authors: Yupeng Tong, Lianze Shan, Jitao Zhao, Di Jin, Dongxiao He

Organizations: School of Computer Science and Technology, Tianjin University, Tianjin, China

Abstract

Inspired by the great success of foundation models in the development of artificial general intelligence, Graph Foundation Models (GFMs) have recently attracted significant attention and have the potential to advance graph intelligence. Exhibiting such intelligence requires effectively learning knowledge from large-scale, multi-domain graph data during pre-training. Increasing domain diversity during pre-training is expected to yield richer and more generalizable knowledge. Most existing methods pursue this goal by designing domain alignment strategies that help a single graph model learn from multiple domains. However, we observe that existing methods struggle to benefit from increasing domain diversity. Through empirical and theoretical analysis, we reveal two key limitations. Firstly, pre-training in most GFMs is confined to a single message-passing paradigm. Since propagation flattens relations among neighboring nodes, the graph model can hardly perceive the knowledge beyond propagation, namely the attribute-predictable component of these relations. Secondly, domain alignment strategies often rely on randomly initialized parameters that are jointly updated with the graph model using immediate loss feedback, leaving early knowledge acquisition vulnerable to cross-domain interference. To address these issues, we propose FLO, a Fast--sLow reciprOcal approach for multi-domain graph pre-training. Specifically, a propagation-free attribute branch predicts the graph model's relations without message passing and returns these predictions to refine the model. We further stabilize knowledge acquisition by pairing a fast process that adapts to the current domain with a slow process that integrates knowledge across domains. On 12 datasets, FLO benefits from increasing domain diversity and achieves the best average performance in cross-domain and cross-task transfer.

Figures & tables

Figure 1: Motivation experiments with Cora as the target graph. (a) Accuracy change as source domains are added. (b)–(c) Pre-training loss of two baselines.

Figure 2: Overview of FLO. (a) Multi-domain pre-training: source graphs are visited in turn, and the graph branch

f_{G}

and the attribute branch

f_{M}

each reconstruct node features. In Fast–Slow Knowledge Acquisition (FSKA), each branch keeps a slow copy updated by an Exponential Moving Average (EMA) after each step. In Propagation–Attribute Relation Refinement (PARR), each branch learns the relation distribution of the other branch’s slow copy over the same candidates. (b) Downstream tasks: only the frozen graph encoder

f_{G}

is applied to graphs from unseen domains.

Method	Cora	CiteSeer	PubMed	CS	Photo	Computers	Avg.
DGI	75.47 ±0.78	65.76 ±1.19	82.83 ±0.24	92.82 ±0.18	91.48 ±0.26	87.24 ±0.21	82.60
GRACE	72.00 ±0.36	56.29 ±0.71	76.80 ±0.37	90.29 ±0.09	90.76 ±0.15	86.82 ±0.15	78.83
BGRL	79.43 ±1.24	64.46 ±1.14	83.91 ±0.26	92.63 ±0.21	92.76 ±0.21	88.98 ±0.16	83.69
FUG	82.79 ±0.72	70.52 ±0.15	84.42 ±0.23	92.41 ±0.02	92.31 ±0.16	87.30 ±0.16	84.96
TIG	84.39 ±1.08	73.08 ±0.52	86.45 ±0.13	93.42 ±0.05	93.27 ±0.22	89.51 ±0.07	86.68
GFT	72.99 ±0.94	64.79 ±0.99	83.59 ±0.12	93.06 ±0.27	90.25 ±0.18	84.18 ±0.19	81.48

Table 1: Accuracy (%) of cross-domain node classification with standard deviations. Each column is an unseen target graph. The best and second-best results are in bold and underlined.

Figure 3: Few-shot node classification. Left: accuracy (%) on Cora, CS, and CiteSeer for FLO and the five baselines with the highest average accuracy. Right: accuracy gain (%) of FLO over each baseline, averaged over the 1-, 3-, and 5-shot settings; Avg. is the mean over the six target graphs.

	Link prediction: Hits@50 (%)				Graph classification: accuracy (%)
Method	Physics	WikiCS	DBLP	Avg.	COIL-RAG	ENZYMES	PROTEINS	Avg.
FUG	55.99 ±3.38	66.23 ±1.22	64.18 ±5.29	62.13	84.97 ±1.50	33.41 ±4.31	71.64 ±1.66	63.34
TIG	44.46 ±2.88	69.93 ±0.59	61.85 ±3.43	58.75	77.16 ±1.53	33.33 ±3.36	71.56 ±2.36	60.68
GFT	42.93 ±2.90	58.78 ±2.13	43.41 ±5.49	48.37	83.42 ±1.62	35.88 ±5.91	72.37 ±2.92	63.89
MDGPT	47.79 ±1.52	68.03 ±1.26	57.25 ±1.56	57.69	80.14 ±2.19	31.24 ±4.22	71.59 ±3.22	60.99
SAMGPT	49.82 ±1.86	70.12 ±0.73	57.60 ±1.75	59.18	81.91 ±1.99	29.57 ±4.13	70.65 ±2.71	60.71

Table 2: Hits@50 (%) of link prediction and accuracy (%) of graph classification with standard deviations. The best and second-best results are in bold and underlined.

Figure 4: Domain diversity and training stability. (a) Accuracy change (%) as source domains are added in the order on the horizontal axis. (b) Min–max normalized log loss in the first 40 epochs.

Variant	Cora	CiteSeer	PubMed	CS	Photo	Computers
w/o $\mathcal{L}_{\rm kd}$	83.26 ±0.84	66.52 ±1.31	86.63 ±0.23	93.13 ±0.09	93.34 ±0.07	90.04 ±0.12
w/o slow attribute branch	83.37 ±0.89	66.98 ±1.11	86.67 ±0.26	93.03 ±0.14	93.47 ±0.15	90.04 ±0.11
w/o slow graph branch	64.65 ±0.95	42.01 ±1.61	80.57 ±0.32	82.58 ±0.23	89.62 ±0.36	84.83 ±0.20
w/o $\mathcal{L}_{\rm rec}$	30.33 ±0.19	20.66 ±0.63	82.48 ±0.31	76.14 ±0.92	59.72 ±1.94	52.40 ±1.16
w/ GCN attribute branch	79.74 ±0.10	55.56 ±0.22	86.60 ±0.26	91.52 ±0.15	92.02 ±0.55	89.48 ±0.17
FLO	85.83 ±0.65	72.50 ±0.91	86.74 ±0.35	93.62 ±0.05	93.62 ±0.23	90.13 ±0.07

Table 3: Ablation study on node classification: accuracy (%) with standard deviations. w/o slow attribute branch removes

\mathcal{L}^{G}_{\rm kd}

, the relation targets returned to the graph branch; w/o slow graph branch removes

\mathcal{L}^{M}_{\rm kd}

, the relation targets that train the attribute branch. Removing one doubles the weight of the other, so the total relation weight is unchanged. The best results are in bold.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Result	What it shows	Main tools	Proof
Eq. ( 9 )	The relational information of the graph branch splits into an attribute-predicted part and the propagation-induced part $R$	Chain rule, tower property	A.2
Lemma A.1	Close embeddings give close relation distributions	Lipschitz bounds of cosine and softmax	A.3
Theorem 3.2	Propagation drives the relations among nearby nodes toward uniform	Edge form of the Dirichlet energy, spectral decomposition	A.4
Theorem 3.3	Targets returned by the attribute branch remove $R$	Conditional mean, convexity of $\operatorname{KL}$ , Pinsker’s inequality	A.5
Proposition A.2	Reconstruction rules out uniform relations	Bounds of the decoder on directions	A.6
Proposition 3.4	Slow copies damp period- $S$ oscillations by $\rho$	Frequency response, Parseval’s identity	A.7

Appendix

Table 4: Results proved in this appendix.

	Cora	CiteSeer	PubMed	CS	Photo	Computers
Relation change, $K=1$	0.70	0.77	0.41	0.44	0.53	0.53
Relation change, $K=2$	1.10	1.09	0.83	0.67	0.61	0.61
Drop without $\mathcal{L}_{\rm kd}$	2.57	5.98	0.11	0.49	0.28	0.09

Appendix

Table 5: Relation change caused by propagation,

\mathbb{E}\,\operatorname{KL}(P\|Q_{B})

, and the accuracy drop (%) when relation learning is removed.

Figure 5: Relation change caused by propagation and the accuracy drop (%) when relation learning is removed, for each target graph. The values are those of Table 5 . The vertical axis is logarithmic, and the dashed lines are least-squares fits of the logarithm of the drop.

Component	Parameter	Value
Input	Common dimension $d$	1,024
Both branches	Hidden width; output width	512; 1,024
Graph branch	Architecture	GCN–PReLU–GCN
Attribute branch	Architecture	Linear–GELU–Linear
Decoders	Architecture	Separate linear maps
Mask	Elementwise keep probability	0.40

Appendix

Table 6: Hyperparameters of FLO. The pre-training hyperparameters are shared across all target graphs; the last two rows apply only to few-shot node classification.

Method	Cora	CiteSeer	PubMed	CS	Photo	Computers	Memory (GB)	Params (M)
Pre-training time per epoch (s)
DGI	0.189	0.189	0.145	0.148	0.177	0.159	19.6	0.39
GRACE	0.747	1.063	0.532	0.560	0.813	0.739	12.9	0.39
BGRL	0.102	0.102	0.099	0.097	0.076	0.064	3.5	0.53
FUG	0.167	0.165	0.152	0.133	0.138	0.110	10.6	10.49
TIG	0.231	0.230	0.209	0.201	0.183	0.152	14.7	3.81

Appendix

Table 7: Computational cost. Top: pre-training time per epoch (s) with each graph held out as the target, peak GPU memory during pre-training, and trainable parameters. Bottom: forward time (ms) of the frozen encoder on each target graph and parameters at inference. All methods are measured on the same NVIDIA RTX 5090 GPU.

Dataset	Graphs	Nodes	Edges	Features	Classes	Domain
Node classification
Cora	1	2,708	5,278	1,433	7	Citation
CiteSeer	1	3,327	4,552	3,703	6	Citation
PubMed	1	19,717	44,324	500	3	Citation
CS	1	18,333	81,894	6,805	15	Co-authorship
Photo	1	7,650	119,081	745	8	Co-purchase

Appendix

Table 8: Statistics of the datasets. Edges are counted once per undirected pair; for graph classification, the numbers of nodes and edges are averaged over the graphs of each dataset.

Method	Code	Method	Code
DGI	github.com/PetarV-/DGI	GFT	github.com/Zehong-Wang/GFT
GRACE	github.com/CRIPAC-DIG/GRACE	MDGPT	anonymous.4open.science/r/MDGPT
BGRL	github.com/nerdslab/bgrl	SAMGPT	github.com/blue-soda/SAMGPT
FUG	github.com/hedongxiao-tju/FUG	MDGFM	github.com/wbkzwqtzw/MDGFM
TIG	github.com/hedongxiao-tju/TIG	BRIDGE	github.com/RingBDStack/BRIDGE

Appendix

Table 9: Code of the baselines.

Component	Specification
Operating system	Ubuntu 22.04.5 LTS, Linux kernel 6.8.0
CPU	Intel Xeon Silver 4410Y, 48 threads
Memory	128 GB
GPU	NVIDIA GeForce RTX 5090, 32 GB
Software	Python 3.10.12, PyTorch 2.11.0 with CUDA 12.8, PyTorch Geometric 2.8.0
	NumPy 2.2.6, scikit-learn 1.7.2

Appendix

Table 10: Experimental environment.

Method	Cora	CiteSeer	PubMed	CS	Photo	Computers	Avg.
DGI	33.09 ±5.51	33.79 ±5.60	47.10 ±6.45	71.82 ±6.36	56.42 ±8.15	42.48 ±9.90	47.45
GRACE	28.13 ±4.46	24.20 ±3.88	39.81 ±5.61	47.23 ±5.77	42.35 ±7.61	38.77 ±7.31	36.75
BGRL	34.19 ±4.42	25.04 ±3.75	43.37 ±5.66	57.58 ±5.84	60.32 ±10.14	50.95 ±11.17	45.24
FUG	46.40 ±8.37	36.07 ±6.81	52.23 ±9.47	78.02 ±5.88	63.62 ±9.77	51.27 ±12.82	54.60
TIG	48.88 ±8.80	44.71 ±9.02	50.94 ±10.00	75.28 ±6.22	63.52 ±10.47	49.47 ±13.08	55.47
GFT	27.25 ±4.78	26.78 ±4.31	45.36 ±6.00	63.49 ±6.42	20.01 ±4.37	14.62 ±5.12	32.92

Appendix

Table 11: Accuracy (%) of 1-shot node classification with standard deviations over the 500 tasks shared by all methods. The best and second-best results are in bold and underlined.

Method	Cora	CiteSeer	PubMed	CS	Photo	Computers	Avg.
DGI	45.56 ±5.08	44.96 ±4.42	55.58 ±5.12	85.26 ±2.08	70.17 ±5.23	56.98 ±7.27	59.75
GRACE	38.59 ±4.00	30.63 ±3.88	45.68 ±4.85	65.49 ±3.51	57.22 ±6.52	50.64 ±6.08	48.04
BGRL	46.74 ±4.09	31.66 ±3.58	51.06 ±5.51	75.12 ±3.29	72.63 ±7.23	62.05 ±8.43	56.54
FUG	61.63 ±5.42	47.62 ±4.94	60.82 ±6.32	86.55 ±1.70	65.91 ±8.90	58.85 ±10.17	63.56
TIG	58.53 ±6.17	56.39 ±5.03	56.48 ±7.81	83.63 ±3.04	68.24 ±10.30	56.31 ±11.03	63.26
GFT	37.84 ±4.25	35.49 ±3.97	51.90 ±5.65	78.95 ±2.74	22.25 ±4.74	17.49 ±6.30	40.65

Appendix

Table 12: Accuracy (%) of 3-shot node classification with standard deviations over the 500 tasks shared by all methods. The best and second-best results are in bold and underlined.

Method	Cora	CiteSeer	PubMed	CS	Photo	Computers	Avg.
DGI	53.07 ±3.79	50.55 ±3.54	60.34 ±4.47	87.77 ±1.20	74.10 ±4.33	63.16 ±4.49	64.83
GRACE	44.50 ±3.25	34.72 ±3.42	49.95 ±4.23	71.96 ±2.75	64.24 ±5.42	56.71 ±5.15	53.68
BGRL	53.28 ±3.37	35.74 ±3.11	56.07 ±4.68	81.12 ±2.22	77.83 ±5.18	67.13 ±6.08	61.86
FUG	67.31 ±3.23	53.28 ±3.86	64.51 ±4.64	88.13 ±1.13	66.29 ±7.50	62.83 ±8.18	67.06
TIG	63.70 ±4.59	60.66 ±3.45	59.64 ±6.58	86.62 ±1.98	69.20 ±8.48	60.80 ±8.49	66.77
GFT	44.13 ±3.73	40.85 ±3.38	56.09 ±4.75	82.69 ±1.62	24.92 ±4.83	19.76 ±6.37	44.74

Appendix

Table 13: Accuracy (%) of 5-shot node classification with standard deviations over the 500 tasks shared by all methods. The best and second-best results are in bold and underlined.

Figure 6: Accuracy (%) of node classification on each target graph at pre-training budgets of 300, 500, 1,000, 1,500, 2,000, 2,500, and 3,000 epochs. Shaded bands show the standard deviation over the three random splits of node classification, all panels share the same vertical span, and the circled point is the budget used in all other experiments.

Figure 7: Accuracy (%) of node classification after 2,000 pre-training epochs when the temperature

\tau

, the relation weight

\lambda

, the momentum

\mu

, the learning rate, or the hidden width of both branches is varied. Each row is a target graph, and the circled point is the default value.

Graph Foundation Models

Graph Foundation Models are pretrained graph-learning systems designed to transfer across datasets, tasks, and domains with different structures and features.

See all 96 papers

May 25, 2026

MDGMIX: Boundary-Aware Subgraph Mixing for Multi-Domain Graph Pre-Training

Multi-domain graph pre-training is a crucial step in constructing foundational graph models with cross-domain generalization capabilities. However, existing methods predominantly rely on jointly training all source domain graphs, resulting in high computational costs. Furthermore, it remains unclear whether all source domain graph data contribute equally to effective transfer. This paper empirically reveals significant data redundancy in multi-domain graph pre-training. Based on this finding, we propose the Multi-domain Graph Pre-training Framework, MDGMIX, which combines boundary-aware subgraph mixing with hierarchical discrimination. By selecting boundary nodes to construct challenging mixed-domain subgraphs, MDGMIX employs coarse-grained domain discrimination and fine-grained domain decomposition losses to decouple shared patterns from domain-specific patterns. During adaptation, MDGMIX employs a lightweight prompt weighting mechanism to transfer source domain knowledge. Extensive experiments demonstrate that MDGMIX consistently outperforms strong baselines in few-shot classification tasks while exhibiting superior time and memory efficiency. The code is available at: https://github.com/zhengziyu77/MDGMIX.

Ziyu Zheng, Yaming Yang, Ziyu Guan +2

School of Computer Science and Technology, Xidian University, Xi’an, China · School of Artificial Intelligence, Xidian University, Xi’an, China

Cross-Domain Generalization Cross-Domain Transfer Learning Graph Foundation Models

Ask PDF

Feb 28, 2026

Multi-Domain Riemannian Graph Gluing for Building Graph Foundation Models

Multi-domain graph pre-training integrates knowledge from diverse domains to enhance performance in the target domains, which is crucial for building graph foundation models. Despite initial success, existing solutions often fall short of answering a fundamental question: how is knowledge integrated or transferred across domains? This theoretical limitation motivates us to rethink the consistency and transferability between model pre-training and domain adaptation. In this paper, we propose a fresh Riemannian geometry perspective, whose core idea is to merge any graph dataset into a unified, smooth Riemannian manifold, enabling a systematic understanding of knowledge integration and transfer. To achieve this, our key contribution is the theoretical establishment of neural manifold gluing, which first characterizes local geometry using an adaptive orthogonal frame and then "glues" the local pieces together into a coherent whole. Building on this theory, we present the GraphGlue framework, which supports batched pre-training with EMA prototyping and provides a transferability measure based on geometric consistence. Extensive experiments demonstrate its superior performance across diverse graph domains. Moreover, we empirically validated GraphGlue's geometric scaling law, showing that larger quantities of datasets improve model transferability by producing a smoother manifold. Codes are available at https://github.com/RiemannGraph/GraphGlue.

Li Sun, Zhenhao Huang, Silei Chen +4

Beijing University of Posts and Telecommunications, Beijing 100876, China · North China Electric Power University, Beijing 102206, China · University of Illinois Chicago, IL, USA

Graph Foundation Models Multi-Source Domain Adaptation Riemannian Geometry

Ask PDF

May 13, 2026

Decoupled and Divergence-Conditioned Prompt for Multi-domain Dynamic Graph Foundation Models

Dynamic graphs are ubiquitous in real-world systems, and building generalizable dynamic Graph Foundation Models has become a frontier in graph learning. However, dynamic graphs from different domains pose fundamental challenges to unified modeling, as their semantic and temporal patterns are inherently inconsistent, making the multi-domain pre-training difficult. Consequently, the widely used "pretrain-then-finetune" paradigm often suffers from severe negative knowledge transfer. To the best of our knowledge, there exists no multi-domain dynamic GFM. In this work, we propose DyGFM, a Dynamic Graph Foundation Model over multiple domains based on decoupled and divergence-conditioned prompting. To disentangle transferable semantics from the domain-specific dynamics, we introduce a dual-branch pre-training strategy with semantic-temporal decoupling. To alleviate negative transfer during domain adaptation, we further develop a cross-domain routing mechanism with divergence-aware expert selection. To enable efficient downstream fine-tuning, we design a divergence-conditioned prompt generator that injects lightweight, learnable graph prompts tailored to semantic and temporal traits. Extensive experiments on continuous dynamic graph benchmarks demonstrate that DyGFM consistently outperforms 12 state-of-the-art baselines on both node classification and link prediction tasks, achieving superior effectiveness and efficiency.

Haonan Yuan, Qingyun Sun, Junhua Shi +3

School of Computer Science and Engineering, Beihang University, Beijing 100191, China · Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin 541004, China · Department of Computer Science, University of Illinois at Chicago, Chicago 60607, USA

Domain-Adaptive Pretraining Prompt Tuning Cross-Domain Transfer Learning Dynamic Graph Learning Graph Foundation Models

Ask PDF

Abstract

Graph Foundation Models

Explore similar work

MDGMIX: Boundary-Aware Subgraph Mixing for Multi-Domain Graph Pre-Training

Multi-Domain Riemannian Graph Gluing for Building Graph Foundation Models

Decoupled and Divergence-Conditioned Prompt for Multi-domain Dynamic Graph Foundation Models