LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training

Lianze Shan; Jitao Zhao; Dongxiao He; Siqi Liu; Jiaxu Cui; Weixiong Zhang

cs.LGFeb 26, 2026

LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training

Authors: Lianze Shan, Jitao Zhao, Dongxiao He, Siqi Liu, Jiaxu Cui, Weixiong Zhang

Organizations: Tianjin University, Tianjin, China · Jilin University, Changchun, China · The Hong Kong Polytechnic University, Kowloon, Hong Kong

Abstract

Recent advances in generic large models, such as GPT and DeepSeek, have motivated the introduction of universality to graph pre-training, aiming to learn rich and generalizable knowledge across diverse domains using graph representations to improve performance in various downstream applications. However, most existing methods face challenges in learning effective knowledge from generic graphs, primarily due to simplistic data alignment and limited training guidance. The issue of simplistic data alignment arises from the use of a straightforward unification for highly diverse graph data, which fails to align semantics and misleads pre-training models. The problem with limited training guidance lies in the arbitrary application of in-domain pre-training paradigms to cross-domain scenarios. While it is effective in enhancing discriminative representation in one data space, it struggles to capture effective knowledge from many graphs. To address these challenges, we propose a novel Latent sEmantic Distribution Alignment (LEDA) model for universal graph pre-training. Specifically, we first introduce a dimension projection unit to adaptively align diverse domain features into a shared semantic space with minimal information loss. Furthermore, we design a variational semantic inference module to obtain the shared latent distribution. The distribution is then adopted to guide the domain projection, aligning it with shared semantics across domains and ensuring cross-domain semantic learning. LEDA exhibits strong performance across a broad range of graphs and downstream tasks. Remarkably, in few-shot cross-domain settings, it significantly outperforms in-domain baselines and advanced universal pre-training models.

Figures & tables

Figure 1. Overview of the proposed LEDA. Given the train dataset

\mathcal{T}_{\mathcal{G}}=\{\mathcal{G}^{1},\mathcal{G}^{2},\dots,\mathcal{G}^{t}\}

, we first get their initial projection basis set

\mathcal{P}_{\mathcal{G}}=\{\textbf{V}^{1},\textbf{V}^{2},\dots,\textbf{V}^{t}\}

by SVD. Subsequently, this set of projection basis is processed by a learnable multi-layer perceptron, which is optimized by

\mathcal{L}_{\text{align}}=\mathcal{L}_{\text{recon}}+\lambda\cdot\mathcal{L}_{\text{ortho}}

. Furthermore, we encode the unified feature processed by DPU through a single-layer GCN and align the posterior distribution with a shared latent distribution. Finally, we jointly optimize the parameters of DPU and LDA using the loss function

\mathcal{L}_{\text{total}}

Method	CD	Cora	CiteSeer	PubMed	Photo	Computers	CS	Physics
Raw features*	$\times$	57.90 $\pm$ 3.90	57.60 $\pm$ 2.85	79.55 $\pm$ 0.98	80.99 $\pm$ 1.65	75.59 $\pm$ 1.69	89.92 $\pm$ 0.95	93.18 $\pm$ 0.45
SVD*	$\times$	56.36 $\pm$ 4.14	48.29 $\pm$ 3.18	82.80 $\pm$ 0.91	74.92 $\pm$ 1.82	71.26 $\pm$ 1.34	87.12 $\pm$ 0.73	92.48 $\pm$ 0.47
DeepWalk*	$\times$	75.70 $\pm$ 0.00	50.50 $\pm$ 0.00	80.50 $\pm$ 0.00	89.44 $\pm$ 0.00	85.68 $\pm$ 0.00	84.61 $\pm$ 0.00	91.77 $\pm$ 0.00
GRACE*	$\times$	83.20 $\pm$ 1.87	70.99 $\pm$ 2.29	85.46 $\pm$ 0.54	91.93 $\pm$ 0.83	85.36 $\pm$ 0.82	91.84 $\pm$ 0.37	OOM
GCA*	$\times$	82.83 $\pm$ 2.29	72.06 $\pm$ 1.91	85.69 $\pm$ 0.68	92.63 $\pm$ 1.12	87.78 $\pm$ 0.78	92.69 $\pm$ 0.49	OOM
DGI*	$\times$	83.24 $\pm$ 2.12	71.23 $\pm$ 2.37	84.62 $\pm$ 0.83	92.32 $\pm$ 0.49	86.12 $\pm$ 0.73	92.47 $\pm$ 0.60	94.47 $\pm$ 0.50

Table 1. Accuracy (%) of cross-domain node classification with standard deviations. Each column represents a test domain, while others are train domains. CD means whether the model is trained on multi-domain datasets and tested on a cross-domain dataset. Aside from GCN, the best results are bolded and the second-best are underlined. Methods with * are reported from ( Zhao et al., 2024b ) .

Method \ Test data	Cora	CiteSeer	PubMed	Photo	Computers	CS
DGI	24.19 $\pm$ 5.43	27.66 $\pm$ 4.93	45.77 $\pm$ 7.16	44.92 $\pm$ 7.71	27.92 $\pm$ 5.35	67.00 $\pm$ 6.72
BGRL	35.29 $\pm$ 6.54	34.99 $\pm$ 6.92	43.56 $\pm$ 7.54	48.25 $\pm$ 6.30	35.84 $\pm$ 8.74	62.55 $\pm$ 4.96
GraphMAE	35.99 $\pm$ 7.81	38.35 $\pm$ 8.36	49.23 $\pm$ 8.36	55.99 $\pm$ 9.62	45.17 $\pm$ 10.91	66.64 $\pm$ 7.03
GPPT*	15.37 $\pm$ 4.51	23.24 $\pm$ 2.94	36.56 $\pm$ 5.31	16.19 $\pm$ 4.73	19.22 $\pm$ 8.71	-
GraphPrompt*	35.90 $\pm$ 7.10	32.76 $\pm$ 7.66	43.34 $\pm$ 10.66	49.88 $\pm$ 8.31	43.03 $\pm$ 10.35	-
GPF*	37.84 $\pm$ 11.07	37.61 $\pm$ 8.87	46.36 $\pm$ 7.48	49.42 $\pm$ 7.04	37.00 $\pm$ 6.52	-

Table 2. Performance on cross-domain 1-shot node classification. Methods with * are reported from ( Yu et al., 2024 ) . Each column represents a test domain, while others are train domains. The best results are bolded and the second-best are underlined.

Panel of Figure 2 — Figure 2. Performance on cross-domain few-shot node classification. The red line denotes our method. For traditional in-domain methods, we simply unify the input data dimensions by SVD.

Method	IMDB-B		PROTEINS		ENZYMES		COLLAB
Method	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
GPPT †*	50.15 $\pm$ 0.75	44.16 $\pm$ 6.70	60.92 $\pm$ 2.47	47.07 $\pm$ 11.95	21.29 $\pm$ 3.79	19.87 $\pm$ 2.99	47.18 $\pm$ 5.93	42.87 $\pm$ 7.70
All-in-one †*	60.07 $\pm$ 4.81	56.88 $\pm$ 0.80	66.49 $\pm$ 6.26	64.68 $\pm$ 5.35	23.96 $\pm$ 1.45	19.66 $\pm$ 3.11	51.66 $\pm$ 0.26	47.78 $\pm$ 0.10
GraphPrompt †*	54.75 $\pm$ 12.43	52.10 $\pm$ 13.61	59.17 $\pm$ 11.26	58.30 $\pm$ 10.88	22.29 $\pm$ 3.50	19.52 $\pm$ 3.36	48.25 $\pm$ 13.64	43.35 $\pm$ 10.75
GPF †*	59.65 $\pm$ 5.06	56.22 $\pm$ 6.17	63.91 $\pm$ 3.26	57.01 $\pm$ 5.79	22.00 $\pm$ 1.25	17.34 $\pm$ 2.45	47.42 $\pm$ 11.22	38.14 $\pm$ 0.44
GPF-Plus †*	57.93 $\pm$ 1.62	55.55 $\pm$ 2.03	62.92 $\pm$ 2.78	57.58 $\pm$ 7.28	22.92 $\pm$ 1.64	18.39 $\pm$ 2.76	47.24 $\pm$ 0.29	41.24 $\pm$ 0.31

Table 3. Performance on zero-shot graph classification. Methods with * are reported from ( Wang and Luo, 2024 ) . The best results are bolded and the second-best are underline.

Method	Cora	CiteSeer	PubMed
w/o DPU	30.87 $\pm$ 7.47	36.55 $\pm$ 7.83	48.50 $\pm$ 9.23
w/o LDA	34.55 $\pm$ 7.24	41.07 $\pm$ 7.60	52.87 $\pm$ 8.26
DPU+CL	34.08 $\pm$ 7.12	40.85 $\pm$ 7.59	52.44 $\pm$ 8.30
LEDA	50.70 $\pm$ 10.67	43.83 $\pm$ 10.04	54.34 $\pm$ 11.08

Table 4. Ablation studies on three cross-domain one-shot node classification datasets. CL means Contrastive Learning. The best result is bolded.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Node-level
Dtasets	Graphs	Nodes	Edges	Features	Classes	Domain
Cora	1	2,708	4,732	1,433	7	Citation networks
CiteSeer	1	3,327	5,429	3,703	6	Citation networks
PubMed	1	19,717	44,338	500	3	Citation networks
Photo	1	7,650	119,081	745	8	Co-purchase networks
Computers	1	13,752	245,861	767	10	Co-purchase networks

Appendix

Table 5. Dataset statistics of node and graph classification.

Method \ Test data	Cora	CiteSeer	PubMed	Photo	Computers	CS
DGI	33.00 $\pm$ 4.79	34.39 $\pm$ 4.73	53.12 $\pm$ 6.73	56.88 $\pm$ 5.95	37.55 $\pm$ 5.49	76.44 $\pm$ 2.54
BGRL	43.84 $\pm$ 4.99	46.71 $\pm$ 5.41	54.86 $\pm$ 6.30	58.58 $\pm$ 6.43	45.65 $\pm$ 7.96	79.69 $\pm$ 2.73
GraphMAE	47.44 $\pm$ 6.18	49.02 $\pm$ 5.77	60.10 $\pm$ 6.43	67.96 $\pm$ 7.99	55.41 $\pm$ 10.07	81.95 $\pm$ 2.58
FUG	57.92 $\pm$ 6.80	49.85 $\pm$ 5.45	61.96 $\pm$ 5.77	68.86 $\pm$ 7.39	56.20 $\pm$ 9.73	85.67 $\pm$ 1.85
LEDA	58.91 $\pm$ 7.15	57.42 $\pm$ 6.45	67.41 $\pm$ 7.91	68.39 $\pm$ 8.43	56.34 $\pm$ 12.05	86.75 $\pm$ 1.98

Appendix

Table 6. Performance on cross-domain 3-shot node classification. Each column represents a test domain, while others are train domains. The best results are bolded and the second-best are underlined.

Method \ Test data	Cora	CiteSeer	PubMed	Photo	Computers	CS
DGI	33.59 $\pm$ 3.97	45.64 $\pm$ 4.11	55.00 $\pm$ 5.15	59.04 $\pm$ 5.42	38.79 $\pm$ 5.42	79.63 $\pm$ 1.71
BGRL	45.84 $\pm$ 4.27	51.64 $\pm$ 3.85	58.04 $\pm$ 5.77	58.08 $\pm$ 5.11	48.71 $\pm$ 7.17	83.99 $\pm$ 2.24
GraphMAE	52.73 $\pm$ 4.66	52.22 $\pm$ 5.58	63.62 $\pm$ 5.28	68.48 $\pm$ 8.26	57.54 $\pm$ 10.21	84.10 $\pm$ 1.71
FUG	64.95 $\pm$ 4.70	55.64 $\pm$ 4.16	64.51 $\pm$ 4.84	70.27 $\pm$ 6.29	58.32 $\pm$ 8.64	88.04 $\pm$ 1.23
LEDA	66.07 $\pm$ 5.72	62.54 $\pm$ 3.26	70.62 $\pm$ 6.07	71.08 $\pm$ 7.51	58.79 $\pm$ 11.8	88.47 $\pm$ 1.41

Appendix

Table 7. Performance on cross-domain 5-shot node classification. Each column represents a test domain, while others are train domains. The best results are bolded and the second-best are underlined.

Cross-domain 1-shot node classification
Method \ Test data	Cora	CiteSeer	PubMed	Photo	Computers	CS
DGI	27.93 $\pm$ 4.83	26.33 $\pm$ 4.08	40.62 $\pm$ 5.39	34.50 $\pm$ 6.82	24.12 $\pm$ 5.00	33.88 $\pm$ 6.14
BGRL	35.97 $\pm$ 5.43	26.03 $\pm$ 3.65	42.07 $\pm$ 4.39	38.49 $\pm$ 6.07	33.49 $\pm$ 7.88	50.87 $\pm$ 4.72
GraphMAE	37.67 $\pm$ 7.79	36.43 $\pm$ 8.45	48.89 $\pm$ 9.20	55.80 $\pm$ 9.48	43.22 $\pm$ 11.55	66.45 $\pm$ 7.58
FUG	40.88 $\pm$ 8.65	37.31 $\pm$ 7.50	50.38 $\pm$ 9.60	60.46 $\pm$ 10.38	45.72 $\pm$ 11.72	72.57 $\pm$ 8.60
LEDA	50.51 $\pm$ 10.94	44.83 $\pm$ 10.05	54.18 $\pm$ 11.03	63.95 $\pm$ 9.64	50.01 $\pm$ 11.93	76.25 $\pm$ 6.94

Appendix

Table 8. Performance on cross-domain few-shot node classification. The best results are bolded and the second-best are underlined.

		3-shot			5-shot
		Cora	CiteSeer	PubMed	Cora	CiteSeer	PubMed
Cross-domain	w/o DPU	44.07 $\pm$ 6.39	50.91 $\pm$ 5.29	60.79 $\pm$ 7.29	50.56 $\pm$ 5.27	56.56 $\pm$ 3.72	65.88 $\pm$ 5.48
	w/o LDA	45.84 $\pm$ 6.01	52.46 $\pm$ 5.24	62.25 $\pm$ 6.41	51.07 $\pm$ 4.48	57.32 $\pm$ 3.67	65.70 $\pm$ 4.85
	DPU+CL	44.98 $\pm$ 6.01	52.15 $\pm$ 5.27	61.52 $\pm$ 6.43	50.01 $\pm$ 4.58	57.02 $\pm$ 3.71	65.08 $\pm$ 4.92
	LEDA	58.91 $\pm$ 7.15	57.42 $\pm$ 6.45	67.41 $\pm$ 7.91	66.07 $\pm$ 5.72	62.54 $\pm$ 3.26	70.62 $\pm$ 6.07

Appendix

Table 9. Supplementary ablation studies. The best results are bolded.

Explore similar work

MDGMIX: Boundary-Aware Subgraph Mixing for Multi-Domain Graph Pre-Training
May 25, 2026Ziyu Zheng, Yaming Yang, Ziyu Guan +2Cross-Domain Generalization Cross-Domain Transfer Learning
School of Computer Science and Technology, Xidian University, Xi’an, China · School of Artificial Intelligence, Xidian University, Xi’an, China
Handling Feature Heterogeneity with Learnable Graph Patches
Jun 16, 2026Yifei Sun, Yang Yang, Xiao Feng +4Graph Representation Learning Cross-Domain Transfer Learning
Zhejiang University Hangzhou, China · Huazhong University of Science and Technology Wuhan, China · Finvolution Group Shanghai, China
Modality-free Graph In-context Alignment
Mar 13, 2026Wei Zhuo, Siqiang LuoGraph Foundation Models In-Context Learning
Nanyang Technological University

May 25, 2026J/K move · Enter open · S save

MDGMIX: Boundary-Aware Subgraph Mixing for Multi-Domain Graph Pre-Training

Ziyu Zheng, Yaming Yang, Ziyu Guan, Wei Zhao+1

School of Computer Science and Technology, Xidian University, Xi’an, China · School of Artificial Intelligence, Xidian University, Xi’an, China

Cross-Domain Generalization Cross-Domain Transfer Learning Graph Foundation Models

Open paper AskPDF arXiv

Multi-domain graph pre-training is a crucial step in constructing foundational graph models with cross-domain generalization capabilities. However, existing methods predominantly rely on jointly training all source domain graphs, resulting in high computational costs. Furthermore, it remains unclear whether all source domain graph data contribute equally to effective transfer. This paper empirically reveals significant data redundancy in multi-domain graph pre-training. Based on this finding, we propose the Multi-domain Graph Pre-training Framework, MDGMIX, which combines boundary-aware subgraph mixing with hierarchical discrimination. By selecting boundary nodes to construct challenging mixed-domain subgraphs, MDGMIX employs coarse-grained domain discrimination and fine-grained domain decomposition losses to decouple shared patterns from domain-specific patterns. During adaptation, MDGMIX employs a lightweight prompt weighting mechanism to transfer source domain knowledge. Extensive experiments demonstrate that MDGMIX consistently outperforms strong baselines in few-shot classification tasks while exhibiting superior time and memory efficiency. The code is available at: https://github.com/zhengziyu77/MDGMIX.

Abstract

Explore similar work

MDGMIX: Boundary-Aware Subgraph Mixing for Multi-Domain Graph Pre-Training

Handling Feature Heterogeneity with Learnable Graph Patches

Modality-free Graph In-context Alignment