LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training

Lianze Shan; Jitao Zhao; Dongxiao He; Siqi Liu; Jiaxu Cui; Weixiong Zhang

cs.LGFeb 26, 2026

LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training

Authors: Lianze Shan, Jitao Zhao, Dongxiao He, Siqi Liu, Jiaxu Cui, Weixiong Zhang

Organizations: Tianjin University, Tianjin, China · Jilin University, Changchun, China · The Hong Kong Polytechnic University, Kowloon, Hong Kong

Abstract

Recent advances in generic large models, such as GPT and DeepSeek, have motivated the introduction of universality to graph pre-training, aiming to learn rich and generalizable knowledge across diverse domains using graph representations to improve performance in various downstream applications. However, most existing methods face challenges in learning effective knowledge from generic graphs, primarily due to simplistic data alignment and limited training guidance. The issue of simplistic data alignment arises from the use of a straightforward unification for highly diverse graph data, which fails to align semantics and misleads pre-training models. The problem with limited training guidance lies in the arbitrary application of in-domain pre-training paradigms to cross-domain scenarios. While it is effective in enhancing discriminative representation in one data space, it struggles to capture effective knowledge from many graphs. To address these challenges, we propose a novel Latent sEmantic Distribution Alignment (LEDA) model for universal graph pre-training. Specifically, we first introduce a dimension projection unit to adaptively align diverse domain features into a shared semantic space with minimal information loss. Furthermore, we design a variational semantic inference module to obtain the shared latent distribution. The distribution is then adopted to guide the domain projection, aligning it with shared semantics across domains and ensuring cross-domain semantic learning. LEDA exhibits strong performance across a broad range of graphs and downstream tasks. Remarkably, in few-shot cross-domain settings, it significantly outperforms in-domain baselines and advanced universal pre-training models.

Figures & tables

Figure 1. Overview of the proposed LEDA. Given the train dataset

\mathcal{T}_{\mathcal{G}}=\{\mathcal{G}^{1},\mathcal{G}^{2},\dots,\mathcal{G}^{t}\}

, we first get their initial projection basis set

\mathcal{P}_{\mathcal{G}}=\{\textbf{V}^{1},\textbf{V}^{2},\dots,\textbf{V}^{t}\}

by SVD. Subsequently, this set of projection basis is processed by a learnable multi-layer perceptron, which is optimized by

\mathcal{L}_{\text{align}}=\mathcal{L}_{\text{recon}}+\lambda\cdot\mathcal{L}_{\text{ortho}}

. Furthermore, we encode the unified feature processed by DPU through a single-layer GCN and align the posterior distribution with a shared latent distribution. Finally, we jointly optimize the parameters of DPU and LDA using the loss function

\mathcal{L}_{\text{total}}

Method	CD	Cora	CiteSeer	PubMed	Photo	Computers	CS	Physics
Raw features*	$\times$	57.90 $\pm$ 3.90	57.60 $\pm$ 2.85	79.55 $\pm$ 0.98	80.99 $\pm$ 1.65	75.59 $\pm$ 1.69	89.92 $\pm$ 0.95	93.18 $\pm$ 0.45
SVD*	$\times$	56.36 $\pm$ 4.14	48.29 $\pm$ 3.18	82.80 $\pm$ 0.91	74.92 $\pm$ 1.82	71.26 $\pm$ 1.34	87.12 $\pm$ 0.73	92.48 $\pm$ 0.47
DeepWalk*	$\times$	75.70 $\pm$ 0.00	50.50 $\pm$ 0.00	80.50 $\pm$ 0.00	89.44 $\pm$ 0.00	85.68 $\pm$ 0.00	84.61 $\pm$ 0.00	91.77 $\pm$ 0.00
GRACE*	$\times$	83.20 $\pm$ 1.87	70.99 $\pm$ 2.29	85.46 $\pm$ 0.54	91.93 $\pm$ 0.83	85.36 $\pm$ 0.82	91.84 $\pm$ 0.37	OOM
GCA*	$\times$	82.83 $\pm$ 2.29	72.06 $\pm$ 1.91	85.69 $\pm$ 0.68	92.63 $\pm$ 1.12	87.78 $\pm$ 0.78	92.69 $\pm$ 0.49	OOM
DGI*	$\times$	83.24 $\pm$ 2.12	71.23 $\pm$ 2.37	84.62 $\pm$ 0.83	92.32 $\pm$ 0.49	86.12 $\pm$ 0.73	92.47 $\pm$ 0.60	94.47 $\pm$ 0.50

Table 1. Accuracy (%) of cross-domain node classification with standard deviations. Each column represents a test domain, while others are train domains. CD means whether the model is trained on multi-domain datasets and tested on a cross-domain dataset. Aside from GCN, the best results are bolded and the second-best are underlined. Methods with * are reported from ( Zhao et al., 2024b ) .

Method \ Test data	Cora	CiteSeer	PubMed	Photo	Computers	CS
DGI	24.19 $\pm$ 5.43	27.66 $\pm$ 4.93	45.77 $\pm$ 7.16	44.92 $\pm$ 7.71	27.92 $\pm$ 5.35	67.00 $\pm$ 6.72
BGRL	35.29 $\pm$ 6.54	34.99 $\pm$ 6.92	43.56 $\pm$ 7.54	48.25 $\pm$ 6.30	35.84 $\pm$ 8.74	62.55 $\pm$ 4.96
GraphMAE	35.99 $\pm$ 7.81	38.35 $\pm$ 8.36	49.23 $\pm$ 8.36	55.99 $\pm$ 9.62	45.17 $\pm$ 10.91	66.64 $\pm$ 7.03
GPPT*	15.37 $\pm$ 4.51	23.24 $\pm$ 2.94	36.56 $\pm$ 5.31	16.19 $\pm$ 4.73	19.22 $\pm$ 8.71	-
GraphPrompt*	35.90 $\pm$ 7.10	32.76 $\pm$ 7.66	43.34 $\pm$ 10.66	49.88 $\pm$ 8.31	43.03 $\pm$ 10.35	-
GPF*	37.84 $\pm$ 11.07	37.61 $\pm$ 8.87	46.36 $\pm$ 7.48	49.42 $\pm$ 7.04	37.00 $\pm$ 6.52	-

Table 2. Performance on cross-domain 1-shot node classification. Methods with * are reported from ( Yu et al., 2024 ) . Each column represents a test domain, while others are train domains. The best results are bolded and the second-best are underlined.

Panel of Figure 2 — Figure 2. Performance on cross-domain few-shot node classification. The red line denotes our method. For traditional in-domain methods, we simply unify the input data dimensions by SVD.

Method	IMDB-B		PROTEINS		ENZYMES		COLLAB
Method	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
GPPT †*	50.15 $\pm$ 0.75	44.16 $\pm$ 6.70	60.92 $\pm$ 2.47	47.07 $\pm$ 11.95	21.29 $\pm$ 3.79	19.87 $\pm$ 2.99	47.18 $\pm$ 5.93	42.87 $\pm$ 7.70
All-in-one †*	60.07 $\pm$ 4.81	56.88 $\pm$ 0.80	66.49 $\pm$ 6.26	64.68 $\pm$ 5.35	23.96 $\pm$ 1.45	19.66 $\pm$ 3.11	51.66 $\pm$ 0.26	47.78 $\pm$ 0.10
GraphPrompt †*	54.75 $\pm$ 12.43	52.10 $\pm$ 13.61	59.17 $\pm$ 11.26	58.30 $\pm$ 10.88	22.29 $\pm$ 3.50	19.52 $\pm$ 3.36	48.25 $\pm$ 13.64	43.35 $\pm$ 10.75
GPF †*	59.65 $\pm$ 5.06	56.22 $\pm$ 6.17	63.91 $\pm$ 3.26	57.01 $\pm$ 5.79	22.00 $\pm$ 1.25	17.34 $\pm$ 2.45	47.42 $\pm$ 11.22	38.14 $\pm$ 0.44
GPF-Plus †*	57.93 $\pm$ 1.62	55.55 $\pm$ 2.03	62.92 $\pm$ 2.78	57.58 $\pm$ 7.28	22.92 $\pm$ 1.64	18.39 $\pm$ 2.76	47.24 $\pm$ 0.29	41.24 $\pm$ 0.31

Table 3. Performance on zero-shot graph classification. Methods with * are reported from ( Wang and Luo, 2024 ) . The best results are bolded and the second-best are underline.

Method	Cora	CiteSeer	PubMed
w/o DPU	30.87 $\pm$ 7.47	36.55 $\pm$ 7.83	48.50 $\pm$ 9.23
w/o LDA	34.55 $\pm$ 7.24	41.07 $\pm$ 7.60	52.87 $\pm$ 8.26
DPU+CL	34.08 $\pm$ 7.12	40.85 $\pm$ 7.59	52.44 $\pm$ 8.30
LEDA	50.70 $\pm$ 10.67	43.83 $\pm$ 10.04	54.34 $\pm$ 11.08

Table 4. Ablation studies on three cross-domain one-shot node classification datasets. CL means Contrastive Learning. The best result is bolded.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Node-level
Dtasets	Graphs	Nodes	Edges	Features	Classes	Domain
Cora	1	2,708	4,732	1,433	7	Citation networks
CiteSeer	1	3,327	5,429	3,703	6	Citation networks
PubMed	1	19,717	44,338	500	3	Citation networks
Photo	1	7,650	119,081	745	8	Co-purchase networks
Computers	1	13,752	245,861	767	10	Co-purchase networks

Appendix

Table 5. Dataset statistics of node and graph classification.

Method \ Test data	Cora	CiteSeer	PubMed	Photo	Computers	CS
DGI	33.00 $\pm$ 4.79	34.39 $\pm$ 4.73	53.12 $\pm$ 6.73	56.88 $\pm$ 5.95	37.55 $\pm$ 5.49	76.44 $\pm$ 2.54
BGRL	43.84 $\pm$ 4.99	46.71 $\pm$ 5.41	54.86 $\pm$ 6.30	58.58 $\pm$ 6.43	45.65 $\pm$ 7.96	79.69 $\pm$ 2.73
GraphMAE	47.44 $\pm$ 6.18	49.02 $\pm$ 5.77	60.10 $\pm$ 6.43	67.96 $\pm$ 7.99	55.41 $\pm$ 10.07	81.95 $\pm$ 2.58
FUG	57.92 $\pm$ 6.80	49.85 $\pm$ 5.45	61.96 $\pm$ 5.77	68.86 $\pm$ 7.39	56.20 $\pm$ 9.73	85.67 $\pm$ 1.85
LEDA	58.91 $\pm$ 7.15	57.42 $\pm$ 6.45	67.41 $\pm$ 7.91	68.39 $\pm$ 8.43	56.34 $\pm$ 12.05	86.75 $\pm$ 1.98

Appendix

Table 6. Performance on cross-domain 3-shot node classification. Each column represents a test domain, while others are train domains. The best results are bolded and the second-best are underlined.

Method \ Test data	Cora	CiteSeer	PubMed	Photo	Computers	CS
DGI	33.59 $\pm$ 3.97	45.64 $\pm$ 4.11	55.00 $\pm$ 5.15	59.04 $\pm$ 5.42	38.79 $\pm$ 5.42	79.63 $\pm$ 1.71
BGRL	45.84 $\pm$ 4.27	51.64 $\pm$ 3.85	58.04 $\pm$ 5.77	58.08 $\pm$ 5.11	48.71 $\pm$ 7.17	83.99 $\pm$ 2.24
GraphMAE	52.73 $\pm$ 4.66	52.22 $\pm$ 5.58	63.62 $\pm$ 5.28	68.48 $\pm$ 8.26	57.54 $\pm$ 10.21	84.10 $\pm$ 1.71
FUG	64.95 $\pm$ 4.70	55.64 $\pm$ 4.16	64.51 $\pm$ 4.84	70.27 $\pm$ 6.29	58.32 $\pm$ 8.64	88.04 $\pm$ 1.23
LEDA	66.07 $\pm$ 5.72	62.54 $\pm$ 3.26	70.62 $\pm$ 6.07	71.08 $\pm$ 7.51	58.79 $\pm$ 11.8	88.47 $\pm$ 1.41

Appendix

Table 7. Performance on cross-domain 5-shot node classification. Each column represents a test domain, while others are train domains. The best results are bolded and the second-best are underlined.

Cross-domain 1-shot node classification
Method \ Test data	Cora	CiteSeer	PubMed	Photo	Computers	CS
DGI	27.93 $\pm$ 4.83	26.33 $\pm$ 4.08	40.62 $\pm$ 5.39	34.50 $\pm$ 6.82	24.12 $\pm$ 5.00	33.88 $\pm$ 6.14
BGRL	35.97 $\pm$ 5.43	26.03 $\pm$ 3.65	42.07 $\pm$ 4.39	38.49 $\pm$ 6.07	33.49 $\pm$ 7.88	50.87 $\pm$ 4.72
GraphMAE	37.67 $\pm$ 7.79	36.43 $\pm$ 8.45	48.89 $\pm$ 9.20	55.80 $\pm$ 9.48	43.22 $\pm$ 11.55	66.45 $\pm$ 7.58
FUG	40.88 $\pm$ 8.65	37.31 $\pm$ 7.50	50.38 $\pm$ 9.60	60.46 $\pm$ 10.38	45.72 $\pm$ 11.72	72.57 $\pm$ 8.60
LEDA	50.51 $\pm$ 10.94	44.83 $\pm$ 10.05	54.18 $\pm$ 11.03	63.95 $\pm$ 9.64	50.01 $\pm$ 11.93	76.25 $\pm$ 6.94

Appendix

Table 8. Performance on cross-domain few-shot node classification. The best results are bolded and the second-best are underlined.

		3-shot			5-shot
		Cora	CiteSeer	PubMed	Cora	CiteSeer	PubMed
Cross-domain	w/o DPU	44.07 $\pm$ 6.39	50.91 $\pm$ 5.29	60.79 $\pm$ 7.29	50.56 $\pm$ 5.27	56.56 $\pm$ 3.72	65.88 $\pm$ 5.48
	w/o LDA	45.84 $\pm$ 6.01	52.46 $\pm$ 5.24	62.25 $\pm$ 6.41	51.07 $\pm$ 4.48	57.32 $\pm$ 3.67	65.70 $\pm$ 4.85
	DPU+CL	44.98 $\pm$ 6.01	52.15 $\pm$ 5.27	61.52 $\pm$ 6.43	50.01 $\pm$ 4.58	57.02 $\pm$ 3.71	65.08 $\pm$ 4.92
	LEDA	58.91 $\pm$ 7.15	57.42 $\pm$ 6.45	67.41 $\pm$ 7.91	66.07 $\pm$ 5.72	62.54 $\pm$ 3.26	70.62 $\pm$ 6.07

Appendix

Table 9. Supplementary ablation studies. The best results are bolded.

Explore similar work

May 25, 2026

MDGMIX: Boundary-Aware Subgraph Mixing for Multi-Domain Graph Pre-Training

Multi-domain graph pre-training is a crucial step in constructing foundational graph models with cross-domain generalization capabilities. However, existing methods predominantly rely on jointly training all source domain graphs, resulting in high computational costs. Furthermore, it remains unclear whether all source domain graph data contribute equally to effective transfer. This paper empirically reveals significant data redundancy in multi-domain graph pre-training. Based on this finding, we propose the Multi-domain Graph Pre-training Framework, MDGMIX, which combines boundary-aware subgraph mixing with hierarchical discrimination. By selecting boundary nodes to construct challenging mixed-domain subgraphs, MDGMIX employs coarse-grained domain discrimination and fine-grained domain decomposition losses to decouple shared patterns from domain-specific patterns. During adaptation, MDGMIX employs a lightweight prompt weighting mechanism to transfer source domain knowledge. Extensive experiments demonstrate that MDGMIX consistently outperforms strong baselines in few-shot classification tasks while exhibiting superior time and memory efficiency. The code is available at: https://github.com/zhengziyu77/MDGMIX.

Ziyu Zheng, Yaming Yang, Ziyu Guan +2

School of Computer Science and Technology, Xidian University, Xi’an, China · School of Artificial Intelligence, Xidian University, Xi’an, China

Cross-Domain Generalization Cross-Domain Transfer Learning Graph Foundation Models

Ask PDF

Jun 16, 2026

Handling Feature Heterogeneity with Learnable Graph Patches

In recent years, the rapid development of foundation models and graph pre-training technologies has spurred increasing interest in constructing a universal pre-trained graph model or Graph Foundation Model (GFM). However, a significant challenge is that existing models are unable to address feature heterogeneity in graph data without textual information, which hinders the transferability of graph models across different datasets. To bridge this gap, we propose the concept of learnable graph patches, which we regard as the smallest semantic units of any graph data. We decompose the graph into learnable graph patches by unfolding the node features and constructing corresponding patch structures separately. We then design a framework that mines transferable information from graph data across domains. Specifically, after extracting graph patches, we propose a patch encoder to extract knowledge from each unit and a patch aggregator to learn how the units are combined into a whole. Due to its domain-agnostic nature, the model can be applied to downstream data across different domains. Furthermore, we analyze the connection between our method and existing graph models, as well as the transferability of the node embeddings it generates. Empirically, our method not only achieves the capability to use multi-domain graphs for pre-training, but also shows enhanced performance across various downstream datasets and tasks. Moreover, we observe consistent improvement in downstream performance as the volume of pre-training data increases.

Yifei Sun, Yang Yang, Xiao Feng +4

Zhejiang University Hangzhou, China · Huazhong University of Science and Technology Wuhan, China · Finvolution Group Shanghai, China

Graph Representation Learning Cross-Domain Transfer Learning Graph Foundation Models

Ask PDF

Mar 13, 2026

Modality-free Graph In-context Alignment

In-context learning (ICL) converts static encoders into task-conditioned reasoners, enabling adaptation to new data from just a few examples without updating pretrained parameters. This capability is essential for graph foundation models (GFMs) to approach LLM-level generality. Yet current GFMs struggle with cross-domain alignment, typically relying on modality-specific encoders that fail when graphs are pre-vectorized or raw data is inaccessible. In this paper, we introduce Modality-Free Graph In-context Alignment (MF-GIA), a framework that makes a pretrained graph encoder promptable for few-shot prediction across heterogeneous domains without modality assumptions. MF-GIA captures domain characteristics through gradient fingerprints, which parameterize lightweight transformations that align pre-encoded features and indexed labels into unified semantic spaces. During pretraining, a dual prompt-aware attention mechanism with episodic objective learns to match queries against aligned support examples to establish prompt-based reasoning capabilities. At inference, MF-GIA performs parameter-update-free adaptation using only a few-shot support set to trigger cross-domain alignment and enable immediate prediction on unseen domains. Experiments demonstrate that MF-GIA achieves superior few-shot performance across diverse graph domains and strong generalization to unseen domains.

Wei Zhuo, Siqiang Luo

Nanyang Technological University

Graph Foundation Models In-Context Learning Graph Neural Networks Few-Shot Domain Adaptation

Ask PDF