MUG: Meta-path-aware Universal Heterogeneous Graph Pre-Training

Lianze Shan; Jitao Zhao; Dongxiao He; Yongqi Huang; Zhiyong Feng; Weixiong Zhang

cs.LGFeb 26, 2026

MUG: Meta-path-aware Universal Heterogeneous Graph Pre-Training

Authors: Lianze Shan, Jitao Zhao, Dongxiao He, Yongqi Huang, Zhiyong Feng, Weixiong Zhang

Organizations: College of Intelligence and Computing, Tianjin University, Tianjin, China · Departments of Health Technology & Informatics; Data Science & Artificial Intelligence; and Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong

Abstract

Universal graph pre-training has emerged as a key paradigm in graph representation learning, offering a promising way to train encoders to learn transferable representations from unlabeled graphs and to effectively generalize across a wide range of downstream tasks. However, recent explorations in universal graph pre-training primarily focus on homogeneous graphs and it remains unexplored for heterogeneous graphs, which exhibit greater structural and semantic complexity. This heterogeneity makes it highly challenging to train a universal encoder for diverse heterogeneous graphs: (i) the diverse types with dataset-specific semantics hinder the construction of a unified representation space; (ii) the number and semantics of meta-paths vary across datasets, making encoding and aggregation patterns learned from one dataset difficult to apply to others. To address these challenges, we propose a novel Meta-path-aware Universal heterogeneous Graph pre-training (MUG) approach. Specifically, for challenge (i), MUG introduces a input unification module that integrates information from multiple node and relation types within each heterogeneous graph into a unified representation.This representation is then projected into a shared space by a dimension-aware encoder, enabling alignment across graphs with diverse schemas.Furthermore, for challenge (ii), MUG trains a shared encoder to capture consistent structural patterns across diverse meta-path views rather than relying on dataset-specific aggregation strategies, while a global objective encourages discriminability and reduces dataset-specific biases. Extensive experiments demonstrate the effectiveness of MUG on some real datasets.

Figures & tables

Figure 1: Homophily ratios on six datasets. Cora and CiteSeer ( Yang et al. 2016 ) are homogeneous graphs, while ACM ( Zhao et al. 2020 ) , DBLP ( Fu et al. 2020 ) , AMiner ( Hu et al. 2019 ) and Freebase ( Li et al. 2021a ) are heterogeneous. For heterogeneous graphs, we compute the homophily ratio for each meta-path-based adjacency matrix and report the average.

Figure 2: Overview of the MUG. Given a heterogeneous graph with diverse node types, relation types, and attributes, we first get embedding

\mathbf{Z}^{\text{struct}}

by contextual structural encoding. The embedding is concatenated with the original node attributes

\mathbf{X}

to obtain unified representation

\tilde{\mathbf{X}}

, which contains diverse type and attribute information. Then, a dimension-aware encoder is applied to align representation spaces and produce the unified input

\mathbf{X}^{\text{unify}}

. Finally, a shared GNN encoder is used to encode each masked adjacency matrix

\tilde{\mathbf{A}}^{\phi}

with the unified input

\mathbf{X}^{\text{unify}}

, producing

\mathbf{Z}^{\phi}

. The model is optimized by three objectives: a dimension alignment loss

\mathcal{L}_{\text{align}}

, a meta-path masked reconstruction loss

\mathcal{L}^{\phi}

and a global scattering regularization loss

\mathcal{L}_{\text{scatter}}

Train	Method	ACM		DBLP		AMiner		Freebase
Train	Method	Ma-F1	Mi-F1	Ma-F1	Mi-F1	Ma-F1	Mi-F1	Ma-F1	Mi-F1
ACM	HeCo	80.22 $\pm$ 2.45	79.71 $\pm$ 3.23	76.76 $\pm$ 0.44	77.97 $\pm$ 0.41	24.48 $\pm$ 1.31	51.18 $\pm$ 6.21	31.22 $\pm$ 1.33	40.67 $\pm$ 0.76
	HGMAE	84.22 $\pm$ 0.52	84.01 $\pm$ 0.50	87.17 $\pm$ 0.19	88.23 $\pm$ 0.23	29.08 $\pm$ 0.83	41.91 $\pm$ 7.06	32.59 $\pm$ 1.12	42.95 $\pm$ 1.31
	HERO	84.37 $\pm$ 0.29	84.12 $\pm$ 0.30	84.60 $\pm$ 0.61	85.80 $\pm$ 0.54	44.08 $\pm$ 1.38	50.14 $\pm$ 1.31	33.69 $\pm$ 1.77	43.32 $\pm$ 0.43
	MUG	85.52 $\pm$ 0.79	84.90 $\pm$ 1.13	91.69 $\pm$ 0.13	92.38 $\pm$ 0.30	76.35 $\pm$ 0.04	87.02 $\pm$ 0.15	46.05 $\pm$ 0.52	49.78 $\pm$ 1.29
DBLP	HeCo	82.93 $\pm$ 0.80	83.43 $\pm$ 0.70	90.11 $\pm$ 0.33	90.73 $\pm$ 0.31	26.89 $\pm$ 1.09	35.23 $\pm$ 5.45	35.58 $\pm$ 0.94	39.53 $\pm$ 2.11

Table 1: Performance on cross-domain node classification with standard deviations. The best results are highlighted in bold.

Shot-num	Method	ACM		DBLP		Aminer		Freebase
Shot-num	Method	Ma-F1	Mi-F1	Ma-F1	Mi-F1	Ma-F1	Mi-F1	Ma-F1	Mi-F1
1-shot	HeCo	59.36 $\pm$ 12.32	63.41 $\pm$ 9.52	45.74 $\pm$ 10.32	49.94 $\pm$ 9.53	23.85 $\pm$ 2.61	39.69 $\pm$ 10.43	30.70 $\pm$ 3.08	39.58 $\pm$ 3.68
	HGMAE	73.17 $\pm$ 5.04	75.48 $\pm$ 4.96	61.46 $\pm$ 5.68	65.94 $\pm$ 5.54	20.65 $\pm$ 4.46	36.88 $\pm$ 7.33	30.65 $\pm$ 3.87	38.20 $\pm$ 4.37
	HERO	51.39 $\pm$ 1.99	57.37 $\pm$ 1.31	40.49 $\pm$ 4.12	41.87 $\pm$ 4.88	44.18 $\pm$ 0.78	50.14 $\pm$ 1.31	32.20 $\pm$ 2.21	42.56 $\pm$ 0.97
	MUG	79.49 $\pm$ 4.26	79.54 $\pm$ 3.93	84.24 $\pm$ 5.64	85.76 $\pm$ 4.95	49.12 $\pm$ 6.16	72.10 $\pm$ 4.68	33.24 $\pm$ 2.63	42.20 $\pm$ 1.96
3-shot	HeCo	73.53 $\pm$ 8.71	74.77 $\pm$ 7.29	64.49 $\pm$ 6.69	66.09 $\pm$ 6.50	24.93 $\pm$ 1.92	40.56 $\pm$ 8.13	33.15 $\pm$ 2.66	40.12 $\pm$ 2.39

Table 2: Performance on few-shot node classification with standard deviations. The best results are highlighted in bold.

Figure 3: Ablation results for cross-domain node classification, trained on Freebase and evaluated on four datasets.

Graph Representation Learning

Graph representation learning develops methods for encoding nodes, edges, and whole graphs for prediction, comparison, and transfer.

See all 305 papers

Handling Feature Heterogeneity with Learnable Graph Patches
Jun 16, 2026Yifei Sun, Yang Yang, Xiao Feng +4Graph Representation Learning Cross-Domain Transfer Learning
Zhejiang University Hangzhou, China · Huazhong University of Science and Technology Wuhan, China · Finvolution Group Shanghai, China
LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training
Feb 26, 2026Lianze Shan, Jitao Zhao, Dongxiao He +3Graph Foundation Models Cross-Domain Generalization
Tianjin University, Tianjin, China · Jilin University, Changchun, China · The Hong Kong Polytechnic University, Kowloon, Hong Kong
The Universal Classifier for Graph Learning
Sep 28, 2026Ben Finkelshtein, André Linhares, Petar Veličković +2Graph Representation Learning Graph Foundation Models
Google Research · Google DeepMind

Jun 16, 2026J/K move · Enter open · S save

Handling Feature Heterogeneity with Learnable Graph Patches

Yifei Sun, Yang Yang, Xiao Feng, Zijun Wang+3

Zhejiang University Hangzhou, China · Huazhong University of Science and Technology Wuhan, China · Finvolution Group Shanghai, China

Graph Representation Learning Cross-Domain Transfer Learning Graph Foundation Models

Open paper AskPDF arXiv

In recent years, the rapid development of foundation models and graph pre-training technologies has spurred increasing interest in constructing a universal pre-trained graph model or Graph Foundation Model (GFM). However, a significant challenge is that existing models are unable to address feature heterogeneity in graph data without textual information, which hinders the transferability of graph models across different datasets. To bridge this gap, we propose the concept of learnable graph patches, which we regard as the smallest semantic units of any graph data. We decompose the graph into learnable graph patches by unfolding the node features and constructing corresponding patch structures separately. We then design a framework that mines transferable information from graph data across domains. Specifically, after extracting graph patches, we propose a patch encoder to extract knowledge from each unit and a patch aggregator to learn how the units are combined into a whole. Due to its domain-agnostic nature, the model can be applied to downstream data across different domains. Furthermore, we analyze the connection between our method and existing graph models, as well as the transferability of the node embeddings it generates. Empirically, our method not only achieves the capability to use multi-domain graphs for pre-training, but also shows enhanced performance across various downstream datasets and tasks. Moreover, we observe consistent improvement in downstream performance as the volume of pre-training data increases.

Abstract

Graph Representation Learning

Explore similar work

Handling Feature Heterogeneity with Learnable Graph Patches

LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training

The Universal Classifier for Graph Learning