Modality-free Graph In-context Alignment

Wei Zhuo; Siqiang Luo

cs.LGMar 13, 2026

Modality-free Graph In-context Alignment

Authors: Wei Zhuo, Siqiang Luo

Organizations: Nanyang Technological University

Abstract

In-context learning (ICL) converts static encoders into task-conditioned reasoners, enabling adaptation to new data from just a few examples without updating pretrained parameters. This capability is essential for graph foundation models (GFMs) to approach LLM-level generality. Yet current GFMs struggle with cross-domain alignment, typically relying on modality-specific encoders that fail when graphs are pre-vectorized or raw data is inaccessible. In this paper, we introduce Modality-Free Graph In-context Alignment (MF-GIA), a framework that makes a pretrained graph encoder promptable for few-shot prediction across heterogeneous domains without modality assumptions. MF-GIA captures domain characteristics through gradient fingerprints, which parameterize lightweight transformations that align pre-encoded features and indexed labels into unified semantic spaces. During pretraining, a dual prompt-aware attention mechanism with episodic objective learns to match queries against aligned support examples to establish prompt-based reasoning capabilities. At inference, MF-GIA performs parameter-update-free adaptation using only a few-shot support set to trigger cross-domain alignment and enable immediate prediction on unseen domains. Experiments demonstrate that MF-GIA achieves superior few-shot performance across diverse graph domains and strong generalization to unseen domains.

Figures & tables

Figure 1: Overview of MF-GIA. (Left) Modality-free Alignment: The pretraining graphs are mapped to a unified space via domain-conditioned transformations. Domain descriptors $e$ ensure similar domains occupy neighboring subspaces. (Middle) Episodic Pretraining: The model learns from $m$ -way $k$ -shot episodes using domain-aligned features and labels. The DPAA mechanism matches queries to classes using only prompts as context. (Right) In-context Prediction: For an unseen graph, the frozen model performs few-shot classification using the support set as a prompt.

Figure 3: Domain-conditioned label alignment.

Figure 4: Backbone selections.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Pretraining curves of MF-GIA.

Configuration	Pretraining Datasets	Domain Coverage	In-Domain	Out-of-Domain			Avg OOD	Overall
Configuration	Pretraining Datasets	Domain Coverage	Cora	Products	Physics	BlogCatalog	Avg OOD	Overall
Single	PubMed	Citation	42.76	12.43	68.15	41.28	40.62	40.91
	Arxiv	Citation	48.31	13.86	76.42	43.67	44.65	45.57
	WikiCS	Web	39.23	14.72	65.83	45.91	42.15	41.67
	Amazon	E-commerce	35.47	17.28	61.26	38.74	39.09	38.19
Two	PubMed + Arxiv	Citation	52.14	14.92	79.87	47.35	47.38	48.57

Appendix

Table 7: Effect of pretraining dataset composition on few-shot node classification accuracy (%). We systematically vary domain coverage from single to full four-domain pretraining. Results are 5-shot accuracy averaged over 20 episodes. Best results are bold .

Method	Computers		Physics		BlogCatalog
Method	1-shot	5-shot	1-shot	5-shot	1-shot	5-shot
MF-GIA	41.49	53.71	79.12	88.92	49.46	67.31
MF-GIA (w. DIA)	41.96	54.60	80.15	89.74	48.37	66.53

Appendix

Table 9: Performance on MF-GIA with expressive dimension unification component.

$\tau$	ogbn-Products	Computers	FB15K237
0.05	21.45	52.10	91.82
0.2 (Default)	22.61	53.71	91.38
0.5	22.60	53.93	90.45
1	22.15	53.29	90.27

Appendix

Table 12: Effect of

\tau

on different datasets (5-shot).

	Computers	Physics	BlogCatalog
1-layer, 1-head (Shared)	53.71	88.92	67.31
1-layer, 1-head (Separated)	51.25	88.10	67.05
1-layer, 4-head	52.47	86.58	66.15
2-layer, 1-head	54.05	88.53	67.46
2-layer, 4-head	53.39	87.52	66.83

Appendix

Table 13: Sensitivity of DPAA configurations

Explore similar work

LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training
Feb 26, 2026Lianze Shan, Jitao Zhao, Dongxiao He +3Graph Foundation Models Cross-Domain Generalization
Tianjin University, Tianjin, China · Jilin University, Changchun, China · The Hong Kong Polytechnic University, Kowloon, Hong Kong
Surprisingly Simple and Effective Multi-Domain Graph Foundation Model through Graph-to-Table Alignment
Jul 13, 2026Chunyu Hu, Tianyin Liao, Ge Lan +4Tabular Foundation Models Graph Representation Learning
Nankai University · Tsinghua University · Beihang University
CrossHGL: A Text-Free Foundation Model for Cross-Domain Heterogeneous Graph Learning
Mar 29, 2026Xuanze Chen, Jiajun Zhou, Yadong Li +2Graph Representation Learning Graph Foundation Models
Xuanze Chen and Yadong Li are with the Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China, and also with the Binjiang Cyberspace Security Institute of ZJUT, Hangzhou, 310056, China · Jiajun Zhou, Shanqing Yu and Qi Xuan are with the Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China, with the Binjiang Cyberspace Security Institute of ZJUT, Hangzhou, 310056, China, and with the Soovar Technologies Co., Ltd., Hangzhou 310056, China

Feb 26, 2026J/K move · Enter open · S save

LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training

Lianze Shan, Jitao Zhao, Dongxiao He, Siqi Liu+2

Tianjin University, Tianjin, China · Jilin University, Changchun, China · The Hong Kong Polytechnic University, Kowloon, Hong Kong

Graph Foundation Models Cross-Domain Generalization Latent Space Alignment Cross-Domain Transfer Learning

Open paper AskPDF arXiv

Recent advances in generic large models, such as GPT and DeepSeek, have motivated the introduction of universality to graph pre-training, aiming to learn rich and generalizable knowledge across diverse domains using graph representations to improve performance in various downstream applications. However, most existing methods face challenges in learning effective knowledge from generic graphs, primarily due to simplistic data alignment and limited training guidance. The issue of simplistic data alignment arises from the use of a straightforward unification for highly diverse graph data, which fails to align semantics and misleads pre-training models. The problem with limited training guidance lies in the arbitrary application of in-domain pre-training paradigms to cross-domain scenarios. While it is effective in enhancing discriminative representation in one data space, it struggles to capture effective knowledge from many graphs. To address these challenges, we propose a novel Latent sEmantic Distribution Alignment (LEDA) model for universal graph pre-training. Specifically, we first introduce a dimension projection unit to adaptively align diverse domain features into a shared semantic space with minimal information loss. Furthermore, we design a variational semantic inference module to obtain the shared latent distribution. The distribution is then adopted to guide the domain projection, aligning it with shared semantics across domains and ensuring cross-domain semantic learning. LEDA exhibits strong performance across a broad range of graphs and downstream tasks. Remarkably, in few-shot cross-domain settings, it significantly outperforms in-domain baselines and advanced universal pre-training models.

Abstract

Explore similar work

LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training

Surprisingly Simple and Effective Multi-Domain Graph Foundation Model through Graph-to-Table Alignment

CrossHGL: A Text-Free Foundation Model for Cross-Domain Heterogeneous Graph Learning