Learning a Prior-Data Fitted Network for Graph Foundation Models via Structural Causal Model Priors

Ziwei Zhang; Tianyin Liao; Chunyu Hu; Ge Lan; Nuo Xu; Yi Huang; Qingyun Sun; Xingxuan Zhang

cs.LGOct 28, 2026

Learning a Prior-Data Fitted Network for Graph Foundation Models via Structural Causal Model Priors

Authors: Ziwei Zhang, Tianyin Liao, Chunyu Hu, Ge Lan, Nuo Xu, Yi Huang, Qingyun Sun, Xingxuan Zhang,

Organizations: Beihang University Haidian District, Beijing, China · Nankai University Binhai New Area, Tianjin, China · Tsinghua University Haidian District, Beijing, China

Abstract

Graph Foundation Models (GFMs) have recently emerged as a promising paradigm for learning transferable representations across different graphs and domains. However, existing GFMs face notable limitations, including limited pre-training scale or reliance on aligning with textual information in Large Language Models (LLMs). To overcome these limitations, recent studies have drawn inspiration from prior-data fitted networks (PFNs), which learn from large synthetic datasets and support multi-domain prediction backed by causal learning. However, existing graph PFNs encounter severe difficulties in modeling causal relationships within and across node and edge sets, generating diverse graph structures, and designing suitable architectures that can learn from complex synthetic datasets. To address these challenges, we propose GSCM-PFN (Graph SCM-based Prior-Data Fitted Network), a graph PFN pre-trained on synthetic data generated from an SCM-based prior that jointly models nodes and graph topology. Specifically, we (1) design a joint feature-edge-label structural causal model (SCM) to guide synthetic graph generation and capture causal relationships within and across node and edge sets; (2) introduce prior-guided node-edge dependency modeling to instantiate diverse graph generation strategies, and prove that our SCM-based generation prior is more expressive than existing priors; and (3) propose a feature-permutation-invariant dual-path graph network with theoretical guarantees for learning from diverse synthetic graphs. Experiments on 23 real-world benchmarks show that GSCM-PFN outperforms both conventional GFMs and PFN-based GFMs, without utilizing real-world graph data for pre-training. Code: https://anonymous.4open.science/r/GSCM-PFN-A3A3/

Figures & tables

Figure 1. The framework of GSCM-PFN . Our proposed method is composed of three modules: (1) the joint feature-edge-label SCM, which defines a causal prior over node features, graph topology, and labels; (2) the prior-guided node-edge dependency modeling module, which generates graphs by updating node–edge variables at the sample level and individual dimensions at the feature level; and (3) the feature-permutation-invariant dual-path graph network, which performs in-context prediction while addressing heterogeneous feature semantics and topological dependencies that vary in range.

Dataset	GCN	GAT	GraphAny	TS-Mean	TS-GAT	RGVT	GraphPFN	NodePFN	GSCM-PFN
Actor	28.55 ±0.68	27.30 ±0.22	28.60 ±0.21	28.09 ±0.93	28.80 ±2.12	32.28 ±0.68	34.46 ±1.18	32.82 ±1.03	35.72 ±1.18
AirBrazil	42.31 ±7.98	57.69 ±14.75	34.61 ±16.09	39.23 ±5.70	36.92 ±10.03	86.16 ±5.76	67.69 ±5.76	72.31 ±6.62	72.31 ±2.88
AirEU	41.88 ±3.60	32.50 ±8.45	41.50 ±6.50	35.88 ±6.91	38.38 ±6.32	54.75 ±1.75	57.75 ±4.87	53.50 ±4.53	56.50 ±1.29
AirUS	46.49 ±1.81	48.47 ±4.17	43.64 ±1.83	42.34 ±2.12	41.69 ±2.59	54.20 ±1.68	61.87 ±1.56	59.14 ±2.30	62.77 ±1.63
Amazon-Comp	85.83 ±0.86	87.01 ±0.50	83.04 ±1.24	81.37 ±1.25	80.23 ±2.44	83.85 ±0.54	77.62 ±1.40	78.44 ±1.64	83.11 ±1.57
Amazon-Photo	91.88 ±0.79	91.86 ±1.07	90.60 ±0.82	90.18 ±1.30	89.66 ±1.23	91.98 ±0.44	87.89 ±2.17	87.88 ±1.32	91.63 ±1.21

Table 1. Performance comparison of different methods on 23 real-world benchmark datasets, measured by accuracy. Numbers after ± indicate standard deviations. The best results are in bold and the second-best results are underlined .

Panel of Figure 2 — Figure 2. The impact of context ratio and label noise.

Prior
Method	Accuracy
Full GSCM-PFN	72.58
w/o prior-guided message passing	68.00 ( $-$ 4.58)
w/o $Y$ Decouple	71.62 ( $-$ 0.96)
w/o $A$ Decouple	71.25 ( $-$ 1.33)
w/o multilayer structure	71.11 ( $-$ 1.47)

Table 2. Results of ablation for the synthetic prior and the model architecture. We report the average test accuracy. Numbers after

-

indicate the change relative to the full model.

Panel of Figure 3 — Figure 3. Histograms comparing the degree assortativity, global clustering, class-degree association, and adjusted homophily of synthetic graphs.

Panel of Figure 4 — Figure 4. Visualizations of synthetic graphs generated by our method.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Nodes	#Edges	#Features	#Classes
Actor	7,600	26,659	932	5
AirBrazil	131	1,003	131	4
AirEU	399	5,993	399	4
AirUS	1,190	13,599	1,190	4
Amazon-Comp	13,752	245,861	767	10
Amazon-Photo	7,650	119,081	745	8

Appendix

Table 3. Statistics of the 23 test datasets. #Edges counts each undirected edge once, with self-loops removed.

Graph Foundation Models

Graph Foundation Models are pretrained graph-learning systems designed to transfer across datasets, tasks, and domains with different structures and features.

See all 97 papers

GraphPFN: A Prior-Data Fitted Graph Foundation Model
Sep 25, 2025Dmitry Eremeev, Oleg Platonov, Gleb Bazhenov +2Synthetic Data Pretraining In-Context Learning
1HSE University · 2Yandex Research
Structure-Centric Graph Foundation Model via Geometric Bases
May 9, 2026Xiaodong He, Haolan He, Ruiyi Fang +2Geometric Representation Learning Graph Foundation Models
University of Electronic Science and Technology of China, Chengdu, Sichuan Province, China · Western University
Path to Prior-Data Fitted Relational Models via Learned Latent Multi-table Representations
Oct 16, 2026Mohammad Sadeq Abolhasani, Viswanath GanapathyRelational Foundation Models Synthetic Data Pretraining
SAP Labs

Sep 25, 2025J/K move · Enter open · S save

GraphPFN: A Prior-Data Fitted Graph Foundation Model

Dmitry Eremeev, Oleg Platonov, Gleb Bazhenov, Artem Babenko+1

1HSE University · 2Yandex Research

Synthetic Data Pretraining In-Context Learning Graph Foundation Models Prior-Data Fitted Networks Node Classification

Open paper AskPDF arXiv

Graph foundation models face several fundamental challenges including transferability across diverse domains and data scarcity, which calls into question the very feasibility of creating such models. However, despite similar challenges, the tabular domain has recently witnessed the emergence of the first successful foundation models such as TabPFN. These models are based on the prior-data fitted networks (PFN) framework, in which models are pretrained on carefully designed synthetic datasets to make predictions in an in-context learning setting. Recently, G2T-FM, a framework that converts graph node-level tasks into tabular tasks, has made the first step towards adopting PFNs for graphs, yet it is limited to hand-crafted features and was never pretrained on graph data. In this work, we make the next step by proposing GraphPFN, a PFN-based model designed and pretrained specifically for graph node-level tasks. Following the PFN framework, we first design a prior distribution of synthetic attributed graphs by using a novel combination of multi-level stochastic block models and a preferential attachment process for structure generation and graph-aware structured causal models for attribute generation. Then, we augment the tabular foundation model LimiX with attention-based graph neighborhood aggregation layers and train it on millions of synthetic graphs sampled from our prior. On diverse real-world graph datasets with node-level tasks, GraphPFN achieves state-of-the-art results in both in-context learning and finetuning regimes, outperforming G2T-FM, prior GFMs, and task-specific GNNs trained from scratch. More broadly, GraphPFN shows the potential of PFN-based models for building graph foundation models.

Abstract

Graph Foundation Models

Explore similar work

GraphPFN: A Prior-Data Fitted Graph Foundation Model

Structure-Centric Graph Foundation Model via Geometric Bases

Path to Prior-Data Fitted Relational Models via Learned Latent Multi-table Representations