cs.LGSep 29, 2026

Message Passing Does More with Less for In-Context Learning on Graphs

Authors: Dooho Lee, Jinmo Lee, Minho Jeong, Kijung Shin, Jaemin Yoo

Organizations: Nums AI · KAIST · Seoul National University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Achieving strong performance with graph neural networks (GNNs) typically requires training and hyperparameter tuning for each dataset, incurring repeated costs and effort. Graph in-context learning (ICL) avoids this by using a single pretrained model to predict unknown node labels directly from labeled context nodes. Existing approaches, however, rely on dense attention across nodes, making inference increasingly expensive as graphs grow. In this work, we present Ephris, a new graph in-context learner built on sparse message passing, scaling linearly with the number of node-feature entries and graph edges. Ephris is pretrained entirely on synthetic graphs generated from structural causal models with diverse graph structures and relational dynamics, exposing the model to varied dependencies among topology, features, and labels. We evaluate Ephris on 51 node-classification datasets against 15 extensively tuned GNNs and existing graph ICL methods under both high- and low-label train/validation/test splits. Across both settings, Ephris ranks first on all four aggregate measures: Elo, improvability, average rank, and accuracy. Its inference cost remains comparable to training a single GNN once, while being over 10 times faster than previous graph ICL models. Together, these results advance the performance-runtime Pareto frontier, demonstrating that strong graph ICL does not require dense attention. Code and model weights are available at https://github.com/nums-ai/ephris.

Figures & tables

Figure 1: Performance-runtime Pareto plot across 51 datasets under the 50/25/25 split.
Figure 2: Architecture overview of Ephris. Ephris tokenizes each scalar node-feature value into a dd -dimensional token, compresses feature tokens into fixed-size node representations while integrating distributional and graph context, and performs ICL through stacked message passing.
ModelComputation
NodePFNO(NF+N2+E)\mathcal{O}(NF+N^{2}+E)
GraphPFNO(N2F+NF2+EF)\mathcal{O}(N^{2}F+NF^{2}+EF)
EphrisO(NF+E)\mathcal{O}(NF+E)
Table 1: Theoretical complexity.
Figure 3: Inference runtime scaling; median over three seeds.
 Stage 1Stage 2
# Node512–1,024128–16,384
# Feature2–1282–1,024
# Steps50,00010,000
Muon LR8×10−48{\times}10^{-4}4×10−54{\times}10^{-5}
AdamW LR8×10−48{\times}10^{-4}10−510^{-5}
Table 2: Two-stage training setup.
Figure 4: Elo across label regimes. Each bar nests a default GNN within its HP -tuned counterpart. Hatching marks OOM results imputed with GCN (Default); error bars show 95% confidence intervals. Methods are ordered by mean Elo across regimes. Ephris achieves the highest Elo in both.
Figure 5: Performance-runtime trade-offs. Mean improvability versus runtime overhead under high-label (left) and low-label (right) regimes; lower is better on both axes. Black and gray dashed lines show Pareto frontiers with and without Ephris, with the improvement shaded blue.
Figure 6: Subgroup performance. Average ranks under the high-label regime.
Figure 7: Architecture–prior factorial study. Bold: column best, underline: row best.
Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

SettingChoices or range
RuleDiffusion, cascade, degree-dependent mixing
AggregationMean, max, min, softmax
DirectionIncoming, outgoing, bidirected
Mixing coefficient α\alpha[0.15,0.85][0.15,0.85]
Steps TT (diffusion and cascade){2,3,4,5,6}\{2,3,4,5,6\}
Seed fraction ss (cascade)[0.08,0.28][0.08,0.28]
Appendix
Table 3: Relational-dynamics settings. Discrete choices and continuous intervals are sampled uniformly. Rule-specific parameters apply only to the indicated operations.
ParameterDistribution
Spectral decay η\etaU(0.5,2.0)\mathcal{U}(0.5,2.0)
Nuisance rank ν\nuUniform over {0,…,min⁡(8,q)}\{0,\ldots,\min(8,q)\}
Nuisance amplitude λ\lambdaU(0,0.5)\mathcal{U}(0,0.5)
Noise scale ϵ\epsilonLogUniform⁡(0.01,0.35)\operatorname{LogUniform}(0.01,0.35)
Lognormal scale σr\sigma_{r}U(0.05,0.5)\mathcal{U}(0.05,0.5)
Appendix
Table 4: Embedding variation. Sampling distributions for projection, noise, and row scaling. Lognormal scaling multiplies row ii by exp⁡(σrzi)\exp(\sigma_{r}z_{i}) , with zi∼N(0,1)z_{i}\sim\mathcal{N}(0,1) .
ComponentSettingValue
TokenizationFeature-token dimension128
Per-feature refinementInducing tokens128
 Attention heads8
Per-node refinementRefinement blocks3
 Node-inducing tokens per node4
 MPICL\mathrm{MP}_{\mathrm{ICL}} layers per block2
Appendix
Table 5: Architecture configuration of Ephris.
ComponentSettingStage 1Stage 2
Task distributionUpdates50,00010,000
Graphs per update6464
Node-count bounds512 to 1,024128 to 16,384
Feature-count bounds2 to 1282 to 1,024
Requested class count2 to 102 to 10
Context fractionUniform on [0.3,0.9][0.3,0.9]0.80.8
Appendix
Table 6: Two-stage pretraining configuration. Ranges specify sampling bounds before integer conversion. Stage 2 restricts the feature-count and target-degree supports using memory budgets.
DatasetDomainNNFFCCdˉ\bar{d}hadjh_{\mathrm{adj}}Citation
actorWeb7,60093257.020.003( Pei et al., 2020 )
amazon_computerCommerce13,7527671035.760.682( Shchur et al., 2018 )
amazon_photoCommerce7,650745831.130.785( Shchur et al., 2018 )
amazon_ratingsCommerce24,49230057.600.140( Platonov et al., 2023 )
amherst41Social2,2351,193281.390.060( Lim et al., 2021 )
artnet-expSocial50,40575211.120.155( Bazhenov et al., 2026 )
Appendix
Table 7: Overview of the 51 node-classification datasets. We report each dataset’s application domain and key characteristics: number of nodes ( NN ), features ( FF ), classes ( CC ), average degree ( dˉ\bar{d} ), and adjusted label homophily ( hadjh_{\mathrm{adj}} ). Citations refer to the original dataset or benchmark sources.
MethodRef.DefaultSearch space
Shared training–Adam; lr =10−2=10^{-2} ; wd =5 ⁣× ⁣10−4=5\!\times\!10^{-4} ; 2,500 epochs; patience =100=100lr ∈{10−3,5 ⁣× ⁣10−3,10−2}\in\{10^{-3},5\!\times\!10^{-3},10^{-2}\} ; wd ∈{0,5 ⁣× ⁣10−5,5 ⁣× ⁣10−4}\in\{0,5\!\times\!10^{-5},5\!\times\!10^{-4}\} ; patience ∈{20,100}\in\{20,100\}
APPNP( Gasteiger et al., 2018 )d=64d=64 ; dropout =.5=.5 ; K=10K=10 ; α=.1\alpha=.1 ; prop. dropout =0=0d∈{64,128,256,512}d\in\{64,128,256,512\} ; dropout ∈{0,.3,.5,.7}\in\{0,.3,.5,.7\} ; K∈{5,10,20}K\in\{5,10,20\} ; α∈{.1,.2,.5}\alpha\in\{.1,.2,.5\} ; prop. dropout ∈{0,.3,.5}\in\{0,.3,.5\}
FAGCN( Bo et al., 2021 )d=32d=32 ; dropout =.5=.5 ; L=2L=2 ; ϵ=.3\epsilon=.3 ; edge dropout =.5=.5d∈{32,64,128}d\in\{32,64,128\} ; feature/edge dropout ∈{0,.3,.5,.7}\in\{0,.3,.5,.7\} ; L∈{1,2,3,4}L\in\{1,2,3,4\} ; ϵ∈{.1,.2,.3,.5}\epsilon\in\{.1,.2,.3,.5\}
GAT( Veličković et al., 2018 )d=64d=64 ; heads =1=1 ; dropout =.5=.5 ; attn. dropout =.2=.2 ; L=2L=2 ; norm=None; residual=False; pre-transform=Falsed∈{64,128,256,512}d\in\{64,128,256,512\} ; heads ∈{1,4}\in\{1,4\} ; dropout ∈{0,.3,.5,.7}\in\{0,.3,.5,.7\} ; attn. dropout ∈{0,.2}\in\{0,.2\} ; L∈{1,2,4,8}L\in\{1,2,4,8\} ; norm ∈{none,layernorm}\in\{\texttt{none},\texttt{layernorm}\} ; residual/pre-transform ∈{F,T}\in\{\mathrm{F},\mathrm{T}\}
GATv2( Brody et al., 2021 )d=64d=64 ; heads =1=1 ; dropout =.5=.5 ; attn. dropout =.2=.2 ; L=2L=2 ; norm=None; residual=False; pre-transform=Falsed∈{64,128,256,512}d\in\{64,128,256,512\} ; heads ∈{1,4}\in\{1,4\} ; dropout ∈{0,.3,.5,.7}\in\{0,.3,.5,.7\} ; attn. dropout ∈{0,.2}\in\{0,.2\} ; L∈{1,2,4,8}L\in\{1,2,4,8\} ; norm ∈{none,layernorm}\in\{\texttt{none},\texttt{layernorm}\} ; residual/pre-transform ∈{F,T}\in\{\mathrm{F},\mathrm{T}\}
GCN( Kipf and Welling, 2017 )d=64d=64 ; dropout =.5=.5 ; L=2L=2 ; norm=None; residual=False; pre-transform=Falsed∈{64,128,256,512}d\in\{64,128,256,512\} ; dropout ∈{0,.3,.5,.7}\in\{0,.3,.5,.7\} ; L∈{1,2,4,8}L\in\{1,2,4,8\} ; norm ∈{none,layernorm}\in\{\texttt{none},\texttt{layernorm}\} ; residual/pre-transform ∈{F,T}\in\{\mathrm{F},\mathrm{T}\}
Appendix
Table 8: Default configurations and hyperparameter search spaces for 15 supervised GNN . We sample 200 configurations from the listed candidate values. dd denotes hidden dimension and LL the number of layers. Unless otherwise specified, methods use the shared training configuration and search space in the first row.
No shift — ID-selected
MethodCoraArxivWebKBTwitchAvg. rank ↓\downarrow
 WordDegreeTimeDegreeUniversityLanguage 
ERM69.41 ±0.3069.41\,{\scriptstyle\pm\,0.30}69.42 ±0.3069.42\,{\scriptstyle\pm\,0.30}{\color[rgb]{0.0859,0.5,0.2344}\mathbf{73.02}}\,{\scriptstyle\pm\,0.14}{\color[rgb]{0.0859,0.5,0.2344}\mathbf{72.99}}\,{\scriptstyle\pm\,0.12}47.85 ±0.8947.85\,{\scriptstyle\pm\,0.89}68.05 ±0.5268.05\,{\scriptstyle\pm\,0.52}5.92
IRM69.42 ±0.3869.42\,{\scriptstyle\pm\,0.38}69.40 ±0.3869.40\,{\scriptstyle\pm\,0.38}72.90 ±0.1472.90\,{\scriptstyle\pm\,0.14}72.92 ±0.0772.92\,{\scriptstyle\pm\,0.07}47.31 ±1.2147.31\,{\scriptstyle\pm\,1.21}68.30 ±0.2968.30\,{\scriptstyle\pm\,0.29}7.25
VREx69.43 ±0.2969.43\,{\scriptstyle\pm\,0.29}69.42 ±0.2969.42\,{\scriptstyle\pm\,0.29}72.84 ±0.0972.84\,{\scriptstyle\pm\,0.09}72.88 ±0.0972.88\,{\scriptstyle\pm\,0.09}47.85 ±0.8947.85\,{\scriptstyle\pm\,0.89}68.07 ±0.5268.07\,{\scriptstyle\pm\,0.52}7.25
GroupDRO69.46 ±0.2569.46\,{\scriptstyle\pm\,0.25}69.40 ±0.3069.40\,{\scriptstyle\pm\,0.30}72.91 ±0.1272.91\,{\scriptstyle\pm\,0.12}72.98 ±0.1072.98\,{\scriptstyle\pm\,0.10}47.85 ±0.8947.85\,{\scriptstyle\pm\,0.89}{\color[rgb]{0.0859,0.5,0.2344}\mathbf{69.19}}\,{\scriptstyle\pm\,0.28}5.75
Appendix
Table 9: GOOD: no shift and covariate shift. Results on the official GOOD splits using either ID or OOD validation for baseline model selection. Scores are mean ±\pm SD in percent (ROC-AUC for Twitch; accuracy otherwise). Baseline results are reported by GOOD ( Gui et al., 2022 ) , while Ephris uses the same frozen checkpoint in both settings without model selection. Red , blue , and green mark the three best distinct scores in each column. Average ranks are computed separately for ID- and OOD-based selection across the six dataset-attribute settings.
ID-selected
MethodCoraArxivWebKBTwitchAvg. rank ↓\downarrow
 WordDegreeTimeDegreeUniversityLanguage 
ERM64.20 ±0.5664.20\,{\scriptstyle\pm\,0.56}60.38 ±0.3360.38\,{\scriptstyle\pm\,0.33}{\color[rgb]{0.8398,0.1523,0.1563}\mathbf{65.70}}\,{\scriptstyle\pm\,0.42}{\color[rgb]{0.082,0.3945,0.7539}\mathbf{61.77}}\,{\scriptstyle\pm\,0.29}24.77 ±0.4324.77\,{\scriptstyle\pm\,0.43}{\color[rgb]{0.0859,0.5,0.2344}\mathbf{48.57}}\,{\scriptstyle\pm\,0.17}4.17
IRM64.16 ±0.6164.16\,{\scriptstyle\pm\,0.61}{\color[rgb]{0.0859,0.5,0.2344}\mathbf{61.00}}\,{\scriptstyle\pm\,0.34}{\color[rgb]{0.082,0.3945,0.7539}\mathbf{65.69}}\,{\scriptstyle\pm\,0.55}61.49 ±0.3661.49\,{\scriptstyle\pm\,0.36}24.16 ±0.8024.16\,{\scriptstyle\pm\,0.80}{\color[rgb]{0.082,0.3945,0.7539}\mathbf{49.77}}\,{\scriptstyle\pm\,0.82}5.50
VREx64.20 ±0.5464.20\,{\scriptstyle\pm\,0.54}60.05 ±0.7260.05\,{\scriptstyle\pm\,0.72}65.40 ±0.5465.40\,{\scriptstyle\pm\,0.54}61.61 ±0.3261.61\,{\scriptstyle\pm\,0.32}24.77 ±0.4324.77\,{\scriptstyle\pm\,0.43}48.56 ±0.1848.56\,{\scriptstyle\pm\,0.18}6.00
GroupDRO{\color[rgb]{0.0859,0.5,0.2344}\mathbf{64.38}}\,{\scriptstyle\pm\,0.34}60.03 ±0.8860.03\,{\scriptstyle\pm\,0.88}{\color[rgb]{0.0859,0.5,0.2344}\mathbf{65.57}}\,{\scriptstyle\pm\,0.66}61.59 ±0.5661.59\,{\scriptstyle\pm\,0.56}24.77 ±0.4324.77\,{\scriptstyle\pm\,0.43}47.44 ±1.0847.44\,{\scriptstyle\pm\,1.08}5.67
Appendix
Table 10: GOOD: concept shift. Results on the official GOOD splits using either ID or OOD validation for baseline model selection. Scores are mean ±\pm SD in percent (ROC-AUC for Twitch; accuracy otherwise). Baseline results are reported by GOOD ( Gui et al., 2022 ) , while Ephris uses the same frozen checkpoint in both settings without model selection. Red , blue , and green mark the three best distinct scores in each column. Average ranks are computed separately for ID- and OOD-based selection across the six dataset-attribute settings.
MethodCoraCiteSeerPubMedCora-CADBLP-CA
Published: AllSet (ICLR 2022), Table 2
AllSetTransformer78.59±1.4778.59{\scriptstyle\pm 1.47}{\color[rgb]{0.0859,0.5,0.2344}\mathbf{73.08}}{\scriptstyle\pm 1.20}88.72±0.3788.72{\scriptstyle\pm 0.37}83.63±1.4783.63{\scriptstyle\pm 1.47}91.53±0.2391.53{\scriptstyle\pm 0.23}
AllDeepSets76.88±1.8076.88{\scriptstyle\pm 1.80}70.83±1.6370.83{\scriptstyle\pm 1.63}{\color[rgb]{0.0859,0.5,0.2344}\mathbf{88.75}}{\scriptstyle\pm 0.33}81.97±1.5081.97{\scriptstyle\pm 1.50}91.27±0.2791.27{\scriptstyle\pm 0.27}
MLP75.17±1.2175.17{\scriptstyle\pm 1.21}72.67±1.5672.67{\scriptstyle\pm 1.56}87.47±0.5187.47{\scriptstyle\pm 0.51}74.31±1.8974.31{\scriptstyle\pm 1.89}84.83±0.2284.83{\scriptstyle\pm 0.22}
CECGN76.17±1.3976.17{\scriptstyle\pm 1.39}70.16±1.3170.16{\scriptstyle\pm 1.31}86.45±0.4386.45{\scriptstyle\pm 0.43}77.05±1.2677.05{\scriptstyle\pm 1.26}88.00±0.2688.00{\scriptstyle\pm 0.26}
CEGAT76.41±1.5376.41{\scriptstyle\pm 1.53}70.63±1.3070.63{\scriptstyle\pm 1.30}86.81±0.4286.81{\scriptstyle\pm 0.42}76.16±1.1976.16{\scriptstyle\pm 1.19}88.59±0.2988.59{\scriptstyle\pm 0.29}
Appendix
Table 11: Transfer to hypergraph node classification. Test accuracy (%, mean ±\pm standard deviation) under random 50/25/25 splits. We evaluate two graph representations: clique , which connects nodes that share a hyperedge, and incidence , which introduces unlabeled hyperedge nodes with mean member features and a node-kind indicator. Both Ephris variants use the same 20 splits and perform inference only, using the same pretrained checkpoint as in the main experiments without additional training. Published baseline results are taken from AllSet ( Chien et al., 2021 ) (Table 2), which uses the same random 50/25/25 splits over 20 runs. Red , blue indicate the first-, second-highest accuracies, respectively, with ties sharing the same color.
Figure 8: Architecture (left) and prior (right) ablations. Models are pretrained for 5,000 steps. Curves show training cross-entropy smoothed with a trailing 100-step mean, with faint traces indicating raw losses; insets magnify the final 300k–320k training graphs. Tables report mean accuracy (Acc., %), average rank (Rank), and the percentage of strict wins against the full model (Win, %) across 51 datasets under high-label regime. Ranks are computed separately within the architecture and prior ablations, and table swatches indicate the corresponding training curves.
CharacteristicSubgroups
Nodes NNTiny ( ≤2\leq 2 K), Small ( 22 K to 1010 K), Medium ( 1010 K to 100100 K), Large ( >100>100 K)
Avg. degree dˉ\bar{d}Low ( ≤5\leq 5 ), Medium ( 55 to 2020 ), High ( >20>20 )
Adj. homophily hadjh_{\mathrm{adj}}Negative ( <0<0 ), Low ( 00 to 0.50.5 ), High ( ≥0.5\geq 0.5 )
Features FFLow ( ≤500\leq 500 ), Medium ( 500500 to 55 K), High ( ≥5\geq 5 K)
Nonzero fraction rrSparse ( ≤1%\leq 1\% ), Medium ( 1%1\% to 50%50\% ), Dense ( >50%>50\% )
Classes CCBinary ( 22 ), Medium ( 33 to 1010 ), Many ( >10>10 )
Appendix
Table 12: Dataset subgroup definitions.
Figure 9: Aggregate performance across evaluation metrics. Paired comparisons under the high- and low-label regimes for (a) Elo, (b) improvability, (c) average rank, and (d) average accuracy. Each bar shows a default GNN alongside its HP -tuned counterpart. Hatching marks results imputed for OOM runs. Methods are ordered by their mean performance across the two label regimes.
Figure 10: Performance-runtime Pareto plots. Trade-offs between average runtime overhead and (a) Elo, (b) average rank, and (c) average accuracy. Each point represents an evaluated method, and the Pareto frontier identifies methods that are not jointly dominated in performance and runtime.
Figure 11: Pairwise win-rate matrices. Pairwise comparisons under (a) the high-label and (b) the low-label regime. Each entry reports the fraction of datasets on which the row method achieves higher test accuracy than the column method.
Figure 12: Performance across dataset subgroups under the high-label regime (50/25/25). Radar plots show subgroup performance measured by (a) Elo, (b) improvability, (c) average rank, and (d) average accuracy. Each axis corresponds to a dataset subgroup, providing a more detailed view of performance across different dataset characteristics.
Figure 13: Performance across dataset subgroups under the low-label regime (10/10/80). Radar plots show subgroup performance measured by (a) Elo, (b) improvability, (c) average rank, and (d) average accuracy. Each axis corresponds to a dataset subgroup, providing a more detailed view of performance across different dataset characteristics.
More in this topic

In-Context Learning

In-context learning studies how models adapt to new tasks from examples or instructions supplied at inference time, without changing their weights.

See all 402 papers

Explore similar work

CardsList
Sep 5, 2026

LoGIC: Budgeted Context Construction for Node-Level Graph In-Context Learning with Tabular Foundation Models

Tabular foundation models have become powerful graph learners. Systems such as G2T-FM and GraphPFN encode each node as a feature row and make predictions through in-context learning (ICL), with labeled rows serving as the prompt. Current protocols employ the complete training table as context, causing attention to scale quadratically with the labeled pool and introducing preprocessing and memory bottlenecks. We investigate context construction for node-level graph ICL: which labeled nodes and auxiliary unlabeled nodes should constitute the prompt for specified queries. We formulate this allocation in terms of two resources: a labeled-context budget for predictive evidence and an unlabeled-halo budget for adapter message passing without using label capacity. We present LoGIC, which retrieves labeled nodes via structural, feature-based, and coverage channels, shares each context across the queries in a graph-local cluster, incorporates an unlabeled halo for adapter backbones, and chooses the channel and context budget without test labels. Across three backbone configurations drawn from two model families on GraphLand, budgeted contexts maintain locally runnable full-context performance, stay competitive with published large-dataset results, and markedly lower peak memory requirements compared with full-context and whole-graph inference. They further permit frozen graph ICL on million-node graphs without retraining. Our analysis identifies when retrieval channels work best and connects their behavior with graph properties.

Mingqi Yang, Zidong Guo, Jihui Yang +1

Department of Electronic Business, South China University of Technology, Guangzhou, Guangdong 510006, China · Meta Superintelligence Lab

In-Context LearningRetrieval-Augmented ICLTabular Foundation Models
AskPDF
May 23, 2026

Advancing Graph Few-Shot Learning via In-Context Learning

Graph few-shot learning, which aims to classify nodes from novel classes with only a few labeled examples, is a widely studied problem in graph learning. However, existing methods often face two key limitations. First, the predominant graph few-shot learning paradigm relies on supervised tasks, failing to leverage the vast number of unlabeled nodes in the graph. Second, many approaches require complex task adaptation or fine-tuning during inference, limiting their efficiency and applicability. Inspired by the powerful in-context learning capabilities of large language models, we propose a novel model named VISION for adVancIng graph few-Shot learning via In-cOntext LearNing to address these challenges. Our model reframes graph few-shot learning as a fine-tuning-free sequence reasoning problem. At its core is a context-aware network that initializes nodes with role embeddings and employs a dual-context fusion module to synergistically integrate local topological structures and global task-level dependencies. This allows our model to dynamically generate class-aware representations for the query set conditioned on the support set context in a single forward pass. To effectively train our model, we introduce an unsupervised task generator that creates structure-adaptive features and constructs diverse pseudo-tasks from abundant unlabeled data. Our method unifies unsupervised meta-learning with graph in-context learning, achieving efficient inference. Extensive experiments on multiple benchmark datasets demonstrate the superiority of our model. Our public code can be found

Renchu Guan, Yajun Wang, Chunli Guo +5

Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education

Meta-LearningFew-Shot LearningIn-Context Learning
AskPDF
Apr 21, 2026

Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors

One of the most challenging problems in graph machine learning is generalizing across graphs with diverse properties. Graph neural networks (GNNs) face a fundamental limitation: they require separate training for each new graph, preventing universal generalization across diverse graph datasets. A critical challenge facing GNNs lies in their reliance on labeled training data for each individual graph, a requirement that hinders the capacity for universal node classification due to the heterogeneity inherent in graphs -- differences in homophily levels, community structures, and feature distributions across datasets. Inspired by the success of large language models (LLMs) that achieve in-context learning through massive-scale pre-training on diverse datasets, we introduce NodePFN. This universal node classification method generalizes to arbitrary graphs without graph-specific training. NodePFN learns posterior predictive distributions (PPDs) by training only on thousands of synthetic graphs generated from carefully designed priors. Our synthetic graph generation covers real-world graphs through the use of random networks with controllable homophily levels and structural causal models for complex feature-label relationships. We develop a dual-branch architecture combining context-query attention mechanisms with local message passing to enable graph-aware in-context learning. Extensive evaluation on 23 benchmarks demonstrates that a single pre-trained NodePFN achieves 71.27 average accuracy. These results validate that universal graph learning patterns can be effectively learned from synthetic priors, establishing a new paradigm for generalization in node classification.

Jeongwhan Choi, Jongwoo Kim, Woosung Kang +1

KAIST Daejeon, Republic of Korea

Synthetic Data PretrainingOOD GeneralizationIn-Context LearningGraph Foundation ModelsNode Classification
AskPDF