Toward a universal foundation model for graph-structured data
Authors: Sakib Mostafa, Lei Xing, Md. Tauhidul Islam
Organizations: Department of Radiation Oncology, Stanford University, Stanford, CA, USA · Institute of Computational and Mathematical Engineering, Stanford University, Stanford, CA, USA · Department of Electrical Engineering, Stanford University, Stanford, CA, USA
Graphs are a central representation in biomedical research, capturing molecular interaction networks, gene regulatory circuits, cell--cell communication maps, and knowledge graphs. Despite their importance, currently there is not a broadly reusable foundation model available for graph analysis comparable to those that have transformed language and vision. Existing graph neural networks are typically trained on a single dataset and learn representations specific only to that graph's node features, topology, and label space, limiting their ability to transfer across domains. This lack of generalization is particularly problematic in biology and medicine, where networks vary substantially across cohorts, assays, and institutions. Here we introduce a graph foundation model designed to learn transferable structural representations that are not specific to specific node identities or feature schemes. Our approach leverages feature-agnostic graph properties, including degree statistics, centrality measures, community structure indicators, and diffusion-based signatures, and encodes them as structural prompts. These prompts are integrated with a message-passing backbone to embed diverse graphs into a shared representation space. The model is pretrained once on heterogeneous graphs and subsequently reused on unseen datasets with minimal adaptation. Across multiple benchmarks, our pretrained model matches or exceeds strong supervised baselines while demonstrating superior zero-shot and few-shot generalization on held-out graphs. On the SagePPI benchmark, supervised fine-tuning of the pretrained backbone achieves a mean ROC-AUC of 95.5%, a gain of 21.8% over the best supervised message-passing baseline. The proposed technique thus provides a unique approach toward reusable, foundation-scale models for graph-structured data in biomedical and network science applications.
Figures & tables
Figure 1: Architecture of the Graph Foundation Model. a , Structural prompt construction. For each node, local context descriptors—degree, clustering coefficient, k -core number, ego-network statistics, and PageRank—are combined with global context indicators derived from two community detection algorithms, Label Propagation and SCoDA, each contributing community identity, community size, and intra-community density. These descriptors are assembled into a natural-language node profile and tokenized by a miniature language model (MiniLM) to produce a context embedding. Node features are projected into a matching space and concatenated with this embedding to form the Graph Feature Embedding (GFE), a representation that encodes both attribute content and structural role without relying on dataset-specific feature semantics. b , Multi-graph pretraining. A GraphSAGE backbone ingests multiple heterogeneous graphs simultaneously, producing a Graph Structure Embedding (GSE) for every node via neighborhood aggregation. Contrastive alignment is applied between the GFE and the GSE: the text embedding of node i is matched against the graph embeddings of structurally related nodes and repelled from structurally dissimilar nodes, training the backbone to map diverse graphs into a shared topological representation space. c , Downstream deployment. The frozen or fine-tuned GFM generates GFE and GSE representations for nodes in an unseen graph. These are concatenated and passed to a lightweight task-specific head for node-level classification or regression, requiring minimal labeled data on the target graph.
Figure 2: The pretrained GFM encodes protein functional organization in SagePPI and generalizes with minimal supervision. a , Few-shot generalization curves. Mean ROC-AUC is shown as a function of the number of labeled proteins per GO process, evaluated on held-out test proteins using a logistic probe trained on K positive and K negative examples per label (K ∈ {1, 5, 10, 20}). Solid lines denote GFM variants; dashed lines denote supervised baselines. b , Multifunctionality–embedding density correspondence. Bars show the Spearman rank correlation between the number of GO process annotations per protein and the local embedding density computed from the 15 nearest neighbors in each model’s embedding space. The dashed vertical line separates the pretrained GFM zero-shot model (left) from supervised GNN baselines (right). Positive values indicate that highly annotated proteins occupy denser embedding neighborhoods. c , Cross-functional co-enrichment in GFM Trained embeddings. Each cell reports the mean fraction of GO-annotated neighbors that a protein from one anchor process accumulates within a 25-nearest-neighbor embedding neighborhood annotated by another process. Rows and columns are ordered by hierarchical clustering of the symmetrized co-enrichment matrix. Darker entries indicate higher cross-process neighborhood sharing. d , Macro-averaged ROC curves across all 121 GO process labels. Each curve corresponds to the mean interpolated true positive rate at fixed false positive rate thresholds, averaged across labels with at least two represented classes. The dotted diagonal denotes random performance. e , Summary classification performance. Mean ROC-AUC and accuracy are shown for GFM zero-shot, GFM Trained, and four supervised GNN baselines (GCN, GraphSAGE, GAT, GIN). The dashed vertical line separates the two performance metrics. All models use the same train/validation/test split.
Figure 3: The pretrained GFM generalizes across the ogbn-proteins benchmark and captures GO hierarchy structure. a , Macro-averaged ROC curves across all GO term labels for GFM zero-shot, GFM Trained, and four supervised GNN baselines (GCN, GraphSAGE, GAT, GIN). Solid lines denote GFM variants; dashed lines denote supervised baselines. The dotted diagonal denotes random performance. b , Summary classification performance. Mean ROC-AUC and accuracy are shown for all six models. The dashed vertical line separates the two reported metrics. All models use the standard ogbn-proteins train/validation/test split. c , Multifunctionality–embedding density correspondence. Bars show the Spearman rank correlation between the number of GO annotations per protein and local embedding density computed from the 15 nearest neighbors. The dashed vertical line separates the pretrained GFM zero-shot model from supervised baselines. Positive values indicate that highly annotated proteins occupy denser embedding neighborhoods. d , Few-shot generalization curves. Mean ROC-AUC is shown as a function of the number of labeled proteins per GO process ( K∈{1,5,10,20} ), evaluated on held-out test proteins using a logistic probe trained on K positive and K negative examples per label. Solid lines denote GFM variants; dashed lines denote supervised baselines. e , GO hierarchy depth stratification. Mean ROC-AUC is shown separately for GO terms at shallow, medium, and deep levels of the GO biological process hierarchy. GFM Trained performance increases with hierarchy depth while baseline performance remains flat or declines.
Figure 4: The pretrained GFM generalizes across GO ontologies in the STRING interaction network. a , Mean ROC-AUC and accuracy for Molecular Function (MF) GO term prediction. All six models are compared under the same train/validation/test split on the StringGO network. b , Mean ROC-AUC and accuracy for Cellular Component (CC) GO term prediction under identical conditions. c , Macro-averaged ROC curves for MF. Solid lines denote GFM variants; dashed lines denote supervised baselines. The dotted diagonal denotes random performance. d , Macro-averaged ROC curves for CC. Same protocol as c . e , Cross-functional co-enrichment in GFM Trained embeddings for MF. Each cell reports the mean fraction of GO-term annotated neighbors that a protein from one anchor MF term accumulates within a 25-nearest-neighbor embedding neighborhood annotated by another term. Rows and columns are ordered by hierarchical clustering of the symmetrized co-enrichment matrix. f , Cross-functional co-enrichment for CC. Same protocol as e . Three distinct spatial modules are visible: a mitochondrial module, a nuclear and cytoplasmic module, and a membrane module.
Figure 5: The pretrained GFM generalizes to unseen protein structural fold classes across 144 tissue-specific interaction networks. a , Classification performance at K=20 labeled examples per fold class. Mean ROC-AUC, accuracy, and specificity are shown for all six models, evaluated on test fold classes that were entirely absent during training. Error bars denote one standard deviation across five evaluation seeds. b , Macro-averaged ROC curves across all test fold classes. Solid lines denote GFM variants; dashed lines denote supervised baselines. The dotted diagonal denotes random performance. c , Few-shot generalization curves on unseen fold classes. Mean accuracy is shown as a function of labeled examples per fold class ( K∈{3,5,7,10,15,20} ), evaluated on test fold classes using a logistic probe trained on K support examples per class. Solid lines denote GFM variants; dashed lines denote supervised baselines.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Full macro-averaged ROC curves for SagePPI. Macro-averaged receiver operating characteristic curves across all 121 GO biological process labels for GFM zero-shot, GFM Trained, GCN, GraphSAGE, GAT, and GIN. Curves are computed by interpolating per-label ROC curves onto a common false positive rate grid and averaging. The clear separation between GFM variants (solid lines) and all supervised baselines (dashed lines) is maintained across the full operating range, with the largest absolute gap occurring at low false positive rates where classifier precision requirements are most stringent.
Figure S2: Same-label neighbor enrichment curves for four GO biological processes in SagePPI. For each model, the same-label fraction is computed as the proportion of k nearest neighbors (in cosine embedding space) that share the same GO process annotation as the query protein, averaged over all proteins annotated with that process. Curves are shown for k ∈ {5, 10, 20, 30, 50, 75, 100}. a , DNA repair (GO:0006281). b , Cell cycle regulation (GO:0051726). c , Nervous system development (GO:0007399). d , Negative regulation of transcription (GO:0000122). Solid lines denote GFM variants; dashed lines denote supervised baselines. GFM zero-shot and GFM Trained consistently maintain higher same-label fractions at all values of k across all four processes.
Figure S3: Same-label neighbor enrichment curves for two additional GO biological processes in SagePPI. a , RNA polymerase II transcription (GO:0006366). b , Cell proliferation regulation (GO:0042127). Format and evaluation protocol are identical to Supplementary Fig. 2. The GFM advantage is maintained across both processes, including cell proliferation regulation, which represents a functionally distinct class from the transcriptional and repair processes shown in Supplementary Fig. 2.
Figure S4: Full macro-averaged ROC curves for ogbn-proteins. Macro-averaged receiver operating characteristic curves across all GO biological process labels for GFM zero-shot, GFM Trained, GCN, GraphSAGE, GAT, and GIN. The clear separation between GFM variants (solid lines) and all supervised baselines (dashed lines) persists across the full operating range. The gap is widest at low false positive rates, consistent with results on SagePPI (Supplementary Fig. S1 ).
Figure S5: Same-label neighbor enrichment curves for six GO annotation categories in ogbn-proteins. For each model, the same-label fraction is computed as the proportion of k nearest neighbors (in cosine embedding space) that share the same GO annotation as the query protein, averaged over all annotated proteins. Curves are shown for k∈{5,10,20,30,50,75,100} . a , Molecular function. b , Cellular process. c , Binding. d , Organic substance metabolic process (GO:0071704). e , Cellular metabolic process (GO:0044237). f , Nitrogen compound metabolic process (GO:0006807). Solid lines denote GFM variants; dashed lines denote supervised baselines. GFM zero-shot and GFM Trained consistently maintain higher same-label fractions at all values of k across all six annotation categories.
Figure S6: Cross-functional co-enrichment in GFM Trained embeddings for ogbn-proteins. Each cell reports the enrichment ratio (observed over expected) for the mean fraction of GO-annotated neighbors that a protein from one anchor biological process accumulates within a 10-nearest-neighbor embedding neighborhood annotated by another process. Rows and columns are ordered by hierarchical clustering of the symmetrized co-enrichment matrix. Enrichment ratios substantially above 1.0 indicate that two processes share embedding-space neighborhoods more than expected by chance. The immune cluster—comprising regulation of immune system process, antigen processing and presentation, acute inflammatory response, immune system process, and immune effector process—forms a prominent high-enrichment block. A separate cytokine cluster—positive regulation of cytokine production, regulation of cytokine production, cytokine production, and cell activation—shows strong internal enrichment and cross-enrichment with the immune block, consistent with the known coupling between cytokine signaling and acute inflammatory programs.
Figure S7: GFM performance on Biological Process (BP) GO term prediction in StringGO. a , Mean ROC-AUC and accuracy for BP GO term prediction across all six models. GFM zero-shot achieves the highest mean ROC-AUC (80.5%), exceeding all supervised baselines including GIN (69.3%). The GFM Trained model does not improve over the zero-shot baseline on this ontology, in contrast to the MF and CC results. This behavior is specific to BP and is discussed in the main text. b , Macro-averaged ROC curves for BP. Solid lines denote GFM variants; dashed lines denote supervised baselines. The dotted diagonal denotes random performance. GFM zero-shot maintains a clear separation from all baselines across the full threshold range despite the fine-tuning failure of GFM Trained.
Figure S8: Noise robustness of GFM and baseline embeddings in StringGO. Mean ROC-AUC is shown as a function of Gaussian noise level applied to the amino acid composition features ( xraw ), for MF ( a ), CC ( b ), and BP ( c ). Noise is parameterized as a fraction of the per-feature standard deviation across proteins (0%, 10%, 30%, 50%). GFM zero-shot retains a mean ROC-AUC above 0.79 for MF and CC at 50% noise, indicating that its embeddings are largely driven by topology-derived structural features that are not affected by amino acid feature perturbation. GFM Trained similarly maintains strong performance under all noise levels for MF and CC. On BP, both GFM variants show modest decline but remain above all baselines at every noise level. Baseline performance declines more steeply because amino acid composition constitutes their full input.
Figure S9: Cross-functional co-enrichment in GFM Trained embeddings for BP in StringGO. Same protocol as Fig. 4 e–f. The BP co-enrichment matrix reveals distinct clusters including an immune response module, a signal transduction module, and a transcriptional regulation module. Consistent with the BP performance results, the co-enrichment structure in the GFM zero-shot embedding space is well organized even though the supervised head does not capitalize on it effectively during fine-tuning.
Figure S10: Cross-ontology neighborhood sensitivity in StringGO embeddings. For each model and ontology, the fraction of k nearest neighbors sharing the same dominant GO term is shown for k∈{25,50,75,100} across MF ( a ), BP ( b ), and CC ( c ). The dominant GO term for each protein is defined as its most frequent annotation within the respective ontology. GFM Trained consistently places proteins with matching dominant annotations in closer embedding neighborhoods than all supervised baselines across all three ontologies and all values of k .
Figure S11: t-SNE visualization of protein embeddings across SCOP fold classes in Fold-PPI. Each point represents a protein colored by its SCOP fold class assignment (FC2–FC30). The four smaller panels (top two rows) show embeddings from GFM zero-shot, GCN, GraphSAGE, and GAT. The large bottom panel shows GFM Trained embeddings, in which fold class clusters are visibly separated and compact. Each cluster corresponds to one of the 29 labeled SCOP fold classes present in the training split. The degree of intra-class compactness and inter-class separation in GFM Trained embeddings is substantially greater than in any baseline, indicating that structural pretraining and fine-tuning produce a representation space that organizes protein fold identity from interaction network topology alone.
Figure S12: Tissue-prevalence centralization of fold class embeddings in Fold-PPI. Bars show centralization strength, defined as the negative Spearman rank correlation between the tissue prevalence of each protein’s fold class and its mean distance to the fold class centroid in embedding space. A positive value indicates that proteins belonging to fold classes that appear across more tissue contexts are placed closer to their class centroid, reflecting greater embedding compactness for broadly expressed structural folds. GFM Trained produces a centralization strength of 0.296, substantially exceeding all supervised baselines. GAT achieves the highest baseline value of 0.126. The dashed vertical line separates the GFM Trained model from supervised baselines.
Graph foundation models (GFMs) seek transferable representations across graph domains but are limited by structural heterogeneity and incompatible node feature spaces. We propose Structure-Centric Graph Foundation Models (SCGFM), which treat graph topology as the primary source of transferable knowledge. Modeling graphs as metric measure spaces, SCGFM introduces learnable geometric bases that define a shared structural coordinate system. Graphs are aligned to these bases via Gromov-Wasserstein distances, yielding structure-aligned latent representations that accommodate heterogeneous graph topologies. To address feature incompatibility, SCGFM employs a structure-aware feature re-encoding mechanism that unifies node representations without assuming a fixed feature dimensionality or requiring dataset-specific preprocessing. Experiments on graph- and node-level tasks demonstrate strong in-domain and cross-domain generalization, outperforming existing GFM approaches.
Xiaodong He, Haolan He, Ruiyi Fang +2
University of Electronic Science and Technology of China, Chengdu, Sichuan Province, China · Western University
Graph foundation models face several fundamental challenges including transferability across diverse domains and data scarcity, which calls into question the very feasibility of creating such models. However, despite similar challenges, the tabular domain has recently witnessed the emergence of the first successful foundation models such as TabPFN. These models are based on the prior-data fitted networks (PFN) framework, in which models are pretrained on carefully designed synthetic datasets to make predictions in an in-context learning setting. Recently, G2T-FM, a framework that converts graph node-level tasks into tabular tasks, has made the first step towards adopting PFNs for graphs, yet it is limited to hand-crafted features and was never pretrained on graph data. In this work, we make the next step by proposing GraphPFN, a PFN-based model designed and pretrained specifically for graph node-level tasks. Following the PFN framework, we first design a prior distribution of synthetic attributed graphs by using a novel combination of multi-level stochastic block models and a preferential attachment process for structure generation and graph-aware structured causal models for attribute generation. Then, we augment the tabular foundation model LimiX with attention-based graph neighborhood aggregation layers and train it on millions of synthetic graphs sampled from our prior. On diverse real-world graph datasets with node-level tasks, GraphPFN achieves state-of-the-art results in both in-context learning and finetuning regimes, outperforming G2T-FM, prior GFMs, and task-specific GNNs trained from scratch. More broadly, GraphPFN shows the potential of PFN-based models for building graph foundation models.
Unlike vision and language domains, graph learning lacks a shared input space, as input features differ across graph datasets not only in semantics, but also in value ranges and dimensionality. This misalignment prevents graph models from generalizing across datasets, limiting their use as foundation models. In this work, we propose ALL-IN, a simple and theoretically grounded method that enables transferability across datasets with different input features. Our approach projects node features into a shared random space and constructs representations via covariance-based statistics, thus eliminating dependence on the original feature space. We show that the computed node-covariance operators and the resulting node representations are invariant in distribution to permutations of the input features. We further demonstrate that the expected operator exhibits invariance to general orthogonal transformations of the input features. Empirically, ALL-IN achieves strong performance across diverse node- and graph-level tasks on unseen datasets with new input features, without requiring architecture changes or retraining. These results point to a promising direction for input-agnostic, transferable graph models.
Moshe Eliasof, Krishna Sri Ipsit Mantri, Beatrice Bevilacqua +2
University of Cambridge · Ben-Gurion University of the Negev · Purdue University