Multimodal graphs couple node attributes in different modalities, such as text and images, with relational structure, enabling topological structure and cross-modality attributes to be modeled jointly. Multimodal graph foundation models seek unified representations from such data that transfer across different graph domains and downstream tasks. However, existing methods exhibit two fundamental limitations. (1) Cross-Scope Context Entanglement. They merge scope-specific graph contexts into a unified representation, obscuring their distinctions during multimodal construction. (2) Scope-Ignorant Modality Routing. They route modalities within a fixed graph scope, overlooking how modality relevance varies across neighborhood ranges. To address these challenges, we propose BRAIN, a unified model that focuses on graph context that combines neighborhood scope with modality composition. BRAIN comprises a scope-conditioned Bridge that combines structural information spanning local-to-global neighborhood scopes with different modality compositions; a hierarchical Router that estimates the relevance between the scope and the task, and selects compositions separately within each scope, allowing modality utility to vary with graph range; and a lightweight residual Adapter that further specializes the routed embedding for downstream prediction. BRAIN is trained through multi-graph pretraining followed by task-specific adaptation. Experiments across nine datasets and four task families demonstrate its broad effectiveness, improving node-classification and link-prediction performance by up to 4.73% relative to the strongest baseline, while achieving an average relative improvement of 14.72% across four graph-to-text and two graph-to-image metrics.
Figures & tables
Figure 1: Comparison of context construction paradigms. Prior methods either construct a unified representation or adapt multimodal processing under a prescribed structural organization. BRAIN constructs and routes a candidate pool across neighborhood scopes and modality compositions according to the task and instance.
Figure 2: BRAIN uses two training stages: Stage 1 pretrains reusable scope representations, and Stage 2 learns routing, post-routing adaptation, and task prediction. The Bridge forms a scope–composition candidate pool; the conditional Router estimates αsβm∣s ; and a residual Adapter specializes the routed representation before task prediction.
Method
NC
LP
ele-fashion
RedditS
Grocery
Movies
Bili_Food
DY
sports
Acc
MRR
Hits@3
MRR
Hits@3
MRR
Hits@3
GNN Models
GCN
82.75 ± 0.21
91.87 ± 0.28
79.95 ± 0.39
51.13 ± 0.34
28.56 ± 0.74
35.96 ± 0.98
41.06 ± 0.56
47.79 ± 0.75
85.16 ± 0.25
96.99 ± 0.12
GAT
85.20 ± 0.18
91.08 ± 0.25
80.97 ± 0.36
51.22 ± 0.31
27.18 ± 0.68
33.13 ± 0.89
49.12 ± 0.51
51.56 ± 0.69
90.44 ± 0.20
96.57 ± 0.09
GraphSAGE
85.92 ± 0.17
92.53 ± 0.22
81.08 ± 0.34
52.31 ± 0.29
23.11 ± 0.73
23.66 ± 0.96
44.74 ± 0.59
48.59 ± 0.77
92.58 ± 0.15
96.17 ± 0.07
Table 1: Performance comparison on node classification (NC) and link prediction (LP). The best and second-best mean results are highlighted in bold and underlined, respectively.
Figure 3: Generative performance on Flickr30k G2Text and SemArt G2Image. We report BLEU-4, ROUGE-L, CIDEr, and BERTScore for G2Text, and CLIPScore and DINOv2 similarity for G2Image. Relative improvements are computed against the strongest baseline for each metric.
Figure 4: Data efficiency on Movies NC (left), DY LP (middle), and Bili_Food LP (right). Random initialization and joint Stage 1 pretraining use the same Stage 2 protocol.
Figure 5: Average scope–composition routing distributions across representative NC and LP datasets. The preferred graph range and modality composition vary across datasets and tasks.
Method
Node Classification
Link Prediction
Grocery
RedditS
Movies
Bili_Food
DY
Bridge and candidate construction
w/o graph context
79.31
91.92
49.31
31.78
61.40
w/o composition modulation
82.45
94.31
52.22
34.16
59.23
Single scope (1-hop only)
82.16
93.91
52.97
33.85
62.36
Routing mechanism
Table 2: Ablation results on representative NC and LP datasets. NC reports Accuracy (%); LP reports MRR (%).
Figure 6: Efficiency and memory consumption analysis.
Multimodal graph foundation models (MGFMs) seek to learn generalizable representations from large-scale graphs with heterogeneous node modalities. However, real-world Multimodal-Attributed Graphs (MAGs) often contain incomplete node attributes, limiting the scale and diversity of available pretraining corpora. Besides, existing MGFMs primarily incorporate graph topology as structural context, overlooking its role in guiding multimodal binding and shaping a unified representation space. To address these challenges, we propose GraphBind, a topology-driven approach that uses graph topology to bind rich modality information into a unified shared space. GraphBind is motivated by the stability of graph topology, which provides structural references and complementary semantic information for multimodal binding. Concretely, GraphBind uses topology to organize self semantics and reliable neighborhood semantics into a global shared space that integrates structure and semantics, and adapts this space to discriminative and generative tasks through lightweight interfaces. Extensive experiments against 11 representative baselines demonstrate that GraphBind achieves leading performance on both discriminative and generative tasks, with relative improvements of up to 28.1% over the strongest baseline.
Xunkai Li, Chenxi Wan, Yinlin Zhu +4
Beijing Institute of Technology · Sun Yat-sen University
Graph foundation models (GFMs) have emerged as a promising paradigm for transferring knowledge across graph domains and tasks. Real-world graphs associate nodes with text, images, and other modalities, making multimodal graphs essential for representing complex entities and relations. Moreover, collecting labels and adapting models for every new graph domain is costly and often infeasible, motivating zero-shot transfer. Unfortunately, zero-shot transfer on multimodal graphs remains underexplored. Existing GNN-based graph foundation models typically require downstream adaptation, whereas LLM-based graph methods mainly address unimodal graphs or tasks within a single domain. This setting presents two key challenges. First, models must generalize knowledge from individual modalities while capturing transferable cross-modal relations. Second, without target-domain fine-tuning, node representations remain entangled with domain-specific structures and modality-specific characteristics, obscuring shared concepts in unseen domains. To address these challenges, we propose CHARM, a multimodal graph foundation model with hierarchical context modeling for zero-shot transfer. CHARM replaces isolated raw nodes with hierarchical graph contexts that capture multimodal semantics and cross-modal relations. These contexts map domain-specific node patterns to shared high-level concepts, reducing reliance on target-domain supervision or adaptation. A modality-aware graph context encoder integrates multimodal information with graph structure and converts the resulting representations into graph tokens for a large language model . Experiments show consistent improvements on zero-shot multimodal graph tasks.
Ankang Yang, Jitao Zhao, Di Jin +2
School of Computer Science and Technology, College of Intelligence and Computing, Tianjin University · Data Science Program, Columbian College of Arts and Sciences, The George Washington University
Multimodal attributed graphs (MAGs) integrate graph topology with heterogeneous modality attributes, such as text and images, thereby enabling richer modeling of complex relational systems. However, such expressiveness also makes learning on MAGs depend on multiple semantic sources, including structural topology, textual and visual attributes, each of which can be regarded as a branch for node representation. Node-level branch semantic imbalance arises when these branches differ across nodes in semantic informativeness and reliability: a branch that provides discriminative semantics for one node may mislead another due to bias in modality quality or structural context. Existing methods often mitigate such heterogeneity through cross-branch agreement or alignment, implicitly treating the dominant prediction as reliable supervision. When the dominant branch is biased, forced imitation may propagate its bias to other branches and suppress original semantics that are useful for classification. We propose GraphMNL, a graph-aware multimodal negative learning framework that addresses this issue by using Negative Learning as cross-branch guidance. Instead of forcing inferior branches to imitate a teacher prediction, the model teaches them which classes a node is unlikely to belong to. GraphMNL builds a branch library, identifies dominant and inferior branches via graph-aware reliability arbitration, gates unstable transfer, and applies target-preserving negative learning over non-target classes. This design decouples target supervision from branch guidance so that supervised losses learn the correct class, while Negative Learning suppresses unlikely alternatives when branch agreement is unreliable. Through the comprehensive experimental evaluation, GraphMNL achieves the best performance on Grocery datasets with 72.47% accuracy and 76.60 F1 score on Reddit M datasets.