Multimodal-attributed graphs (MAGs) are a fundamental data structure for multimodal graph learning (MGL), enabling both graph-centric and modality-centric tasks. However, our empirical analysis reveals inherent topology quality limitations in real-world MAGs, including noisy interactions, missing connections, and task-agnostic relational structures. A single graph derived from generic relationships is therefore unlikely to be universally optimal for diverse downstream tasks. To address this challenge, we propose Task-aware Modality and Topology co-Evolution (TMTE), a novel MGL framework that jointly and iteratively optimizes graph topology and multimodal representations toward the target task. TMTE is motivated by the bidirectional coupling between modality and topology: multimodal attributes induce relational structures, while graph topology shapes modality representations. Concretely, TMTE casts topology evolution as multi-perspective metric learning over modality embeddings with an anchor-based approximation, and formulates modality evolution as smoothness-regularized fusion with cross-modal alignment, yielding a closed-loop task-aware co-evolution process. Extensive experiments on 9 MAG datasets and 1 non-graph multimodal dataset across 6 graph-centric and modality-centric tasks show that TMTE consistently achieves state-of-the-art performance. Our code is available at https://anonymous.4open.science/r/TMTE-1873.
Figures & tables
Figure 1. Experimental results of our empirical study . We compare the proposed TMTE and MM-GCN (representative MGL baseline) on the Toys dataset. (a) Node classification performance under three topology settings. (b) G2Image performance under three topology settings. All results are presented as percentages.
Figure 2. Overview of TMTE, which jointly and evolutionarily optimizes the topology and modality toward the downstream task.
Table 1. Performance comparison on three graph-centric downstream tasks. The best, second best and third best results are highlighted in red , blue and orange , respectively.
Table 2. Performance comparison on three modality-centric downstream tasks. The best, second best and third best results are highlighted in red , blue and orange , respectively.
Table 3. Ablation study on 6 MAG datasets with graph-centric and modality-centric tasks.
Figure 3. Experimental results of our robustness analysis . We investigate two types of topological noise, including noise interactions (i.e., randomly adding edges), which are presented in (a) and (b); and missing interactions (i.e., randomly removing edges), which are presented in (c) and (d).
Figure 4. Accuracy curves under the MVSA dataset.
Method
E-Train. (s)
E-Infer. (s)
Param.
MMGCN
0.0891
0.0352
15.8 M
GraphGPT-O
1.8257
0.5167
48.7 M
Graph4MM
0.1599
0.5311
33.2 M
InstructG2I
24.9133
0.8211
62.3 M
DMGC
0.1362
0.0441
0.85 M
Table 4. Per-epoch efficiency on Movies.
Figure 5. Hyperparameter analysis for α and T on four datasets and tasks.
K
ACC
F1-Score
1
57.59±0.17
48.23±0.35
4
60.31±0.24
53.48±0.18
8
59.40±0.32
51.26±0.53
16
57.38±0.29
47.96±0.33
Table 5. Impact of K on Movies.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Table 6. Additional performance comparison on Grocery, Bili Dance and RedditS datasets. The best, second best and third best results are highlighted in red , blue and orange , respectively.
Datasets
# Modalities
# Nodes
# Edges
# Classes
Domain
Movies
Text, Image
16,672
218,390
20
Movie Network
Grocery
Text, Image
17,074
171,340
20
E-Commerce Network
DY
Text, Image
8,299
35,627
-
Video Network
Bili Dance
Text, Image
2,307
9,127
-
Video Network
Toys
Text, Image
20,695
126,886
18
E-Commerce Network
RedditS
Text, Image
15,894
566,160
20
Social Network
Appendix
Table 7. The statistical information of the experimental datasets.
Multimodal Attributed Graphs (MAGs) model real-world entities by coupling graph topology with heterogeneous attributes such as text and images. They support graph-centric tasks requiring structural and class-discriminative representations, and modality-centric tasks requiring fine-grained cross-modal correspondence. However, existing MAG methods often rely on fixed graph contexts or uniformly fused representations, causing task-agnostic propagation and over-compressed fusion that hinder diverse task requirements and modality-specific evidence preservation. To address this, we propose CoMAG, a unified MAG backbone that learns task-adaptive reliable contexts and modality-preserving alignment within them. CoMAG first conducts Reliable Context Learning by estimating edge reliability from multimodal semantic consistency, complementing raw topology with semantic neighbors, and selecting context components through a task-aware gate. It then performs Modality-preserving Hop-token Alignment by maintaining modality-specific multi-hop trajectories, matching modality-hop tokens across modalities, and decoupling shared and private representations. Thus, CoMAG produces graph and modality representations from one forward pass while retaining modality-specific cues. We further analyze stable propagation, over-smoothing mitigation, and modality-collapse control. Experiments on nine OpenMAG datasets compare CoMAG with feature-only, graph-only, multimodal, and unified MAG baselines across graph-level prediction, modality matching, and graph-conditioned generation. Results show that CoMAG achieves the best reported performance, demonstrating that task-adaptive reliable contexts and modality-preserving alignment improve structural prediction, cross-modal matching, and graph-conditioned generation while retaining sparse edge-linear complexity.
Multimodal graph learning requires jointly training over graph structure and heterogeneous node attributes, yet existing methods largely decouple these processes: prior multimodal graph neural networks (GNNs) focus on aligning modalities in a shared embedding space while operating on fixed or weakly adapted graph structures, and graph structure learning approaches infer topology from unimodal node representations without accounting for multimodal interactions. This separation fundamentally limits the ability of GNNs to capture semantically meaningful relationships in multimodal settings, where observed edges are often noisy, incomplete, or misaligned with underlying semantics. We propose ReCoG (Reciprocal Co-Evolution for Multimodal Graph Learning), a new learning paradigm that tightly couples graph structure learning and multimodal representation learning through end-to-end reciprocal interaction. Concretely, ReCoG integrates (i) a multimodal graph refiner that infers and corrects edges using cross-modal semantic evidence, and (ii) a coupled cross-modal message passing mechanism that performs joint intra- and inter-modality propagation over the refined graph. This unified design yields greater expressiveness than decoupled or two-stage formulations and allows dynamic interaction between topology and representation learning. Across diverse benchmarks for node classification and link prediction, ReCoG consistently outperforms strong multimodal graph structure learning baselines, including graph foundation models. Our results demonstrate that reciprocal co-evolution of structure and semantics is important for effective multimodal graph learning, challenging the prevailing separation between topology and representation learning.
Rui Xue, Tianfu Wu
Department of Electrical and Computer Engineering (ECE) North Carolina State University, Raleigh, NC 27695, USA
Multimodal graph foundation models (MGFMs) seek to learn generalizable representations from large-scale graphs with heterogeneous node modalities. However, real-world Multimodal-Attributed Graphs (MAGs) often contain incomplete node attributes, limiting the scale and diversity of available pretraining corpora. Besides, existing MGFMs primarily incorporate graph topology as structural context, overlooking its role in guiding multimodal binding and shaping a unified representation space. To address these challenges, we propose GraphBind, a topology-driven approach that uses graph topology to bind rich modality information into a unified shared space. GraphBind is motivated by the stability of graph topology, which provides structural references and complementary semantic information for multimodal binding. Concretely, GraphBind uses topology to organize self semantics and reliable neighborhood semantics into a global shared space that integrates structure and semantics, and adapts this space to discriminative and generative tasks through lightweight interfaces. Extensive experiments against 11 representative baselines demonstrate that GraphBind achieves leading performance on both discriminative and generative tasks, with relative improvements of up to 28.1% over the strongest baseline.
Xunkai Li, Chenxi Wan, Yinlin Zhu +4
Beijing Institute of Technology · Sun Yat-sen University