Multimodal Graph Learning: A Survey of Models, Benchmarks, and Frontiers
Organizations: Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC.
Abstract
Multimodal attributed graphs (MAGs), in which nodes carry text, images, and other modalities while edges encode relations among entities, underlie e-commerce, social media, content platforms, and creative applications. Learning on MAGs has developed along two complementary lines: graph-centric methods redesign how modalities are fused, propagated, and connected within message passing, while foundation-model-centric methods pretrain transferable multimodal graph encoders or bring graph context into multimodal large language models for prediction and generation. In parallel, several benchmarks have appeared with different datasets and evaluation protocols. This survey takes an algorithm-centric and evaluation-grounded view of the field and makes three contributions. Modeling: we introduce a unified message-passing formulation and organize methods into five integration levels (feature, propagation, topology, learning, and foundation model), covering discriminative and graph-conditioned generative models; within this formulation we derive representative methods as special cases, compare their assumptions, expressiveness, and cost, and identify cross-modal sensitivity as a property that separates otherwise similar architectures. Datasets and evaluation: we review recent benchmarks, summarize their datasets and metrics, define model-independent descriptors (label homophily, average degree, clustering, and edge provenance), and map each evaluation protocol to the baselines it uses. Frontiers: we connect multimodal graph learning to AI agents, covering the multimodal graphs that agents perceive and remember and graph engineering, the practice of organizing agent workflows as explicit graphs; we then discuss open problems, concrete efforts for the community, and research directions, including when graph structure helps in the presence of strong multimodal foundation models.
Figures & tables
| Organizing principle | Coverage (method years) | Algorithm-level unified formulation | 2024–2026 MAG methods | Benchmark & protocol synthesis | FM / MLLM methods | |
| Ektefaie et al. (2023) | Blueprint + domain | 2022 | ✗ | ✗ | ✗ | |
| Peng et al. (2024) | Backbone operator + graph type | 2023 | ✗ | ✗ | ✗ | |
| Wang et al. (2025) | Desired MG-LLM capabilities | 2025 | ✓ | |||
| Ours | Integration level (5 levels) | 2026 | ✓ | ✓ | ✓ | ✓ |
| Method | Venue | Feat | Prop | Topo | Learn | FM | Graph-centric tasks | Modality-centric tasks |
| MMGCN ( Wei et al., 2019 ) | MM’19 | Rec | – | |||||
| MGAT ( Tao et al., 2020 ) | IPM’20 | Rec | – | |||||
| GRCN ( Wei et al., 2020 ) | MM’20 | Rec | – | |||||
| LATTICE ( Zhang et al., 2021 ) | MM’21 | Rec | – | |||||
| FREEDOM ( Zhou & Shen, 2023 ) | MM’23 | Rec | – | |||||
| BM3 ( Zhou et al., 2023 ) | WWW’23 | Rec | – |
| Design | Examples | Offline | Per epoch |
| Early fusion ( Section 4 ) | GCN, SAGE on fused features | – | |
| Modality-decoupled ( Section 5.1 ) | MMGCN, MGAT | – | |
| Coupled | ReCoG | – | |
| Precomputed coupled + hop attention | CAMPA |
| Method | Output | Generator | Context selection | Graph interface | Control |
| MMGL ( Yoon et al., 2023 ) | Text | Pretrained LM | Section and page neighbors | Neighbor content and structure encodings | — |
| Graph4MM ( Ning et al., 2025 ) | Text | Pretrained LM | Multi-hop graph neighbors | Hop-Diffused Attention and MM-QFormer | — |
| InstructG2I ( Jin et al., 2024 ) | Image | Diffusion model | PPR and semantic re-ranking | Graph-QFormer tokens | Graph CFG |
| GraphGPT-O ( Fang et al., 2025 ) | Text, image | MLLM with diffusion decoder | PPR-based sampling | Linearization or hierarchical aligner | Generation order |
| PLANET ( Liu et al., 2026b ) | Text, image | Existing pipelines | Inherited from pipeline | Topology-enriched node embeddings | Inherited |
| Platform | Venue | #Datasets | Domains / encoders | Main tasks | Compared paradigms |
| MM-GRAPH ( Zhu et al., 2025 ) | CVPR’25 | 7 | Products, books, MMKG; CLIP, ImageBind, T5, ViT, DINOv2 | NC, LP, KGC | MLP, GCN, SAGE, BUDDY; MMGCN, MGAT; MoSE, VISTA |
| MAGB ( Yan et al., 2025 ) | KDD’25 | 5 | E-commerce and social media; text, image, and VLM features | NC; zero-shot category prediction | GNN-as-Predictor; VLM-as-Predictor |
| GraphVLM ( Liu et al., 2026a ) | CVPR’26 | 6 | Amazon products and Reddit; CLIP-style and VLM features | NC, including cross-dataset transfer | VLM-as-Encoder / Aligner / Predictor |
| OpenMAG ( Wan et al., 2026 ) | ICML’26 | 19 | Six domains; 16 static/trainable encoders | NC, LP, Clust; MM, MR, MA, G2T, G2I | 24 GNN, MAG, and FM-enhanced models |
| MM-OpenFGL ( Li et al., 2026a ) | arXiv’26 | 19 | Seven domains; heterogeneous client modalities and topologies | Six downstream tasks (federated) | 57 methods; eight federation simulations |
| Dataset | Source / version | Domain | #Cls | Tasks | Metrics | ||
| Ele-fashion | MM-GRAPH | Products | 97,766 | 199,602 | 12 | NC | Acc, F1 |
| Goodreads-NC | MM-GRAPH | Books | 685,294 | 7,235,084 | 10 | NC | Acc, F1 |
| Amazon-Sports | MM-GRAPH | Products | 50,250 | 356,202 | – | LP | MRR, Hits@ |
| Amazon-Cloth | MM-GRAPH | Products | 125,839 | 951,271 | – | LP | MRR, Hits@ |
| Goodreads-LP | MM-GRAPH | Books | 636,502 | 3,437,017 | – | LP | MRR, Hits@ |
| MM-CoDEx-s | MM-GRAPH | MMKG | 1,383 | 15,884 | – | KGC | MRR, Hits@ |
| Dataset | Source | CC | Tr | ||
| Movies | MAGB | 0.4470 | 9.65 | 0.3442 | 0.2584 |
| Toys | MAGB | 0.7241 | 5.48 | 0.2398 | 0.1908 |
| Grocery | MAGB | 0.6813 | 8.33 | 0.3071 | 0.1445 |
| Reddit-S | MAGB | 0.9585 | 17.81 | 0.6993 | 1.0000 |
| Reddit-M | MAGB | 0.6716 | 5.86 | 0.5762 | 1.0000 |
| Ele-fashion | MM-GRAPH | 0.7675 | 4.08 | 0.1730 | 0.0560 |
| Setting | Datasets / task | Encoders and partitions | Baseline families | Representative use |
| MM-GRAPH | Ele-fashion, Goodreads-NC (NC); Sports, Cloth, Goodreads-LP (LP); MM-CoDEx-s/m (KGC) | CLIP, ImageBind, T5+ViT, T5+DINOv2; NC 60/10/30; Amazon LP 80/10/10; Goodreads LP 60/10/30 | Feature-only MLP; GCN, SAGE, BUDDY (LP); MMGCN, MGAT; MoSE, VISTA (KGC) | MM-GRAPH ( Zhu et al., 2025 ) ; ReCoG ( Xue & Wu, 2026 ) |
| MAGB | Movies, Toys, Grocery, Reddit-S, Reddit-M (NC) | Text, image, and VLM embeddings; released splits | MLP and conventional GNNs; zero-shot VLM predictor | MAGB ( Yan et al., 2025 ) ; NSG-MoE ( Zhang & Kuruoglu, 2026 ) ; ReCoG ( Xue & Wu, 2026 ) |
| OpenMAG | 19 datasets; 3 graph and 5 modality tasks | 16 frozen/trainable encoders; 60/20/20 | MLP; GNNs; graph-enhanced, multimodal-enhanced, and FM-enhanced methods | OpenMAG ( Wan et al., 2026 ) |
| GraphVLM | Movies, Toys, Grocery, Arts, CDs, Reddit (NC) | CLIP-style and VLM encoders; 60/20/20; zero-shot or fine-tuned | GNNs on VLM features; graph-LLM aligners; VLM predictors | GraphVLM ( Liu et al., 2026a ) |
| ReCoG (method-specific) | MM-GRAPH NC/LP plus MAGB NC (nine datasets) | Four MM-GRAPH and three MAGB encoder configurations | MLP, GCN, SAGE, MMGCN, MGAT, NSG-MoE, LION, PLANET, UniGraph2 | ReCoG ( Xue & Wu, 2026 ) |
| MGL principle | Potential connection to agent systems |
| Fusion timing | Independent agents followed by aggregation (late fusion) versus intermediate information exchange during reasoning. |
| Cross-modal coupling | Evidence from one channel can influence the processing of other channels before the final decision. |
| Graph-aware alignment | Grounding different channels in shared entities, evidence, or subgoals to maintain semantic consistency. |
| Role-aware propagation | Routing information according to source reliability, complementarity, or verification needs ( Zuo et al., 2026b ) . |
| Shared–private learning | Preserving independent evidence while extracting common conclusions across agents. |
| Over-smoothing and residuals | Avoiding premature consensus by retaining independent initial reasoning across communication rounds. |
Multimodal Graph Learning
Multimodal Graph Learning combines graph structure with text, images, signals, and other modalities for prediction, retrieval, and reasoning.
See all 119 papers