ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learning

Rui Xue; Tianfu Wu

cs.LGAug 24, 2026

ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learning

Authors: Rui Xue, Tianfu Wu

Organizations: Department of Electrical and Computer Engineering (ECE) North Carolina State University, Raleigh, NC 27695, USA

Abstract

Multimodal graph learning requires jointly training over graph structure and heterogeneous node attributes, yet existing methods largely decouple these processes: prior multimodal graph neural networks (GNNs) focus on aligning modalities in a shared embedding space while operating on fixed or weakly adapted graph structures, and graph structure learning approaches infer topology from unimodal node representations without accounting for multimodal interactions. This separation fundamentally limits the ability of GNNs to capture semantically meaningful relationships in multimodal settings, where observed edges are often noisy, incomplete, or misaligned with underlying semantics. We propose ReCoG (Reciprocal Co-Evolution for Multimodal Graph Learning), a new learning paradigm that tightly couples graph structure learning and multimodal representation learning through end-to-end reciprocal interaction. Concretely, ReCoG integrates (i) a multimodal graph refiner that infers and corrects edges using cross-modal semantic evidence, and (ii) a coupled cross-modal message passing mechanism that performs joint intra- and inter-modality propagation over the refined graph. This unified design yields greater expressiveness than decoupled or two-stage formulations and allows dynamic interaction between topology and representation learning. Across diverse benchmarks for node classification and link prediction, ReCoG consistently outperforms strong multimodal graph structure learning baselines, including graph foundation models. Our results demonstrate that reciprocal co-evolution of structure and semantics is important for effective multimodal graph learning, challenging the prevailing separation between topology and representation learning.

Figures & tables

Figure 1: Overview of the proposed ReCoG framework. Left: The input multimodal graph with text and vision features. Center-top: The Structure Refiner module with two variants: a similarity-based version, and a residual MLP-based version. Center-bottom: Cross-modal coupled message passing over $L$ layers, where each layer performs intra-modal neighbor aggregation and cross-modal injection. Right: Modality fusion followed by a task-specific head. The entire pipeline is trained end-to-end. Best viewed in magnification. See text for detail.

Figure 2: Topology–semantics alignment before and after refinement. Left : distribution of multimodal cosine similarity for connected and disconnected node pairs in the original graph (RedditM). Right : similarity distribution of edges added by the cosine refiner compared to disconnected pairs.

	Ele-fashion				Goodreads-NC
Model	CLIP	T5+ViT	IBind	T5+DINO	CLIP	T5+ViT	IBind	T5+DINO
MMGCN	86.10 $\pm$ 0.50	82.39 $\pm$ 0.30	86.21 $\pm$ 0.94	85.53 $\pm$ 0.33	83.29 $\pm$ 0.20	81.85 $\pm$ 0.22	80.58 $\pm$ 1.08	82.44 $\pm$ 0.11
MGAT	84.66 $\pm$ 0.29	84.01 $\pm$ 0.08	86.12 $\pm$ 0.08	84.54 $\pm$ 0.27	76.48 $\pm$ 0.59	75.43 $\pm$ 0.76	69.45 $\pm$ 6.25	74.98 $\pm$ 1.23
GCN	79.83 $\pm$ 0.03	79.63 $\pm$ 0.07	80.35 $\pm$ 0.02	79.37 $\pm$ 0.04	81.61 $\pm$ 0.01	81.67 $\pm$ 0.03	78.91 $\pm$ 0.04	81.71 $\pm$ 0.03
SAGE	87.10 $\pm$ 0.02	84.41 $\pm$ 0.09	87.71 $\pm$ 0.13	85.31 $\pm$ 0.09	83.30 $\pm$ 0.02	83.03 $\pm$ 0.04	80.39 $\pm$ 0.21	82.99 $\pm$ 0.08
MLP	85.16 $\pm$ 0.03	84.98 $\pm$ 0.05	88.73 $\pm$ 0.01	84.87 $\pm$ 0.01	72.29 $\pm$ 0.02	67.82 $\pm$ 0.07	58.75 $\pm$ 0.05	68.83 $\pm$ 0.03

Table 1: Node classification accuracy (%) on the MM-Graph [ 38 ] . Best and runner-up are highlighted.

		Movies			Toys			Grocery			Reddit-S			Reddit-M
	Model	L+C	QWen	LLaVL	L+C	QWen	LLaVL	L+C	QWen	LLaVL	L+C	QWen	LLaVL	L+C	QWen	LLaVL
	GCN	53.98	55.19	54.92	81.59	80.95	81.57	84.64	82.96	84.17	89.77	90.88	91.13	71.42	72.96	75.64
	SAGE	56.08	58.01	57.39	82.35	81.45	82.35	86.65	85.55	86.81	91.08	92.57	92.94	80.84	81.23	85.61
sim	ReCoG-GCN	56.97	57.72	57.51	81.98	81.20	82.19	85.89	83.46	85.03	97.20	96.98	97.26	90.55	90.24	90.87
sim	ReCoG-SAGE	56.94	58.92	60.00	82.58	82.27	82.75	86.86	86.24	89.05	96.76	96.38	96.73	88.75	89.18	90.11
res	ReCoG-GCN	57.45	60.12	58.62	82.07	82.27	82.27	86.18	86.39	86.56	96.89	96.82	96.73	87.28	87.31	87.42

Table 2: Node classification accuracy (%) on the MAGB datasets [ 34 ] .

	Node Classification (Acc. %)							Link Pred. (MRR %)
Model	Fashion	GoodReads	Movies	Toys	Grocery	R-S	R-M	Sports	Cloth
NSG-MoE	88.42	—	53.98	80.92	86.28	95.93	85.03	34.39	24.10
LION	—	78.54	58.61	—	88.30	94.80	—	—	—
PLANET	87.37	84.16	57.06	81.22	85.16	96.62	—	27.51	20.25
UniGraph2	87.91	81.97	*52.92	*79.49	*85.24	*93.50	*70.19	31.61	25.01
ReCoG-sim	89.31	90.51	60.00	82.75	89.05	97.26	90.87	38.53	28.30

Table 3: Comparison with recent multimodal graph methods. R-S = Reddit-S, R-M = Reddit-M.

	Configuration	Fashion (Clip)	Movies (L+C)	Reddit-S (L+C)
	MMGCN † (w/o both)	86.10	54.18	91.13
sim	w/o Cross-Modal	86.83	55.16	94.28
	w/o Refiner	87.09	56.19	95.97
	Full Model	88.56	56.97	97.20
res	w/o Cross-Modal	86.41	55.74	92.74
	w/o Refiner	87.09	56.19	95.97

Table 4: Ablation study on node classification accuracy (%).

\dagger

MMGCN uses separate GCNs for each modality with late fusion.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	Encoder	Amazon-Sports			Amazon-Cloth
		MRR	H@1	H@10	MRR	H@1	H@10
MMGCN	CLIP	31.96 $\pm$ 0.10	16.35 $\pm$ 0.11	68.46 $\pm$ 0.08	22.20 $\pm$ 0.05	10.76 $\pm$ 0.10	46.62 $\pm$ 0.12
	ViT+T5	30.33 $\pm$ 0.08	15.01 $\pm$ 0.05	66.41 $\pm$ 0.11	19.45 $\pm$ 0.34	9.22 $\pm$ 0.20	40.49 $\pm$ 0.61
	ImageBind	31.74 $\pm$ 0.21	16.45 $\pm$ 0.13	67.39 $\pm$ 0.74	24.72 $\pm$ 0.19	12.47 $\pm$ 0.09	51.32 $\pm$ 0.56
	DINO+T5	30.04 $\pm$ 0.27	14.98 $\pm$ 0.07	64.56 $\pm$ 0.56	21.77 $\pm$ 0.23	10.47 $\pm$ 0.12	45.81 $\pm$ 0.52
MGAT	CLIP	27.56 $\pm$ 0.30	13.55 $\pm$ 0.29	60.21 $\pm$ 0.21	21.38 $\pm$ 0.23	10.39 $\pm$ 0.22	44.60 $\pm$ 0.36

Appendix

Table 5: Link prediction performance on Amazon-Sports and Amazon-Cloth. [ 38 ] . The Best and runner-up are highlighted.

Refiner	$K$	Node Classification (%)		Link Prediction (MRR %)
Refiner	$K$	Ele-Fashion	Goodreads	Amazon-Sports	Amazon-Clothing
sim	1	87.60 $\pm$ 0.05	89.65 $\pm$ 0.05	36.72 $\pm$ 0.11	27.07 $\pm$ 0.10
	3	88.17 $\pm$ 0.03	90.43 $\pm$ 0.09	37.81 $\pm$ 0.06	27.93 $\pm$ 0.07
	5	88.56 $\pm$ 0.05	90.31 $\pm$ 0.05	38.53 $\pm$ 0.03	28.30 $\pm$ 0.03
	8	88.53 $\pm$ 0.05	90.16 $\pm$ 0.06	38.32 $\pm$ 0.09	28.01 $\pm$ 0.06
	12	87.95 $\pm$ 0.06	89.43 $\pm$ 0.10	37.23 $\pm$ 0.10	26.68 $\pm$ 0.05

Appendix

Table 6: Ablation study on the number of candidate neighbors

K

in the graph refiner.

Component	Complexity
Cosine similarity ( $S_{T}$ , $S_{V}$ )	$O(BNd)$
Original edge scoring (MLP)	$O(E\cdot d\cdot d_{h})$
Coarse filtering	$O(BNd_{h})$
Fine scoring (MLP)	$O(Brk\cdot d\cdot d_{h})$
Cross-modal message passing ( $\times L$ )	$O(L(E\cdot d_{h}+N\cdot d_{h}^{2}))$

Appendix

Table 7: Per-batch computational complexity of ReCoG major components.

Dataset	#Nodes	#Edges	#Classes	Task
Ele-fashion ‡	97,766	199,602	12	NC
Goodreads-NC ‡	685,294	7,235,084	10	NC
Movies †	16,672	218,390	20	NC
Toys †	20,695	126,886	18	NC
Grocery †	17,074	171,340	20	NC
Reddit-S †	15,894	566,160	20	NC

Appendix

Table 8: Dataset statistics. Datasets marked with

\dagger

are from the MAGB benchmark [ 34 ] ; those marked with

\ddagger

are from MM-Graph [ 38 ] . NC = node classification, LP = link prediction.

Explore similar work

TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution
Mar 29, 2026Yinlin Zhu, Xunkai Li, Di Wu +3Multimodal Graph Learning Graph Structure Learning
Sun Yat-sen University, Guangzhou, China · Beijing Institute of Technology, Beijing, China
Multimodal Graph Representation Learning with Dynamic Information Pathways
Mar 10, 2026Xiaobin Hong, Mingkai Lin, Xiaoli Wang +2Multimodal Graph Learning Multimodal Embedding
State Key Laboratory for Novel Software Technology, Nanjing University · Nanjing Forest University, College of Information Science and Technology · Zhejiang Provincial Seaport Investment & Operation Group Co. Ltd
CAMPA: Efficient and Aligned Multimodal Graph Learning via Decoupled Propagation and Aggregation
May 12, 2026Daohan Su, Hao Liu, Xunkai Li +6Efficient Multimodal Inference Cross-Modal Alignment

Mar 29, 2026J/K move · Enter open · S save

TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution

Yinlin Zhu, Xunkai Li, Di Wu, Wang Luo+2

Sun Yat-sen University, Guangzhou, China · Beijing Institute of Technology, Beijing, China

Multimodal Graph Learning Graph Structure Learning Cross-Modal Representation Learning Multimodal Fusion

Open paper AskPDF arXiv

Multimodal-attributed graphs (MAGs) are a fundamental data structure for multimodal graph learning (MGL), enabling both graph-centric and modality-centric tasks. However, our empirical analysis reveals inherent topology quality limitations in real-world MAGs, including noisy interactions, missing connections, and task-agnostic relational structures. A single graph derived from generic relationships is therefore unlikely to be universally optimal for diverse downstream tasks. To address this challenge, we propose Task-aware Modality and Topology co-Evolution (TMTE), a novel MGL framework that jointly and iteratively optimizes graph topology and multimodal representations toward the target task. TMTE is motivated by the bidirectional coupling between modality and topology: multimodal attributes induce relational structures, while graph topology shapes modality representations. Concretely, TMTE casts topology evolution as multi-perspective metric learning over modality embeddings with an anchor-based approximation, and formulates modality evolution as smoothness-regularized fusion with cross-modal alignment, yielding a closed-loop task-aware co-evolution process. Extensive experiments on 9 MAG datasets and 1 non-graph multimodal dataset across 6 graph-centric and modality-centric tasks show that TMTE consistently achieves state-of-the-art performance. Our code is available at https://anonymous.4open.science/r/TMTE-1873.

Abstract

Explore similar work

TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution

Multimodal Graph Representation Learning with Dynamic Information Pathways

CAMPA: Efficient and Aligned Multimodal Graph Learning via Decoupled Propagation and Aggregation