RoleMAG: Learning Neighbor Roles in Multimodal Graphs

Yilong Zuo; Xunkai Li; Zhihan Zhang; Ronghua Li; Guoren Wang

cs.LGApr 14, 2026

RoleMAG: Learning Neighbor Roles in Multimodal Graphs

Authors: Yilong Zuo, Xunkai Li, Zhihan Zhang, Ronghua Li, Guoren Wang

Organizations: Beijing Institute of Technology, Beijing, China

Abstract

Multimodal attributed graphs (MAGs) combine multimodal node attributes with structured relations. However, existing methods usually perform shared message passing on a single graph and implicitly assume that the same neighbors are equally useful for all modalities. In practice, neighbors that benefit one modality may interfere with another, blurring modality-specific signals under shared propagation. To address this issue, we propose RoleMAG, a multimodal graph framework that learns how different neighbors should participate in propagation. Concretely, RoleMAG distinguishes whether a neighbor should provide shared, complementary, or heterophilous signals, and routes them through separate propagation channels. This enables cross-modal completion from complementary neighbors while keeping heterophilous ones out of shared smoothing. Extensive experiments on three graph-centric MAG benchmarks show that RoleMAG achieves the best results on RedditS and Bili_Dance, while remaining competitive on Toys. Ablation, robustness, and efficiency analyses further support the effectiveness of the proposed role-aware propagation design. Our code is available at https://anonymous.4open.science/r/RoleMAG-7EE0/

Figures & tables

Figure 1. Illustration of the key motivation behind RoleMAG. Existing multimodal graph methods usually adopt a unified neighbor propagation pattern, implicitly treating all neighbors as if they contributed in a similar way. In practice, however, neighbors may play different roles in multimodal propagation: some provide shared support, some serve as complementary neighbors that enable cross-modal completion, while others correspond to heterophilous relations that should not be mixed into shared smoothing. This motivates a fine-grained, role-aware view of neighborhood propagation.

Panel of Figure 2 — Figure 2. Empirical observations behind RoleMAG. (a) Neighbor utility is modality-dependent: text and image branches prefer different neighborhood organizations. (b) When text representations are degraded, complementary completion is most effective along the $I\!\rightarrow\!T$ direction. (c) When image representations are degraded, the preferred direction reverses to $T\!\rightarrow\!I$ . (d) As heterophily increases, directly mixing all neighbors into shared smoothing becomes less reliable, whereas a dedicated heterophily-aware propagator remains more stable.

Figure 3. Framework of RoleMAG. RoleMAG performs role-aware multimodal propagation in three stages. First, an edge role router infers edge roles from semantic cues and uses structural features to calibrate routing strength. Second, the routed neighbors are sent to three specialized experts, including a shared expert for consistent smoothing, a directional complementary expert for cross-modal completion, and a heterophily expert for discrepant neighborhood interactions. Finally, the expert outputs are integrated by residual gating fusion and used for downstream tasks such as node classification, link prediction, and retrieval.

Model	Toys (Node Classification)		RedditS (Node Classification)		Bili_Dance (Link Prediction)
Model	ACC	F1	ACC	F1	MRR	Hits@3
GCN	77.24 ±0.31	73.18 ±0.42	91.48 ±0.24	84.68 ±0.38	14.25 ±0.41	15.23 ±0.57
GAT	77.29 ±0.34	72.92 ±0.49	91.45 ±0.27	85.65 ±0.41	24.19 ±0.88	25.42 ±1.06
MMGCN	77.60 ±0.29	75.03 ±0.45	90.44 ±0.36	82.85 ±0.56	8.64 ±0.33	8.55 ±0.39
MGAT	78.12 ±0.42	74.21 ±0.61	91.73 ±0.31	85.12 ±0.48	16.84 ±0.51	18.92 ±0.72
LGMRec	78.81 ±0.73	72.31 ±1.14	92.36 ±0.19	86.10 ±0.32	11.46 ±0.37	11.78 ±0.44

Table 1. Main results on graph-centric tasks. All metrics are reported as percentages. The best, second best, and third best results are highlighted in dark red, royal blue, and orange, respectively.

Variant	Toys ACC	RedditS ACC
Shared-only	75.23	89.73
w/o Role Routing	77.23	90.85
w/o Complementary Expert	77.85	91.44
w/o Complementary Direction	77.93	91.62
w/o Heterophily Expert	76.97	90.44
Full RoleMAG	78.83	92.57

Table 2. Ablation study of RoleMAG on node classification. All metrics are reported as percentages.

Panel of Figure 4 — Figure 4. Robustness and efficiency analysis.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning	Shape / Range
Input graph and node representations
$\mathcal{G}=(\mathcal{V},\mathcal{E},\{\mathbf{X}^{(m)}\}_{m\in\mathcal{M}})$	Multimodal attributed graph	–
$\mathcal{M}=\{T,I\}$	Set of modality indices, representing text and image, respectively	$\|\mathcal{M}\|=2$
$\mathbf{x}_{i}^{(m)},\mathbf{X}^{(m)}$	Modal input features of node $v_{i}$ and the modal feature matrix of the entire graph	$\mathbb{R}^{d_{m}}$ , $\mathbb{R}^{N\times d_{m}}$
$\mathbf{A},\mathbf{D},\tilde{\mathbf{A}}$	Adjacency matrix, degree matrix, and symmetric normalized adjacency matrix	$\mathbb{R}^{N\times N}$
$\mathcal{N}(i)$	First-order neighborhood of node $v_{i}$	$\subseteq\mathcal{V}$

Appendix

Table 3. Notation used in Appendix B. The notations above are used consistently throughout this appendix. For detailed definitions of each variable, please refer to sections B.1 , B.2 , B.3 , B.4 , B.5 and B.6 .

Dataset	Task	#Nodes	#Edges	#Classes	Image Dim.	Text Dim.
Toys	Node Classification	20,695	63,443	18	768	768
RedditS	Node Classification	15,894	283,080	20	768	768
Bili_Dance	Link Prediction	2,307	9,127	–	768	768

Appendix

Table 4. Datasets used in the main-paper graph-centric experiments. Bili_Dance is used for link prediction and therefore does not involve node classes.

Component	Symbol / Hyperparameter	Value Used in Current Experiments
Node representation dimension	$d$	$256$
Dropout	$p$	$0.2$
Router hidden dimension	$d_{\mathrm{router}}$	$64$
Complementary TopK size	$K$	$16$
Number of query tokens	$Q$	$4$
Attention bias scale	$\lambda_{\mathrm{bias}}$	$1.0$

Appendix

Table 5. Key hyperparameters of RoleMAG used in the main experiments.

Item	Specification
Operating system	Ubuntu 22.04.5 LTS
Python	3.10.18
PyTorch	2.8.0
CUDA runtime	12.8
Graph libraries	PyG 2.6.1, DGL 2.4.0
GPU	NVIDIA RTX PRO 6000 Blackwell Server Edition

Appendix

Table 6. Main software and hardware environment used in the experiments.

Dataset	RoleMAG		DGF		LGMRec		Graph4MM
	Acc	F1	Acc	F1	Acc	F1	Acc	F1
Toys	78.83	75.49	77.68	71.46	78.81	72.31	78.91	75.62
RedditS	92.57	87.04	91.73	82.63	92.36	86.10	92.34	86.31
Grocery	83.45	73.56	83.02	72.56	84.12	76.83	83.57	75.92
Bili_Dance	44.82	39.14	41.73	36.21	43.90	37.88	45.21	39.87

Appendix

Table 7. Additional node classification results on four MAG datasets. All metrics are reported in %.

Sweep	Value	Acc	F1
TopK	8	93.49	87.10
TopK	16	92.92	86.25
TopK	32	93.14	86.68
Queries	2	93.81	87.27
Queries	4	92.64	85.75
Queries	8	93.14	86.60

Appendix

Table 8. Hyperparameter sensitivity of structural interaction parameters on RedditS node classification. All metrics are reported in %.

Sweep	Value	Acc	F1
$\lambda_{\mathrm{qca}}$	0.0	92.74	85.79
$\lambda_{\mathrm{qca}}$	0.2	92.70	86.20
$\lambda_{\mathrm{qca}}$	0.5	93.21	86.07
$\lambda_{\mathrm{qca}}$	1.0	92.67	86.70
$\lambda_{\mathrm{bal}}$	0.0	92.61	85.56
$\lambda_{\mathrm{bal}}$	0.05	93.11	86.69

Appendix

Table 9. Hyperparameter sensitivity of auxiliary objective weights on RedditS node classification. All metrics are reported in %.

Reproducibility Item	Specification	Reference
Execution entrypoint	The graph-centric experiments are launched through python src/main.py , where the three fields task , dataset , and model jointly determine a run configuration.	sections D.6 and E.2
Environment specification	The dependency set is frozen by the environment file at the repository root, and the main software and hardware environment is listed in the environment table.	sections D.6 and 6
Task scope	The main paper covers two node classification tasks (Toys and RedditS) and one link prediction task (Bili_Dance).	sections D.1 and E.1
Data split protocol	Node classification uses a $60/20/20$ node split, while link prediction uses a $70/10/20$ edge split and is evaluated with ranking metrics.	sections D.1 and D.3
Input construction	Node inputs are constructed from frozen multimodal embeddings together with task-specific graph preprocessing.	section D.2
Model instantiation	RoleMAG is integrated into the training and evaluation workflow through the unified benchmark interface and instantiated along the router-expert-fusion pipeline specified in this paper.	sections D.4 and E.4

Appendix

Table 10. RoleMAG supplementary reproducibility checklist. The listed items specify the execution interface, configuration conventions, and output fields underlying the graph-centric experiments in the main paper.

Explore similar work

Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs
Jun 12, 2026Sirui Zhang, Xu Wang, Zhengyu Wu +2Cross-Modal Alignment Multimodal Graph Learning
TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution
Mar 29, 2026Yinlin Zhu, Xunkai Li, Di Wu +3Multimodal Graph Learning Graph Structure Learning
Sun Yat-sen University, Guangzhou, China · Beijing Institute of Technology, Beijing, China
CAMPA: Efficient and Aligned Multimodal Graph Learning via Decoupled Propagation and Aggregation
May 12, 2026Daohan Su, Hao Liu, Xunkai Li +6Efficient Multimodal Inference Cross-Modal Alignment

Jun 12, 2026J/K move · Enter open · S save

Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs

Sirui Zhang, Xu Wang, Zhengyu Wu, Xunkai Li+1

Cross-Modal Alignment Multimodal Graph Learning

Open paper AskPDF arXiv

Multimodal Attributed Graphs (MAGs) model real-world entities by coupling graph topology with heterogeneous attributes such as text and images. They support graph-centric tasks requiring structural and class-discriminative representations, and modality-centric tasks requiring fine-grained cross-modal correspondence. However, existing MAG methods often rely on fixed graph contexts or uniformly fused representations, causing task-agnostic propagation and over-compressed fusion that hinder diverse task requirements and modality-specific evidence preservation. To address this, we propose CoMAG, a unified MAG backbone that learns task-adaptive reliable contexts and modality-preserving alignment within them. CoMAG first conducts Reliable Context Learning by estimating edge reliability from multimodal semantic consistency, complementing raw topology with semantic neighbors, and selecting context components through a task-aware gate. It then performs Modality-preserving Hop-token Alignment by maintaining modality-specific multi-hop trajectories, matching modality-hop tokens across modalities, and decoupling shared and private representations. Thus, CoMAG produces graph and modality representations from one forward pass while retaining modality-specific cues. We further analyze stable propagation, over-smoothing mitigation, and modality-collapse control. Experiments on nine OpenMAG datasets compare CoMAG with feature-only, graph-only, multimodal, and unified MAG baselines across graph-level prediction, modality matching, and graph-conditioned generation. Results show that CoMAG achieves the best reported performance, demonstrating that task-adaptive reliable contexts and modality-preserving alignment improve structural prediction, cross-modal matching, and graph-conditioned generation while retaining sparse edge-linear complexity.

Abstract

Explore similar work

Context-aware Modality-Topology Co-Alignment for Multimodal Attributed Graphs

TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution

CAMPA: Efficient and Aligned Multimodal Graph Learning via Decoupled Propagation and Aggregation