Organizations: Department of Computer Science and Networks, Higher School of Communication of Tunis, Tunis, Tunisia · LTCI, Télécom Paris, IP Paris, Palaiseau, France
Modern e-commerce platforms naturally provide three complementary modalities: textual item descriptions, item images, and user reviews. However, existing multimodal recommendation methods often omit reviews or do not integrate all three modalities within a unified framework. In this paper, we propose MPIUR, a multimodal graph-based recommendation model that encodes reviews in heterogeneous graphs, aligns image and text representations, and adapts collaborative--semantic alignment to item interaction density. Experiments on the Amazon Reviews 2023 Musical Instruments and Baby categories show that MPIUR achieves the best global nDCG@10 and Recall@10 among the evaluated methods on both datasets. It also obtains the best nDCG@10 and Recall@10 on sparse items, indicating that jointly exploiting reviews and multimodal item content is particularly useful when interaction histories are limited.
Figures & tables
Figure 1. Overview of the proposed architecture. The architecture processes item text and images with a Modal Encoder and reviews with HADSF. A Collaborative Graph and a Topic Graph then produce representations optimized by a joint loss system for top-k recommendation.
Musical Instruments
Baby
Users
17 318
25 474
Items
4 886
5 492
Interactions
58 469
70 205
Sparsity
99.93%
99.95%
Table 1. Dataset statistics.
Musical Instruments
Baby
Global
Sparse items
Popular items
Global
Sparse items
Popular items
Model
N@10
R@10
N@10
R@10
N@10
R@10
N@10
R@10
N@10
R@10
N@10
R@10
LightGCN
0.1613
0.2577
0.1142
0.1943
0.2217
0.3389
0.1636
0.2984
0.1230
0.2255
0.2013
0.3660
TGNN
0.2249
0.3990
0.2289
0.4152
0.2198
0.3782
0.1977
0.3538
0.1941
0.3498
0.2010
0.3575
BM3
0.2276
0.3955
0.2183
0.3870
0.2395
0.4065
0.2025
0.3859
0.2048
0.3976
0.2004
0.3751
MPIUR
0.2344
0.4042
0.2347
0.4153
0.2340
0.3900
0.2194
0.3891
0.2384
0.4161
0.2018
0.3640
Table 2. Performance comparison in terms of nDCG@10 and Recall@10 across all item groups.
Dataset
OFR
ADR
Musical Instruments
0.9358
0.0125
Baby
0.9375
0.0126
Table 3. Quality of the HADSF extraction.
Dataset
Recall@10
nDCG@10
Musical Instruments
0.0527
0.0314
Baby
0.0601
0.0343
Random (expected)
0.0019
00087
Table 4. Topic Graph performance on triplet ranking.
Variant
Global
Sparse Items
Popular Items
Full model
0.2344
0.2347
0.2340
w/o Modality Gap Loss
0.2135
0.2126
0.2147
w/o Fusion Loss
0.2150
0.2130
0.2176
w/o Image & Text Features
0.2177
0.2176
0.2178
w/o Review Features
0.1972
0.1869
0.2104
w/o All Modalities
0.1919
0.1814
0.2053
Table 5. Ablation study of the proposed model in terms of nDCG@10 on the “Musical Instruments” dataset.
Figure 2. t-SNE visualization of collaborative and modal embeddings before and after alignment on a sample of the Musical Instruments dataset. Two scatter plots compare collaborative and modal item embeddings. The two groups are separated before alignment and overlap substantially after alignment.
Figure 3. Sensitivity to the number of negative samples k on the Musical Instruments validation set. A line chart of validation nDCG at 10 as the number of negative samples varies from 1 to 64. The best value occurs at five negative samples.
Figure 4. nDCG@10 of the full model and the variant without modalities across item-degree bins on Musical Instruments. A line chart compares the full model with the variant without modalities across item interaction-degree bins. The full model performs better in every bin, with the largest gap in the 8–10 interaction bin.
Dataset
ResNet50
Qwen2.5-VL
LLaVA-13B
Musical Instruments
0.2344
0.2381
0.2316
Baby
0.2194
0.2113
0.2091
Table 6. Global nDCG@10 comparison between CNN-based and VLM-based visual encoding.
Multimodal Graph Neural Networks have become standard for recommendation by augmenting sparse interaction data with content features. Yet current architectures face two bottlenecks: structural rigidity, from a reliance on static precomputed similarity graphs that cannot adapt to evolving preferences; and semantic fragility, where noisy modality signals are indiscriminately fused, distorting the collaborative signal. We propose MURAL (Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning), a unified framework that shifts multimodal recommendation from fixed structural augmentation to dynamic topology discovery. To address structural rigidity, an Adaptive Edge Learner combines a differentiable retrieval-augmented strategy with an approximate nearest neighbor search to discover latent item-item correlations that are both semantically adaptive and computationally scalable (O(NlogN)). To address semantic fragility, an Uncertainty-Aware Fusion module models the aleatoric uncertainty of heterogeneous modalities, dynamically down-weighting unreliable features while prioritizing high-confidence signals as a defense against cross-modal noise. We further employ a contrastive teacher-student alignment that anchors modality-specific representations to stable behavioral signals, ensuring optimization stability without gradient leakage. Experiments on large-scale benchmarks including TikTok and Amazon show that MURAL significantly surpasses both structural and generative state-of-the-art baselines, achieving superior accuracy while offering interpretability through domain-specific modality dominance and robustness under extreme data corruption.
Ahmad Mousavi, Majid Alikhani, Yeon-Chang Lee +2
Department of Mathematics and Statistics American University · Independent Researcher Toronto, Ontario, Canada · Independent Researcher +4
Multimodal recommender systems (MMRSs) enhance collaborative filtering by leveraging item-side modalities, but their reliance on a fixed set of modalities and task-specific objectives limits both modality extensibility and task generalization. We propose E-MMKGR, a framework that constructs an e-commerce-specific Multimodal Knowledge Graph E-MMKG and learns unified item representations through GNN-based propagation and KG-oriented optimization. These representations provide a shared semantic foundation applicable to diverse tasks. Experiments on real-world Amazon datasets show improvements of up to 10.18% in Recall@10 for recommendation and up to 21.72% over vector-based retrieval for product search, demonstrating the effectiveness and extensibility of our approach.
MultiModal Recommendation (MMR) systems have emerged as a promising solution for improving recommendation quality by leveraging rich item-side modality information, prompting a surge of diverse methods. Despite these advances, existing methods still face two critical limitations. First, they use raw modality features to construct item-item links for enriching the behavior graph, while giving limited attention to balancing collaborative and modality-aware semantics or mitigating modality noise in the process. Second, they use a uniform alignment weight across all entities and also maintain a fixed alignment strength throughout training, limiting the effectiveness of modality-behavior alignment. To address these challenges, we propose EGRA. First, instead of relying on raw modality features, it alleviates sparsity by incorporating into the behavior graph an item-item graph built from representations generated by a pretrained MMR model. This enables the graph to capture both collaborative patterns and modality aware similarities with enhanced robustness against modality noise. Moreover, it introduces a novel bi-level dynamic alignment weighting mechanism to improve modality-behavior representation alignment, which dynamically assigns alignment strength across entities according to their alignment degree, while gradually increasing the overall alignment intensity throughout training. Extensive experiments on five datasets show that EGRA significantly outperforms recent methods, confirming its effectiveness.
Xiaoxiong Zhang, Xin Zhou, Zhiwei Zeng +2
College of Computing and Data Science, Nanyang Technological University, Singapore