Organizations: Department of Computer Science and Networks, Higher School of Communication of Tunis, Tunis, Tunisia · LTCI, Télécom Paris, IP Paris, Palaiseau, France
Modern e-commerce platforms naturally provide three complementary modalities: textual item descriptions, item images, and user reviews. However, existing multimodal recommendation methods often omit reviews or do not integrate all three modalities within a unified framework. In this paper, we propose MPIUR, a multimodal graph-based recommendation model that encodes reviews in heterogeneous graphs, aligns image and text representations, and adapts collaborative--semantic alignment to item interaction density. Experiments on the Amazon Reviews 2023 Musical Instruments and Baby categories show that MPIUR achieves the best global nDCG@10 and Recall@10 among the evaluated methods on both datasets. It also obtains the best nDCG@10 and Recall@10 on sparse items, indicating that jointly exploiting reviews and multimodal item content is particularly useful when interaction histories are limited.
Figures & tables
Figure 1. Overview of the proposed architecture. The architecture processes item text and images with a Modal Encoder and reviews with HADSF. A Collaborative Graph and a Topic Graph then produce representations optimized by a joint loss system for top-k recommendation.
Musical Instruments
Baby
Users
17 318
25 474
Items
4 886
5 492
Interactions
58 469
70 205
Sparsity
99.93%
99.95%
Table 1. Dataset statistics.
Musical Instruments
Baby
Global
Sparse items
Popular items
Global
Sparse items
Popular items
Model
N@10
R@10
N@10
R@10
N@10
R@10
N@10
R@10
N@10
R@10
N@10
R@10
LightGCN
0.1613
0.2577
0.1142
0.1943
0.2217
0.3389
0.1636
0.2984
0.1230
0.2255
0.2013
0.3660
TGNN
0.2249
0.3990
0.2289
0.4152
0.2198
0.3782
0.1977
0.3538
0.1941
0.3498
0.2010
0.3575
BM3
0.2276
0.3955
0.2183
0.3870
0.2395
0.4065
0.2025
0.3859
0.2048
0.3976
0.2004
0.3751
MPIUR
0.2344
0.4042
0.2347
0.4153
0.2340
0.3900
0.2194
0.3891
0.2384
0.4161
0.2018
0.3640
Table 2. Performance comparison in terms of nDCG@10 and Recall@10 across all item groups.
Dataset
OFR
ADR
Musical Instruments
0.9358
0.0125
Baby
0.9375
0.0126
Table 3. Quality of the HADSF extraction.
Dataset
Recall@10
nDCG@10
Musical Instruments
0.0527
0.0314
Baby
0.0601
0.0343
Random (expected)
0.0019
00087
Table 4. Topic Graph performance on triplet ranking.
Variant
Global
Sparse Items
Popular Items
Full model
0.2344
0.2347
0.2340
w/o Modality Gap Loss
0.2135
0.2126
0.2147
w/o Fusion Loss
0.2150
0.2130
0.2176
w/o Image & Text Features
0.2177
0.2176
0.2178
w/o Review Features
0.1972
0.1869
0.2104
w/o All Modalities
0.1919
0.1814
0.2053
Table 5. Ablation study of the proposed model in terms of nDCG@10 on the “Musical Instruments” dataset.
Figure 2. t-SNE visualization of collaborative and modal embeddings before and after alignment on a sample of the Musical Instruments dataset. Two scatter plots compare collaborative and modal item embeddings. The two groups are separated before alignment and overlap substantially after alignment.
Figure 3. Sensitivity to the number of negative samples k on the Musical Instruments validation set. A line chart of validation nDCG at 10 as the number of negative samples varies from 1 to 64. The best value occurs at five negative samples.
Figure 4. nDCG@10 of the full model and the variant without modalities across item-degree bins on Musical Instruments. A line chart compares the full model with the variant without modalities across item interaction-degree bins. The full model performs better in every bin, with the largest gap in the 8–10 interaction bin.
Dataset
ResNet50
Qwen2.5-VL
LLaVA-13B
Musical Instruments
0.2344
0.2381
0.2316
Baby
0.2194
0.2113
0.2091
Table 6. Global nDCG@10 comparison between CNN-based and VLM-based visual encoding.