E-MMKGR: A Unified Multimodal Knowledge Graph Framework for E-commerce Applications

Jiwoo Kang; Yeon-Chang Lee

cs.IRFeb 24, 2026

E-MMKGR: A Unified Multimodal Knowledge Graph Framework for E-commerce Applications

Authors: Jiwoo Kang, Yeon-Chang Lee

Organizations: UNIST Ulsan, Korea

Abstract

Multimodal recommender systems (MMRSs) enhance collaborative filtering by leveraging item-side modalities, but their reliance on a fixed set of modalities and task-specific objectives limits both modality extensibility and task generalization. We propose E-MMKGR, a framework that constructs an e-commerce-specific Multimodal Knowledge Graph E-MMKG and learns unified item representations through GNN-based propagation and KG-oriented optimization. These representations provide a shared semantic foundation applicable to diverse tasks. Experiments on real-world Amazon datasets show improvements of up to 10.18% in Recall@10 for recommendation and up to 21.72% over vector-based retrieval for product search, demonstrating the effectiveness and extensibility of our approach.

Figures & tables

Figure 1. General MMKG and the proposed E-MMKG.

Figure 2. Overview of the \emmkgr framework.

Statistic	Office	Grocery	Pet	Toys	Beauty	Clothing
# User	4,905	14,681	19,856	19,412	22,363	39,387
# Item	2,420	8,713	8,510	11,924	12,101	23,033
# Interaction	53,258	151,254	157,836	167,597	198,502	278,677

Table 1. Dataset statistics

Variant	Recommendation	Search
Variant	Recommendation	Fine	Coarse
original	0.0611	0.5813	0.6839
w/ interaction	0.0595	0.5086	0.6296
w/ inter-modal	0.0601	0.5643	0.6805
w/ item-item	0.0580	0.5386	0.6538

Table 2. Ablation results of the proposed \emmkg on recommendation (R@10) and search (MAP@10) for Clothing

Method	Office	Grocery	Pet	Toys	Beauty	Clothing
LightGCN	0.0767	0.0922	0.0839	0.0845	0.0832	0.0361
GRCN	0.0856	0.1008	0.0976	0.0874	0.0918	0.0428
DualGNN	0.0850	0.1065	0.0916	0.0867	0.0917	0.0438
LATTICE	0.0920	0.1056	0.1042	0.0940	0.0940	0.0493
BM3	0.0706	0.0876	0.0907	0.0761	0.0865	0.0427
FREEDOM	0.0930	0.1197	0.1157	0.1080	0.1079	0.0621

Table 3. Recommendation comparison (R@10)

Metric	Fine-grained Query		Coarse-grained Query
Metric	Vector-based	E-MMKGR	Vector-based	E-MMKGR
R@5	0.6538	0.7048	0.6297	0.7665
R@10	0.7514	0.7789	0.7905	0.8759
MAP@5	0.4966	0.5713	0.4923	0.6423
MAP@10	0.5096	0.5813	0.5448	0.6839

Table 4. Search comparison for Clothing

Figure 3. t-SNE visualization of unified item representations for Clothing with K-means clustering

Multimodal KGs

Multimodal knowledge graphs connect entities and relations with text, images, video, time, and other evidence for structured reasoning.

See all 42 papers

MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search
Jul 13, 2026Zhen-Lin Chen, Maosen Sheng, Peng Lin +4Learning to Rank E-Commerce Search
JD.COM Beijing, China
EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation
Aug 22, 2025Xiaoxiong Zhang, Xin Zhou, Zhiwei Zeng +2Cross-Modal Alignment Multimodal Embedding
College of Computing and Data Science, Nanyang Technological University, Singapore
When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs
Apr 3, 2026Linyu Li, Zhi Jin, Yichi Zhang +5Multimodal KGs Continual Learning
Key Laboratory of High Confidence Software Technologies (PKU), Ministry of Education School of Computer Science, Peking University Beijing, China · School of Computer Science Zhejiang University Hangzhou, China · School of Cyber Science and Engineering Wuhan University Wuhan, China +1

Jul 13, 2026J/K move · Enter open · S save

MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search

Zhen-Lin Chen, Maosen Sheng, Peng Lin, Jianmin Chen+3

JD.COM Beijing, China

Learning to Rank E-Commerce Search Multimodal Large Language Models Multi-Task Learning Multimodal Embedding

Open paper AskPDF arXiv

Multimodal information is pivotal for e-commerce search ranking. Existing works leverage multimodal data typically by fine-tuning general Multimodal Large Language Models (MLLMs) via collaborative signals, subsequently integrating the derived representations into ranking models as item features. Despite their efficacy, these methods face two primary limitations: (1) they rely on a single collaborative signal for MLLM fine-tuning, failing to exploit the heterogeneous signals essential for multitask ranking; and (2) they treat multimodal representations as regular item features in ranking models, underutilizing their latent potential for user behavior modeling. To address these challenges, we propose the Multiplex Multimodal Representation Model (MMRM), a unified framework that aligns MLLMs with diverse collaborative signals. By employing a shared backbone with task-specific tokens and projection layers, MMRM simultaneously learns from multiple signals and generates comprehensive multiplex item representations in a single inference pass. Furthermore, we introduce a multiplex user representation strategy in ranking models, which derives task-specific user representations via search-based behavior sequence modeling leveraging multiplex item representations. Extensive experiments demonstrate MMRM's superior efficiency and effectiveness. Notably, MMRM has been successfully deployed in the JD e-commerce search engine, yielding significant performance gains for millions of daily users.

Abstract

Multimodal KGs

Explore similar work

MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search

EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation

When Modalities Remember: Continual Learning for Multimodal Knowledge Graphs