Multimodal Recommendation

Follow this topic Ask about this topic

Multimodal recommendation studies how to combine interaction histories with item content to predict what users will want, especially when behavior data is sparse or items are new. Common approaches include graph-based collaborative filtering, multimodal representation alignment and fusion, temporal and sequential models, and language-model-based semantic representations. Research also explores modality routing, generative recommendation, and production settings such as feeds, shopping, music, and place discovery. A central challenge is that modalities can be noisy, missing, or in conflict, and combining them does not always improve rankings; systems must balance content signals with collaborative evidence and remain robust under changing contexts.

Latest papers

48 papers5 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

User-Aware Conditional Generative Total Correlation Learning for Multi-Modal Recommendation
Apr 3, 2026Jing Du, Zesheng Ye, Congbo Ma +2Multimodal Recommendation Multimodal Fusion
The University of New South Wales, Sydney, Australia · University of Melbourne, Melbourne, Australia · New York University Abu Dhabi, Abu Dhabi, UAE
Agent4POI: Agentic Context-Conditioned Affordance Reasoning for Multimodal Point-of-Interest Recommendation
Apr 3, 2026Jinze Wang, Yangchen Zeng, Tiehua Zhang +5Recommender Systems Multimodal Recommendation
Tongji University · Swinburne University of Technology · Southeast University +4
TRU: Targeted Reverse Update for Efficient Multimodal Recommendation Unlearning
Apr 2, 2026Zhanting Zhou, KaHou Tam, Zeyu Ma +2Recommender Systems Multimodal Recommendation
University of Electronic Science and Technology of China Chengdu, China · University of Macau Macau, Macao
Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE
Mar 31, 2026Hejin Huang, Jusheng Zhang, Kaitong Cai +2Recommender Systems Direct Preference Optimization
Sun Yat-sen University · Snap Inc
TrendGen: An Outfit Recommendation and Display System
Mar 28, 2026Theodoros Koukopoulos, Dimos Klimenof, Ioannis XarchakosMultimodal Recommendation
SabinoDB, Athens, Greece · University of Toronto
VLM2Rec: Resolving Modality Collapse in Vision-Language Model Embedders for Multimodal Sequential Recommendation
Mar 18, 2026Junyoung Kim, Woojoo Kim, Wonbin Kweon +3Multimodal Recommendation Multimodal Embedding
Pohang University of Science and Technology · University of Illinois Urbana-Champaign
Anchored Alignment: Preventing Positional Collapse in Multimodal Recommender Systems
Mar 13, 2026Yonghun Jeong, David Yoon Suk Kang, Yeon-Chang LeeMultimodal Recommendation Cross-Modal Alignment
UNIST, Ulsan, Korea · Chungbuk National University, Cheongju, Korea
VLM4Rec: Multimodal Semantic Representation for Recommendation with Large Vision-Language Models
Mar 13, 2026Ty Valencia, Burak Barlas, Varun Singhal +2Multimodal Recommendation Multimodal Embedding
University of Southern California Los Angeles, USA
PinCLIP: Large-scale Foundational Multimodal Representation at Pinterest
Mar 3, 2026Josh Beal, Eric Kim, Jinfeng Rao +3Multimodal Embedding Multimodal IR
Pinterest Inc., San Francisco, CA, USA
Deep Interest Mining for Intent-Enriched Semantic IDs in Multimodal Generative Recommendation
Mar 3, 2026Yangchen Zeng, Jinze WangRecommender Systems Semantic ID Recommendation
MealRec: Multi-granularity Sequential Modeling via Hierarchical Diffusion Models for Micro-Video Recommendation
Mar 2, 2026Xinxin Dong, Haokai Ma, Yuze Zheng +3Recommender Systems Sequential Recommendation
National University of Defense Technology Changsha, China · National University of Singapore Singapore, Singapore
IDProxy: CTR Prediction with Multimodal LLMs for Cold-Start Recommendation at Xiaohongshu
Mar 2, 2026Yubin Zhang, Haiming Xu, Guillaume Salha-Galvan +6Cold-Start Recommendation Multimodal Embedding
Xiaohongshu Inc. Shanghai, China · Shanghai Jiao Tong University Shanghai, China · Fudan University Shanghai, China
GeMi: A Graph-based, Multimodal Recommendation System for Narrative Scroll Paintings
Mar 1, 2026Haimonti Dutta, Pruthvi Moluguri, Jin Dai +1Recommender Systems Multimodal Recommendation
Department of Management Science and Systems, The State University of New York, Jacobs Management Center, Buffalo, 14260, NY, USA. · Institute for Artificial Intelligence and Data Science, The State University of New York, Lockwood Hall, Buffalo, 14260, NY, USA. · Department of Computer Science and Engineering, The State University of New York, Davis Hall, Buffalo, 14260, NY, USA. +1
Stop Treating Collisions Equally: Qualification-Aware Semantic ID Learning for Recommendation at Industrial Scale
Feb 28, 2026Zheng Hu, Yuxin Chen, Yongsen Pan +13Semantic ID Recommendation Multimodal Recommendation
University of Electronic Science and Technology of China, Chengdu, China · Kuaishou Technology, Beijing, China
E-MMKGR: A Unified Multimodal Knowledge Graph Framework for E-commerce Applications
Feb 24, 2026Jiwoo Kang, Yeon-Chang LeeMultimodal KGs Multimodal Recommendation
UNIST Ulsan, Korea
Modality-Guided Mixture of Structured Experts with Entropy-Triggered Routing for Multimodal Recommendation
Feb 24, 2026Ji Dai, Quan Fang, DeSheng CaiSparse Mixture-of-Experts Recommender Systems
Beijing University of Posts and Telecommunications, China · Beijing University of Posts and Telecommunications, Beijing, China · Tianjin University of Technology, China
DReX: An Explainable Deep Learning-based Multimodal Recommendation Framework
Feb 23, 2026Adamya Shyam, Venkateswara Rao Kagita, Bharti Rana +1Gated Recurrent Unit Multimodal Recommendation
University of Delhi, Delhi, India · National Institute of Technology, Warangal, India
EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation
Aug 22, 2025Xiaoxiong Zhang, Xin Zhou, Zhiwei Zeng +2Cross-Modal Alignment Multimodal Embedding
College of Computing and Data Science, Nanyang Technological University, Singapore

Apr 3, 2026J/K move · Enter open · S save

User-Aware Conditional Generative Total Correlation Learning for Multi-Modal Recommendation

Jing Du, Zesheng Ye, Congbo Ma, Feng Liu+1

The University of New South Wales, Sydney, Australia · University of Melbourne, Melbourne, Australia · New York University Abu Dhabi, Abu Dhabi, UAE

Multimodal Recommendation Multimodal Fusion Cross-Modal Alignment User Preference Modeling

Open paper AskPDF arXiv

Multi-modal recommendation (MMR) enriches item representations by introducing item content, e.g., visual and textual descriptions, to improve upon interaction-only recommenders. The success of MMR hinges on aligning these content modalities with user preferences derived from interaction data, yet dominant practices based on disentangling modality-invariant preference-driving signals from modality-specific preference-irrelevant noises are flawed. First, they assume a one-size-fits-all relevance of item content to user preferences for all users, which contradicts the user-conditional fact of preferences. Second, they optimize pairwise contrastive losses separately toward cross-modal alignment, systematically ignoring higher-order dependencies inherent when multiple content modalities jointly influence user choices. In this paper, we introduce GTC, a conditional Generative Total Correlation learning framework. We employ an interaction-guided diffusion model to perform user-aware content feature filtering, preserving only personalized features relevant to each individual user. Furthermore, to capture complete cross-modal dependencies, we optimize a tractable lower bound of the total correlation of item representations across all modalities. Experiments on standard MMR benchmarks show GTC consistently outperforms state-of-the-art, with gains of up to 28.30% in NDCG@5. Ablation studies validate both conditional preference-driven feature filtering and total correlation optimization, confirming the ability of GTC to model user-conditional relationships in MMR tasks. The code is available at: https://github.com/jingdu-cs/GTC.