Multimodal Decoupling for Diversified Recommendation through Semantic and Structural Shaping

Changran He; Jiaming Wei; Jianghong Ma; Tianjun Wei; Haijun Zhang; Xiaofeng Zhang

cs.IROct 19, 2026

Multimodal Decoupling for Diversified Recommendation through Semantic and Structural Shaping

Authors: Changran He, Jiaming Wei, Jianghong Ma, Tianjun Wei, Haijun Zhang, Xiaofeng Zhang

Organizations: Harbin Institute of Technology(Shenzhen) Shenzhen, China · Shenzhen Key Laboratory of Internet Information Collaboration Shenzhen, China · Shenzhen Research Institute, City University of Hong Kong Shenzhen, China · Nanyang Technological University Singapore, Singapore

Abstract

Recommender systems often suffer from over-concentrated results when optimized for accuracy, limiting recommendation diversity. While diversity has become an important objective, existing methods still face fundamental limitations from both semantic and structural perspectives. On the one hand, multimodal recommender systems provide rich semantic signals, yet are often treated as auxiliary to ID embeddings, leading to ineffective semantic utilization for diversity. On the other hand, the accuracy--diversity trade-off persists during training, where accuracy gains often come at the cost of diversity. We show that this phenomenon is fundamentally governed by both the structural properties of learned representations and the dynamics of optimization. To address these challenges, we propose MOSS, a unified framework for multimodal decoupling through semantic and structural shaping. Specifically, we first introduce an MLLM-based Decoupled Feature Extraction module to disentangle multimodal content into informative representations. To enhance semantic utilization, we design a Category-aware Embedding Selection module that extracts compact, category-aware semantic features. To address structural limitations, we propose two complementary modules: Alternating Dual-Graph Propagation, which alleviates representation collapse by integrating collaborative and multimodal propagation, and Entropy-based Diversity Regularization, which mitigates optimization bias by promoting balanced category exposure. Extensive experiments on real-world datasets demonstrate that MOSS consistently achieves a superior accuracy--diversity trade-off over strong baselines.

Figures & tables

Figure 1. Training dynamics and the accuracy–diversity trade-off of LightGCN on Anime, with Recall@10 for accuracy and Coverage@10 for diversity. The left figure shows Recall@10 and Coverage@10 over the first 50 epochs, and the right figure shows the corresponding Coverage@10–Recall@10 trajectory, colored by epoch and labeled every five epochs.

Features	Norm. Eff.Rank	R@10	C@10
ID embedding	0.99	0.1924	0.4074
Visual embedding	0.65	0.1710	0.4032
Textual embedding	0.68	0.1698	0.4096

Table 1. Effect of different item initialization strategies on representation quality and recommendation performance on Anime dataset.

Figure 2. Category-wise density visualization of item embeddings for four representative genres on Anime dataset. Embeddings are projected into 2D by t-SNE and normalized onto the unit circle, with filled contours represent density.

Figure 3. Category-aware Embedding Selection.

Figure 4. An illustration of the proposed MOSS model. The MLLM-based Decoupled Feature Extraction module disentangles multimodal content into informative representations. The Category-aware Embedding Selection module selects compact, category-aware features to enhance semantic utilization. The Alternating Dual-Graph Propagation module alleviates dimensional collapse via collaborative and multimodal propagation. The Entropy-based Diversity Regularization module promotes balanced category exposure while preserving accuracy.

Figure 5. Comparison of the top- $50$ normalized singular value of two two-layer propagation operators on Anime. The curves show the normalized singular value ratios $\sigma_{j}/\sigma_{1}$ for $Q_{2UI}$ and $Q_{UI+II}$ , sorted by singular value magnitude.

Table 2. Statistics of datasets.

Dataset	Metric	Accuracy-based Methods			Diversity-based Methods				Our Method
Dataset	Metric	LightGCN	SimGCL	Freedom	DGCN	DGRec	DRGame	DivGCL	MOSS	MOSS acc
Anime	Recall@10	0.1924	0.2100	0.1985	0.1106	0.2024	0.0569	0.2132	0.1860	0.2211
	NDCG@10	0.2124	0.2248	0.2014	0.1169	0.1868	0.0975	0.2329	0.1756	0.2374
	Coverage@10	0.4074	0.4387	0.4345	0.4890	0.4428	0.4279	0.4197	0.5536	0.4417
	Entropy@10	5.0054	5.7738	5.3682	6.1741	5.5891	5.6576	5.1921	6.9645	6.0342
	GM@10	0.2800	0.3035	0.2937	0.2326	0.2994	0.1560	0.2991	0.3209	0.3125

Table 3. Comparison of models on the Anime and Game datasets.

	Anime					Game
Model	Recall@10	NDCG@10	Coverage@10	Entropy@10	GM@10	Recall@10	NDCG@10	Coverage@10	Entropy@10	GM@10
$w/o\ \mathrm{EBDR}$	0.2172	0.2306	0.4577	6.0258	0.3153	0.1092	0.0607	0.3642	2.9347	0.1994
$w/o\ \mathrm{ADGP}$	0.1730	0.1619	0.5498	6.5569	0.3084	0.0863	0.0384	0.6127	3.8642	0.2299
$w/o\ \mathrm{CAES}$	0.1722	0.1591	0.5187	6.5487	0.2989	0.0852	0.0372	0.5932	3.8533	0.2248
MOSS	0.1860	0.1756	0.5536	6.9645	0.3209	0.0907	0.0391	0.6348	3.9764	0.2400

Table 4. Ablation study on the Anime and Game datasets.

Panel of Figure 6 — Figure 6. Comparison of the geometric mean (GM) between MOSS and baseline methods.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 516 papers

MURAL: Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning
Sep 4, 2026Ahmad Mousavi, Majid Alikhani, Yeon-Chang Lee +2Multimodal Recommendation Multimodal Graph Learning
Department of Mathematics and Statistics American University · Independent Researcher Toronto, Ontario, Canada · Independent Researcher +4
Anchored Alignment: Preventing Positional Collapse in Multimodal Recommender Systems
Mar 13, 2026Yonghun Jeong, David Yoon Suk Kang, Yeon-Chang LeeMultimodal Recommendation Cross-Modal Alignment
UNIST, Ulsan, Korea · Chungbuk National University, Cheongju, Korea
DReX: An Explainable Deep Learning-based Multimodal Recommendation Framework
Feb 23, 2026Adamya Shyam, Venkateswara Rao Kagita, Bharti Rana +1Gated Recurrent Unit Multimodal Recommendation
University of Delhi, Delhi, India · National Institute of Technology, Warangal, India

Sep 4, 2026J/K move · Enter open · S save

MURAL: Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning

Ahmad Mousavi, Majid Alikhani, Yeon-Chang Lee, Roberto Corizzo+1

Department of Mathematics and Statistics American University · Independent Researcher Toronto, Ontario, Canada · Independent Researcher+4

Multimodal Recommendation Multimodal Graph Learning Uncertainty-Aware Multimodal Fusion Graph Structure Learning

Open paper AskPDF arXiv

Multimodal Graph Neural Networks have become standard for recommendation by augmenting sparse interaction data with content features. Yet current architectures face two bottlenecks: structural rigidity, from a reliance on static precomputed similarity graphs that cannot adapt to evolving preferences; and semantic fragility, where noisy modality signals are indiscriminately fused, distorting the collaborative signal. We propose MURAL (Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning), a unified framework that shifts multimodal recommendation from fixed structural augmentation to dynamic topology discovery. To address structural rigidity, an Adaptive Edge Learner combines a differentiable retrieval-augmented strategy with an approximate nearest neighbor search to discover latent item-item correlations that are both semantically adaptive and computationally scalable (O(NlogN)). To address semantic fragility, an Uncertainty-Aware Fusion module models the aleatoric uncertainty of heterogeneous modalities, dynamically down-weighting unreliable features while prioritizing high-confidence signals as a defense against cross-modal noise. We further employ a contrastive teacher-student alignment that anchors modality-specific representations to stable behavioral signals, ensuring optimization stability without gradient leakage. Experiments on large-scale benchmarks including TikTok and Amazon show that MURAL significantly surpasses both structural and generative state-of-the-art baselines, achieving superior accuracy while offering interpretability through domain-specific modality dominance and robustness under extreme data corruption.

Abstract

Recommender Systems

Explore similar work

MURAL: Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning

Anchored Alignment: Preventing Positional Collapse in Multimodal Recommender Systems

DReX: An Explainable Deep Learning-based Multimodal Recommendation Framework