cs.IRNov 10, 2026

LUMIR: Decoupled Multimodal Representation Learning from Heterogeneous Data for Next-POI Recommendation

Authors: Zehui Wang, Wolfram Höpken, Dietmar Jannach

Organizations: University of Applied Sciences Ravensburg-Weingarten, Doggenriedstrasse, Weingarten, Weingarten, 88250, Germany · University of Klagenfurt, Universitätsstraße 65–67, Klagenfurt am Wörthersee, 9020, Austria

Abstract

As multimodal tourism-related information collected from heterogeneous data sources has become increasingly available, learning effective representations has attracted attention for improving recommendation performance. Existing approaches often tightly couple multimodal fusion with the architecture and optimization objective of a specific downstream model, which limits the generality and reusability of the learned representations. To address these limitations, we propose LUMIR, a decoupled multimodal representation learning framework based on contrastive learning. Multimodal information collected from heterogeneous data sources is first semantically unified into a shared textual representation space using large language models (LLMs). An interaction-aware fusion mechanism is then designed to model cross-modal interactions, while sequential signals are incorporated as a structural constraint to guide representation learning. To evaluate the learned representations, experiments are conducted on a multimodal point-of-interest (POI) dataset under the next-POI recommendation setting across multiple sequential recommendation backbones. The results show that LUMIR consistently outperforms existing multimodal fusion baselines in both accuracy-oriented and beyond-accuracy metrics, demonstrating that decoupling representation learning from downstream optimization provides an effective paradigm for learning reusable multimodal representations. The code is publicly available at https://github.com/wzehui/LUMIR.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. VLM4Rec: Multimodal Semantic Representation for Recommendation with Large Vision-Language Models

    Mar 13, 2026Ty Valencia, Burak Barlas, Varun Singhal +2Multimodal RecommendationMultimodal Embedding

  2. MURAL: Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning

    Sep 4, 2026Ahmad Mousavi, Majid Alikhani, Yeon-Chang Lee +2Multimodal RecommendationMultimodal Graph Learning

  3. Agent4POI: Agentic Context-Conditioned Affordance Reasoning for Multimodal Point-of-Interest Recommendation

    Apr 3, 2026Jinze Wang, Yangchen Zeng, Tiehua Zhang +5Recommender SystemsMultimodal Recommendation