Conversational recommender systems have recently adopted vision-language models to match user dialogue intents with multimodal product information. However, extending these methods to many product domains faces a dilemma: deploying a separately adapted VLM per domain scales cost linearly with the number of domains and requires routing each dialogue to the right model, while naively merging all domains into a single VLM degrades accuracy because product characteristics vary widely across categories. We propose LightCRS, a lightweight, query-conditioned framework for unified multi-domain multimodal conversational recommendation. LightCRS precomputes multimodal product representations once offline with frozen encoders. Online, a compact 5.3M-parameter module fuses this evidence for each dialogue via query-conditioned cross-attention, whose parameters are shared across all domains, and compares candidates jointly with an inter-candidate Transformer. To enable rigorous evaluation, we construct MM-R2A-25, a 25-domain benchmark pairing natural Reddit recommendation-seeking conversations with Amazon product listings. LightCRS achieves state-of-the-art average HR@1, NDCG@5, and MRR, improving HR@1 by 35.4% over the strongest baseline while reducing per-query latency by 266 times. Joint multi-domain training itself improves HR@1 by 15.2% over separately trained single-domain models.
Figures & tables
Figure 1. Architecture of LightCRS. The offline stage extracts and caches multi-view product representations using frozen GTE-base and LLaVA-v1.6. The online stage encodes the query, performs query-conditioned cross-attention over candidate representations, models candidate interactions with an inter-candidate Transformer, and outputs relevance scores through an MLP scoring head.
Method
Rec. Perf.
Training Cost
Inference Efficiency
HR@1 ↑
NDCG@5 ↑
MRR ↑
Train. P. ↓
Total P. ↓
Size ↓
GPU Mem. ↓
MFLOPs ↓
Lat. ↓
Thpt. ↑
Retrieval
BM25 ( Robertson and Zaragoza, 2009 )
0.0142
0.0434
0.1724
–
–
8.4MB
0.0GB
0.0
65.7
15.21
SBERT ( Reimers and Gurevych, 2019 )
0.0603
0.1224
0.2282
–
0.1B
0.4GB
0.5GB
70,675.1
17.5
58.14
RoBERTa ( Liu et al., 2019 )
0.0483
0.1065
0.2164
–
0.4B
1.4GB
1.5GB
154,634.3
24.5
40.94
SimCSE ( Gao et al., 2021 )
0.0195
0.0567
0.1837
–
0.4B
1.4GB
1.5GB
295,071.7
29.1
34.72
Table 1. Average recommendation performance and resource usage across methods. Gray -shaded rows denote multimodal task-specific CRS methods, training and inference costs for our method are compared only with those of methods in this group. “–” indicates that the value cannot be calculated. Bold and underlined values denote the best and second-best results. Abbreviations: Rec. Perf.=recommendation performance, Train. P.=trainable parameters, Total P.=total parameters, GPU Mem.=peak GPU memory, Lat.=latency (ms/query), and Thpt.=throughput (queries/s).
Model Variant
HR@1 ↑
NDCG@5 ↑
MRR ↑
w/o ICC
0.1432 ±0.0079
0.3505 ±0.0107
0.3389 ±0.0089
w/o ID
0.1109 ±0.0069
0.3004 ±0.0034
0.3015 ±0.0039
w/o Feat
0.1054 ±0.0064
0.2994 ±0.0054
0.2969 ±0.0047
w/o Desc
0.1021 ±0.0002
0.3011 ±0.0017
0.2972 ±0.0006
w/o Title
0.1002 ±0.0012
0.2832 ±0.0023
0.2871 ±0.0015
w/o CA
0.0998 ±0.0036
0.3052 ±0.0041
0.2971 ±0.0001
Table 2. Per-component ablation. Each row removes a single component from the full model, and we report the resulting average HR@1, NDCG@5, and MRR across all 25 product domains. Abbreviations: ICC – Inter-Candidate Comparison; ID – Image Descriptions; Feat – Features; Desc – Description; CA – Cross-Attention; VT – Visual Tokens.
Training Regime
HR@1 ↑
NDCG@5 ↑
MRR ↑
Sing.
0.2272 ±0.0090
0.4305 ±0.0030
0.4113 ±0.0040
Multi. (Ours)
0.2618 ±0.0104
0.4540 ±0.0093
0.4367 ±0.0083
Table 3. Single-domain vs. Multi-domain. The single-domain regime trains a separate model per product category and we average metrics across the 25 domains; the full model trains one shared model jointly on all 25 domains. Abbreviations: Sing.=Single-domain, Multi.=Multi-domain.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Category
#Conv
#Turns
#Items
#Images
Appliances
2,291
4,582
739
4,062
Arts, Crafts & Sewing
1,267
2,315
823
3,473
Automotive
1,512
2,869
811
3,657
Baby Products
3,336
6,672
1,975
11,561
Beauty
7,672
22,966
5,433
28,082
Books
4,707
13,250
1,978
1,864
Appendix
Table 4. Summary of the MM-R2A-25 dataset. #Conv: number of conversations. #Turns: number of conversational turns. #Items: number of unique products. #Images: total number of verified product images.
Dataset
#Conv
#Turns
#Items
Domain
Source
Facebook Rec ( Dodge et al., 2015 )
1M
6M
–
Movies
Synthetic
ReDial ( Li et al., 2018 )
10K
182K
6.2K
Movies
Crowd-sourced
GoRecDial ( Kang et al., 2019 )
9K
170K
–
Movies
Crowd-sourced
OpenDialKG ( Moon et al., 2019 )
15K
91K
–
Movies, Music, etc.
Crowd-sourced
TG-ReDial ( Zhou et al., 2020c )
10K
129K
–
Movies
Synthetic
DuRecDial 2.0 ( Liu et al., 2021 )
16.5K
255K
–
Movies, Music, etc.
Crowd-sourced
Appendix
Table 5. Comparison of existing conversational recommendation datasets. MM-R2A-25 consists of natural Reddit conversations aligned with Amazon Reviews 2023 across 25 diverse product categories.
HR@1 ↑
Task-specific CRSs
Ours
Domain
NBCRS ( Xie et al., 2024 )
RTA ( He et al., 2024 )
LaViC ( Jeon et al., 2025 )
MSCRS ( Wei et al., 2025 )
PRIME ( Yue et al., 2025 )
LightCRS
Appliances
0.0957 ±0.0101
0.2130 ±0.0090
0.2709 ±0.0014
0.2391 ±0.0063
0.2261 ±0.0223
0.2873 ±0.0152
Arts & Crafts
0.0078 ±0.0091
0.0469 ±0.0048
0.1013 ±0.0031
0.0859 ±0.0059
0.1016 ±0.0036
0.1349 ±0.0270
Automotive
0.0197 ±0.0090
0.0395 ±0.0051
0.0560 ±0.0253
0.0526 ±0.0053
0.0852 ±0.0128
0.1184 ±0.0139
Baby Products
0.0567 ±0.0190
0.0090 ±0.0058
0.3373 ±0.0113
0.5254 ±0.0168
0.5791 ±0.0136
0.7194 ±0.0093
Appendix
Table 6. Per-domain comparison of the task-specific CRS baselines and LightCRS across 25 Amazon product categories. Panel A reports HR@1 for all methods, whereas Panel B reports NDCG@5 and MRR only for ranking-based methods. Bold and underline indicate the best and second-best results, respectively.
HR@1 ↑
Retrieval
Generative
Ours
Domain
BM25 ( Robertson and Zaragoza, 2009 )
SBERT ( Reimers and Gurevych, 2019 )
RoB. ( Liu et al., 2019 )
SimCSE ( Gao et al., 2021 )
Vic. ( Chiang et al., 2023 )
1.5 ( Liu et al., 2024a )
1.6 ( Liu et al., 2024b )
1.6+I ( Liu et al., 2024b )
M2.7
M3
LightCRS
Appliances
0.0076
0.0684
0.0456
0.0000
0.0783 ±0.0024
0.0783 ±0.0059
0.0739 ±0.0133
0.1174 ±0.0105
0.1957 ±0.0033
0.2174 ±0.0014
0.0994 ±0.0208
Arts & Crafts
0.0000
0.0153
0.0230
0.0154
0.0547 ±0.0061
0.0546 ±0.0076
0.0551 ±0.0154
0.0560 ±0.0110
0.0703 ±0.0240
0.0880 ±0.0035
0.1198 ±0.0252
Automotive
0.0000
0.0128
0.0132
0.0134
0.0197 ±0.0008
0.0395 ±0.0126
0.0197 ±0.0168
0.0526 ±0.0100
0.0612 ±0.0151
0.0329 ±0.0211
0.1096 ±0.0312
Baby Products
0.0627
0.0657
0.0478
0.0209
0.1164 ±0.0128
0.1194 ±0.0162
0.1463 ±0.0156
0.0657 ±0.0042
0.1007 ±0.0220
0.1025 ±0.0174
0.1363 ±0.0125
Appendix
Table 7. Comparison of retrieval, generative baselines, and LightCRS on unseen domains across 25 Amazon product categories. Panel A reports HR@1; Panel B reports NDCG@5 and MRR for ranking-based methods. Bold and underline indicate the best and second-best results within each evaluation setting. Abbreviations: RoB. denotes RoBERTa large ; Vic. denotes Vicuna-v1.5; 1.5, 1.6, and 1.6+I denote LLaVA-v1.5, LLaVA-v1.6, and LLaVA-v1.6+image; M3 denotes MiniMax-M3; and M2.7 denotes MiniMax-M2.7.
You are given a Reddit conversation containing multiple numbered messages. Each message follows the format <message>(i) [content]</message> . Determine whether the final message explicitly recommends one or more specific items in response to a recommendation request in an earlier message. Answer yes if both conditions are satisfied, no otherwise, or uncertain if the conversation does not provide enough information. Return only yes , no , or uncertain , without any explanation. Conversation: {conversation}
Appendix
Table 8. Prompt used to identify product-recommendation conversations. Only conversations labeled yes by GPT-3.5-turbo are retained in MM-R2A-25 .
You are given an Amazon product title and image. Generate a detailed, visually grounded description that helps users evaluate the product. Focus on its appearance, visible features, and other visually informative details. Do not repeat or mention the product title. Product title: {title} Product image: {image}
Appendix
Table 9. Prompt used to generate visually grounded product descriptions.
You are given a conversation between a user requesting product recommendations, marked by <submission> , and other users providing responses, marked by <comment> . You are also given candidate products formatted as ID: title . Select the candidate that best matches the user’s needs. Return only its ID without any explanation. Conversation: {conversation} Candidates: {list of candidates}
Appendix
Table 10. Prompt for text-only product recommendation, used by generative baselines.
You are given a conversation between a user requesting product recommendations, marked by <submission> , and other users providing responses, marked by <comment> . You are also given candidate products, each formatted as ID: title followed by its image. Select the candidate that best matches the user’s needs based on the conversation, title, and image. Return only its ID without any explanation. Conversation: {conversation} Candidates: {list of candidates}
Appendix
Table 11. Prompt for image-aware product recommendation, used by generative baselines.
#
Stage
View
Encoder
Wall-clock
Throughput
Peak GPU mem
Full MM-R2A-25
(50 items)
(items/s)
(GB)
(GPU-hours)
1
Text encoding
title / features / description
GTE-base
0.47 s
319.3
0.29
0.13
2
Caption generation
image captions
LLaVA-v1.6 (full VLM)
487.18 s
0.10
15.03
593.46
3
Caption encoding
caption embeddings
GTE-base
0.18 s
277.3
0.28
0.22
4
Visual extraction
visual tokens
LLaVA-v1.6 vision tower
9.32 s
5.36
14.43
2.60
Appendix
Table 12. Per-stage offline preprocessing cost measurement. “Wall-clock” denotes the measured runtime on the 50-item sample after one warm-up pass; “Throughput” denotes the processing rate in items or encodings per second; “Peak GPU mem” denotes the maximum allocated GPU memory during the timed run; “Full MM-R2A-25” reports the extrapolated cost for 50,121 products and 219,268 product images.
View
Tensor shape
Source
Per-product
Full MM-R2A-25
(KB, FP32)
(GB, FP32)
Title embedding
(768,)
GTE-base
3.1
0.15
Features embedding
(768,)
GTE-base
3.1
0.15
Description embedding
(768,)
GTE-base
3.1
0.15
Caption embeddings
(N,768)
GTE-base over LLaVA-v1.6 captions
13.5
0.68
Visual tokens
(5,4096)
LLaVA-v1.6 vision tower
81.9
4.11
Appendix
Table 13. Cached product-representation footprint. Each view is stored as a tensor with the indicated shape and FP32 precision. Per-product KB uses the catalog-wide average of 4.38 images per product for the caption row; full-catalog totals apply this same average to all products.
Multimodal Vision-Language Models (VLMs) have demonstrated remarkable capabilities in cross-modal understanding, yet a fundamental challenge persists when applying them to recommendation: as representations propagate through multi-step reasoning, both visual and textual signals progressively attenuate - a phenomenon we term cross-modal dilution. To address this, we propose LARK (Latent-Aligned Reasoning frameworK), a two-stage latent reasoning framework with complementary alignment mechanisms within a single VLM. In the first stage, learnable latent tokens are interleaved with multi-step chain-of-thought (CoT) reasoning and explicitly aligned with a frozen vision encoder, serving as visual checkpoints that preserve perceptual details throughout the reasoning chain. In the second stage, the latent representations are projected via a bridge MLP and trained with item-to-item contrastive learning; to prevent the reasoning semantics from fading, intermediate features are aligned with the CoT hidden states from the first stage, anchoring the final embeddings to the model's own reasoning output. Experiments on three public benchmarks and one industrial dataset show that LARK achieves state-of-the-art performance across multiple recommendation architectures, with controlled ablations confirming the distinct contribution of each component.
Conversational Recommender Systems (CRS) powered by Large Language Models (LLMs) enable users to articulate explicit and dynamic preferences, overcoming the limitations of fixed templates. However, despite their superior semantic proficiency, LLMs have not yet achieved corresponding improvements in recommendation accuracy. This discrepancy arises from a fundamental representation gap: while LLMs operate within a semantic space, they lack the behavioral grounding needed to encode user behavioral patterns, such as item co-occurrences, which are crucial for accurate recommendations. To address this, we propose a model-agnostic Variational EM Framework for Bridging Semantics and Behavior in LLM-based Conversational Recommendation (LatentCRS). Based on the observation that dialogue and interactions reflect the same latent intent, LatentCRS uses a variational expectation-maximization (EM) procedure, where user intent connects semantic representations with behavioral patterns. Extensive experiments on real-world datasets demonstrate that LatentCRS effectively bridges the representation gap and outperforms baselines.
Recent advances in multimodal large language models (MLLMs) have substantially expanded the capabilities of multimodal retrieval, enabling systems to align and retrieve information across visual and textual modalities. Yet, existing benchmarks largely focus on coarse-grained or single-condition alignment, overlooking real-world scenarios where user queries specify multiple interdependent constraints across modalities. To bridge this gap, we introduce MCMR (Multi-Conditional Multimodal Retrieval): a large-scale benchmark designed to evaluate fine-grained, multi-condition cross-modal retrieval under natural-language queries. MCMR spans five product domains: upper and bottom clothing, jewelry, shoes, and furniture. It also preserves rich long-form metadata essential for compositional matching. Each query integrates complementary visual and textual attributes, requiring models to jointly satisfy all specified conditions for relevance. We benchmark a diverse suite of MLLM-based multimodal retrievers and vision-language rerankers to assess their condition-aware reasoning abilities. Experimental results reveal: (i) distinct modality asymmetries across models; (ii) visual cues dominate early-rank precision, while textual metadata stabilizes long-tail ordering; and (iii) MLLM-based pointwise rerankers markedly improve fine-grained matching by explicitly verifying query-candidate consistency. Overall, MCMR establishes a challenging and diagnostic benchmark for advancing multimodal retrieval toward compositional, constraint-aware, and interpretable understanding. Our code and dataset is available at https://github.com/EIT-NLP/MCMR
Xuan Lu, Kangle Li, Haohang Huang +3
Shanghai Jiao Tong University · Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative +1