Deep Interest Mining for Intent-Enriched Semantic IDs in Multimodal Generative Recommendation
Abstract
Semantic IDs (SIDs) provide the discrete item vocabulary used by generative recommendation, but their quality depends on what item evidence is preserved before quantization. In product recommendation, surface metadata often misses latent usage intent, visual evidence may be only weakly reflected in text, and downstream policy learning provides sparse feedback about whether a generated SID corresponds to a semantically useful item. We introduce DeepInterestGR, an intent-enriched SID framework for generative recommendation. Before SID quantization, CMSA enriches item representations through two complementary evidence paths: recommendation-oriented VLM captions and projected image embeddings. DCIM then uses an LLM to mine item-side intent descriptors -- latent usage motivations implied by product content rather than personalized user states. During policy training over the constructed SIDs, QARM adds a relevance-gated semantic-quality bonus on top of standard SID rewards, applying the bonus only when the generated SID decodes to the target item. Thus, semantic quality cannot reward a fluent but irrelevant item prediction. Experiments on three Amazon Product Review categories (Beauty, Sports, and Instruments) show that DeepInterestGR improves over competitive generative and RL-based baselines, with relative gains of up to 15.1% in NDCG@5 and 13.9% in NDCG@10 over the strongest per-metric baseline. Component ablations, CMSA branch analyses, reward variants, and SID-level case studies support a bounded claim: enriching pre-quantization item evidence with visual cues and item-side intent descriptors, together with relevance-gated semantic rewards, improves SID-based generative recommendation under the evaluated settings.
Figures & tables
| Method | Beauty | Sports | Instruments | |||||||||
| HR@5 | HR@10 | N@5 | N@10 | HR@5 | HR@10 | N@5 | N@10 | HR@5 | HR@10 | N@5 | N@10 | |
| Traditional Sequential Models | ||||||||||||
| GRU4Rec | 0.0312 | 0.0518 | 0.0189 | 0.0256 | 0.0198 | 0.0324 | 0.0118 | 0.0161 | 0.0285 | 0.0467 | 0.0172 | 0.0231 |
| Caser | 0.0287 | 0.0483 | 0.0171 | 0.0234 | 0.0175 | 0.0291 | 0.0103 | 0.0142 | 0.0261 | 0.0432 | 0.0156 | 0.0212 |
| HGN | 0.0335 | 0.0549 | 0.0201 | 0.0271 | 0.0212 | 0.0348 | 0.0126 | 0.0172 | 0.0302 | 0.0495 | 0.0183 | 0.0246 |
| Transformer-based Models | ||||||||||||
| Item | Surface Text | Visual Evidence / Text-only Failure | Descriptor and SID Effect |
| Case 1: Similar title, different intent . Similar soccer-ball titles can hide different user intents; visual cues help separate youth recreation from adult training. | |||
| Youth / beginner ball | Title: Classic Soccer Ball, Size 4, Blue/Yellow Surface: soccer ball, size 4, training, durable | Bright blue/yellow color; youth-oriented appearance; soft rubber texture; backyard or school-field scene. Failure: text-only SID may place it near generic training or match balls. | parent-purchased youth beginner soccer practice ; school/recreational play. QARM: 1; Conf.: 0.89 SID: a1,b33,c192 a1,b08,c144 |
| Adult / match ball | Title: Classic Soccer Ball, Size 5, White/Black Surface: soccer ball, size 5, training, durable | Standard match-ball texture; stitched panels; grass/turf field; full-size goal and club-practice cues. Failure: surface text remains close to the youth ball, but usage intent differs. | competitive outdoor soccer training ; adult club practice; match preparation. QARM: 1; Conf.: 0.86 SID: a1,b33,c197 a1,b61,c209 |
| Case 2: Short text, visual context needed . When titles are short and underspecified, image context reveals indoor futsal control versus outdoor field practice. | |||
| Indoor futsal ball | Title: Soccer Training Ball, Size 4 Surface: soccer training ball, size 4 | Indoor court/gym-floor setting; compact size; futsal-style use; low-bounce or hard-court control cues. Failure: short title can collapse into ordinary soccer training balls. | indoor futsal low-bounce control training ; small-sided court play. QARM: 1; Conf.: 0.91 SID: a1,b42,c083 a1,b74,c031 |
| Outdoor training ball | Title: Soccer Training Ball, Size 5 Surface: soccer training ball, size 5 | Grass/turf field; outdoor durability cues; water-resistant usage; team practice and full-size field context. Failure: surface text is close to the indoor futsal item, but context indicates outdoor match training. | outdoor field soccer practice ; team drills; grass/turf match training. QARM: 1; Conf.: 0.88 SID: a1,b42,c091 a1,b27,c186 |
| Variant | HR@5 | HR@10 | N@5 | N@10 |
| DeepInterestGR (Full) | 0.0678 | 0.1032 | 0.0436 | 0.0558 |
| w/o DCIM | 0.0598 | 0.0921 | 0.0378 | 0.0489 |
| w/o CMSA (Text-Only) | 0.0641 | 0.0983 | 0.0412 | 0.0529 |
| w/o QARM Reward | 0.0635 | 0.0973 | 0.0408 | 0.0526 |
| w/o RL (SFT only) | 0.0567 | 0.0879 | 0.0358 | 0.0463 |
| Method | Beauty | Sports | Instruments |
| w/o DCIM | 0.0832 | 0.0558 | 0.0770 |
| LLM Summarization | 0.0864 | 0.0582 | 0.0802 |
| LLM Paraphrasing | 0.0855 | 0.0575 | 0.0794 |
| DCIM | 0.0952 | 0.0645 | 0.0889 |
| Setting | HR@5 | HR@10 | N@5 | N@10 |
| Text-Only | 0.0641 | 0.0983 | 0.0412 | 0.0529 |
| Caption-only | 0.0652 | 0.0995 | 0.0420 | 0.0537 |
| Image-embedding-only | 0.0648 | 0.0989 | 0.0417 | 0.0533 |
| Caption + Image Embedding | 0.0678 | 0.1032 | 0.0436 | 0.0558 |
| vs. Text-Only | +5.8% | +5.0% | +5.8% | +5.5% |
| Setting | HR@5 | HR@10 | N@5 | N@10 |
| Reward Strategy Comparison | ||||
| Standard SID reward (exact + format) | 0.0635 | 0.0973 | 0.0408 | 0.0526 |
| Collaborative | 0.0623 | 0.0954 | 0.0398 | 0.0513 |
| Prefix-Match | 0.0642 | 0.0981 | 0.0411 | 0.0528 |
| Descriptor-Quality Signal Variants | ||||
| Uniform quality bonus | 0.0635 | 0.0973 | 0.0408 | 0.0526 |
Recommender Systems
Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.
See all 508 papers