Organizations: Taobao & Tmall Group of Alibaba, Hangzhou, China · Nankai University, Tianjin, China · University of Science and Technology of China, Hefei, China
Generative retrieval assigns each item a discrete semantic identifier (SID) and retrieves items by autoregressively generating SID sequences. In dynamic environments such as e-commerce, new items arrive continuously, requiring periodic updates to the SID-to-Item mapping and model to maintain retrieval timeliness. Each update, however, can invalidate search paths already learned by the retriever, and the quantitative relationship between mapping change and retrieval quality has not been systematically characterized. We study this problem with extensive controlled experiments over 67 days of large-scale e-commerce search logs and obtain three core findings. (1) SID semantic drift causes severe retrieval degradation that may be masked by large beam sizes: mapping updates reduce HitRate by 41.78-62.05% at beam size 1 but only 8.33-10.85% at beam size 1000, as redundant candidate generation conceals the true severity of drift. (2) The fraction of changed SIDs is strictly monotonically related to retrieval quality: as the remapped fraction increases from 0% to 72.75%, HitRate decreases monotonically, dropping up to 42.66% at beam size 1. Full remapping remains substantially worse than the preservation policy under warm-start training, indicating that short-budget continued training alone cannot repair topology disruption. (3) The SID topology-preserving principle makes even the simplest warm-start strategy highly effective: fixing old-item SIDs, inserting new-items, and warm-starting from the previous model improves old-item HitRate by 10% and new-item HitRate by 109%, validated across two non-overlapping update cycles. These findings distill into a concise deployment principle: insert new-items, freeze old-items; warm-start when possible, otherwise freeze; never change SIDs the model has already learned. A three-week online A/B test on TmallAPP shows UCTCVR +0.29% and GMV +1.16%.
Figures & tables
Figure 1. Framework overview. Top: Three-level SID construction, two semantic layers from RQ-VAE with QI contrastive learning produce prefix (s1,s2) ; a business-value layer appends s3 by ranking items with engagement signals. Bottom: Retrieval pipeline from embedding through SID construction, LLM autoregressive generation, beam search, and SID-to-item map resolution. There are two divergent types of update paths: the red paths rewrite old item SIDs and use a soup model (combining old and new models), which degrades metrics; the green paths freeze old SIDs, insert new items, and directly apply the new model fine-tuned from the old one, improving performance on both old and new items.
Retrieval Quality
Resolved Items
SID
HR@1
HR@10
HR@100
HR@1000
MRR
Avg@B1
Med@B1
Avg@B1000
Med@B1000
FS
0.0584
0.2409
0.5134
0.7175
0.1181
1
1
925
966
HC
0.0589
0.2457
0.5215
0.7183
0.1199
1
1
1200
986
Table 1. Day1 comparison of FS and HC identifiers: HitRate, MRR, and resolved items per query.
SID
Map
Feat
HR@1
HR@10
HR@100
HR@1000
MRR
FS
×
×
0.0585
0.2324
0.4927
0.7002
0.1158
FS
✓
✓
0.0319
0.1268
0.3262
0.6333
0.0647
FS
✓
×
0.0222
0.0982
0.2890
0.6242
0.0492
HC
×
×
0.0584
0.2344
0.5005
0.7035
0.1164
HC
✓
✓
0.0340
0.1394
0.3535
0.6449
0.0706
HC
✓
×
0.0226
0.1056
0.3172
0.6373
0.0519
Table 2. Old-item metrics on Day30 without adding new items. ✓ : refreshed; × : frozen.
Policy
HR@1
HR@10
HR@100
HR@1000
MRR
Full-Re
0.0187
0.0828
0.2499
0.5133
0.0417
Frozen
0.0180
0.0835
0.2507
0.5127
0.0415
Table 3. New-item HitRate and MRR (Day30).
Policy
γ
HR@1
HR@10
HR@100
HR@1000
MRR
Full-Re
72.75%
0.0332
0.1368
0.3523
0.6455
0.0692
t=10
29.08%
0.0507
0.1996
0.4332
0.6568
0.1004
t=100
7.36%
0.0553
0.2196
0.4694
0.6712
0.1097
t=500
1.65%
0.0557
0.2217
0.4749
0.6769
0.1106
Frozen
0%
0.0579
0.2343
0.5011
0.7043
0.1160
Table 4. Old-item metrics versus the remapped fraction γ .
Figure 2. Monotonic remapping effect. Metric decline rate (%) relative to the frozen baseline as the remapping threshold becomes more aggressive.
Items
Policy
HR@1
HR@10
HR@100
HR@1000
MRR
Old
Full-Re
0.0466
0.1867
0.4309
0.6923
0.0941
Frozen
0.0637
0.2597
0.5355
0.7272
0.1276
New
Full-Re
0.0330
0.1474
0.3684
0.6193
0.0711
Frozen
0.0376
0.1590
0.3833
0.6200
0.0779
Table 5. Warm-start comparison after the 30-day cycle.
Figure 3. Warm-start training improvement over frozen baseline (Day-30). Relative change (%) for Full-Re and Frozen compared with the frozen (no-retraining) policy.
Items
α
HR@1
HR@10
HR@100
HR@1000
MRR
Old
0.0
0.0637
0.2597
0.5355
0.7272
0.1276
0.1
0.0631
0.2590
0.5321
0.7270
0.1276
0.5
0.0628
0.2502
0.5228
0.7193
0.1243
1.0
0.0579
0.2343
0.5011
0.7043
0.1160
New
0.0
0.0376
0.1590
0.3833
0.6200
0.0779
0.1
0.0339
0.1503
0.3744
0.6153
0.0727
Table 6. Weight-mixing ablation for Frozen after 30 days.
Items
α
HR@1
HR@10
HR@100
HR@1000
MRR
Old
0.0
0.0687
0.2754
0.5490
0.7328
0.1357
0.1
0.0686
0.2756
0.5491
0.7329
0.1356
0.5
0.0674
0.2714
0.5443
0.7306
0.1335
0.9
0.0647
0.2631
0.5348
0.7248
0.1291
New
0.0
0.0492
0.2114
0.4468
0.6551
0.1021
0.1
0.0466
0.2043
0.4434
0.6535
0.0983
Table 7. Weight-mixing ablation for Frozen after an additional 7-day cycle (Day37).
Figure 4. Asymmetric sensitivity to model mixing: HR@1000. Day-30 (left) and Day-37 (right) weight-mixing ablations. The horizontal axis is the new-model weight (1−α) . Solid lines show HR@1000 (left axis), dashed lines show MRR (right axis). Both metrics rise monotonically with the new-model weight, confirming that the pure new checkpoint ( 1−α=1 ) is optimal or near-optimal.
Generative retrieval offers a promising alternative by unifying the fragmented multi-stage retrieval process into a single end-to-end model. However, its practical adoption in industrial e-commerce search remains challenging, given the massive and dynamic product catalogs, strict latency requirements, and the need to align retrieval with downstream ranking goals. In this work, we propose a retrieval framework tailored for real-world recall scenarios, positioning generative retrieval as a recall-stage supplement rather than an end-to-end replacement. Our method, CQ-SID (Category-and-Query constrained Semantic ID), employs category-aware and query-item contrastive learning along with Residual Quantized VAEs to encode items into hierarchical semantic cluster identifiers, significantly reducing beam search complexity. Additionally, we develop EG-GRPO (Expert-Guided Group Relative Policy Optimization), a reinforcement learning approach that aligns generative recall with downstream ranking under sparse rewards by injecting ground-truth samples to stabilize training. Offline experiments on TmallAPP search logs show that CQ-SID achieves up to 26.76% and 11.11% relative gains in semantic and personalized click hitrate over RQ-VAE baselines, while halving beam search size. EG-GRPO further improves multi-objective performance. Online A/B tests confirm gains in GMV (+1.15%) and UCTCVR (+0.40%). The generative recall channel now contributes substantially in production, accounting for over 50.25% of exposures, 58.96% of clicks, and 72.63% of purchases, demonstrating a viable path for deploying generative retrieval in real-world e-commerce systems.
Semantic ID tokenizers assign code sequences to catalog items, which generative recommenders predict and resolve back to items. Reusing a released mapping requires checking its code organization, item resolution, and compatibility with an existing model after a refresh. SIDScope provides these checks through source traced adapters, mapping diagnostics, candidate retrieval tests, and generated trace analysis. Its inventory contains nine exports from seven tokenizer families across Amazon and Yelp, including eight executable routes and one auditable snapshot. The analyses establish three distinctions. First, code allocation and item addressability differ: in GRID, tail items occupy 95.7 distinct codes per 100 items, yet only 82.2 percent have globally unique codes. Second, prefix alignment calibrates retrieval operations that use prefixes. With fixed candidate budgets and ranking, original assignments on Beauty and Instruments expose more targets than controls preserving code capacity and coarse popularity. Resolving identical generated paths also shows that an explicit item selection rule recovers part of the gap between target path survival and singleton credit. Third, a released refresh repairs missing codes, while model adaptation separately restores new item recall. SIDScope supplies code, diagnostic summaries, and verification tools to compare mappings, evaluate their use in retrieval and generation, and test model reuse after changes.
Semantic IDs (SIDs) are now a central component of generative recommendation. Current SID-based systems assign three roles to the same token sequence. Shared prefixes are intended to organize related items, the complete SID identifies an individual item, and each generated token narrows the items that can still be returned. We systematically investigate SIDs from item encoding and SID construction to autoregressive generation and final recommendation. We examine how SID construction changes item representations and how those changes affect generation. Across three Amazon domains and eight SID constructions, SID neighborhoods recover only 32.2% of the encoder's ten nearest neighbors on average. Alternative item descriptions still retrieve the corresponding item first in 99.57% of controlled cases, yet change 38.4% of exact SIDs. These results show that SIDs retain broad organization but lose much of the encoder's fine local structure, while their exact tokens are not determined by item meaning alone. This loss becomes consequential during generation. After the final semantic token, TIGER retains only 29.9% of held-out targets that were plausible recommendations before SID filtering. Motivated by these findings, we propose Item-Supported Decoding (ISD), a lightweight inference-time method that allows a user-specific item ranking to support corresponding SID prefixes before beam search discards them. The same ranking then orders the generated items. ISD requires no additional parameters or retraining of the SID constructor or decoder. We empirically show that ISD improves NDCG@10 over the corresponding SID backbone in every evaluated setting, with relative gains of up to 31.2%. Our results show that SIDs provide useful coarse item organization, but their fine boundaries should not alone determine which items remain available during generation.
Junting Wang, Xinrui He, Yunzhe Li +1
University of Illinois Urbana-Champaign Urbana, Illinois, USA