Multi-merchant e-commerce catalogs contain equivalent and related products under different merchant-scoped identifiers, fragmenting behavioral evidence across merchants. Expert-defined taxonomies, meanwhile, are often too coarse for fine-grained discovery. We investigate whether a single hierarchical Semantic ID (\sid{}) representation can support personalized ranking and query reformulation. Learned once from product-content embeddings, the hierarchy defines product concepts at multiple granularities that each application combines with its own behavioral and serving context. For ranking, we aggregate consumer affinity and product performance over \sid{} prefixes and derive sequence features for candidate products and consumer histories. Controlled ablations show improved offline relevance, while online evaluation of the full ranking treatment shows stronger top-slot add-to-cart engagement and broader exposure for less-popular products. For query reformulation, we ground queries and session transitions in \sid{} concepts, use the hierarchy for navigation and refinement, and filter suggestions against the merchant's assortment. Offline evaluation shows finer intent preservation than taxonomy and higher-quality suggestions than raw query-string transitions; online evaluation shows reduced search effort and earlier access to purchasable products. These results show that a shared semantic product hierarchy can support both recommendation and search while preserving the task-specific context required by each application.
Figures & tables
Figure 1. Overview of shared SID construction and reuse across search and recommendation. Catalog product profiles are embedded and residual-quantized into a three-level SID hierarchy. Search queries and consumer interactions are then grounded and aggregated through SID prefixes, enabling cross-merchant preference transfer for personalized ranking and coarse-to-fine query reformulation through hierarchical descent. A shared construction spine transforms product text into a three-level Semantic ID and two feature families. It branches into a personalized-ranking lane showing cross-merchant evidence transfer and a query-reformulation lane showing grounding, navigation, hierarchical descent, business-kind context, and merchant assortment.
Prefix level
Gini
DBI
Held-out cosine
L1 (coarse)
0.472
3.934
0.957
L2 (intermediate)
0.508
2.065
0.965
L3 (fine)
0.467
0.976
0.981
Table 1. Intrinsic characteristics of product groups induced by SID prefix levels. Lower Gini and DBI indicate more balanced code usage and better-separated groups, respectively; higher held-out cosine indicates stronger semantic coherence on unseen products.
Feature family
Feature Scope
Description
Dense aggregates
Consumer
Order frequency, purchase recency, and subtotal statistics for each SID prefix in the consumer’s history.
Submarket
Impressions, clicks, ATC actions, purchases, and associated rates for each SID prefix within a submarket.
Sequence
Item
SPM token sequence obtained from the candidate item’s SID .
Consumer
SPM tokens aggregated from previously ordered products and ranked by their associated order counts.
Table 2. Examples of SID -derived ranking features.
Model
MRR@5
NDCG@5
FC-A
+2.10%
+2.92%
FC
+6.98%
+6.76%
Table 3. Offline ablation of SID -derived ranking features. All values are relative gains over the production baseline.
Outcome
Relative gain
Subtotal
+0.31%
Average carousel ATC rate
+5.5%
Item ATC rate, position 1
+8%
Item ATC rate, position 2
+16%
Item ATC rate, position 3
+6%
Table 4. Online results for the full ranking treatment. The treatment includes SID -derived features alongside concurrent non- SID updates.
Metric
Relative change
95% CI
Purchase MRR
+0.558%
[+0.294,+0.823]%
ATC position
−1.571%
[−2.159,−0.982]%
Search scroll depth
−1.866%
[−2.262,−1.470]%
Table 5. Online impact of SID -based query reformulation on search efficiency. Changes are relative to control; lower ATC position and scroll depth are better.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
SID property
Example
Observed effect
Personalized item ranking
Cross-merchant transfer
Prior purchases of mushrooms and chicken at another merchant
Matching products rise toward the top despite having different merchant-scoped identifiers.
Fine-grained semantic affinity
Prior purchase of a makeup brush
A related brush set rises above a makeup sponge, and more brush products appear among the top results.
Query reformulation
Hierarchical refinement
Source query: clay mask
L3 descent introduces more specific intents, including sheet mask and hydrating mask .
Coarse-to-fine navigation
Source query: car clean
L3 adds interior wipes , interior cleaner , and leather wipes , while L2 retains a useful lateral pivot.
Appendix
Table 6. Illustrative effects of SID transferability and hierarchical granularity.
SID role
Personalized item ranking
Query reformulation
Input unit
Merchant-scoped product listings
Raw query strings
Mapping into SID space
Products in consumer interaction histories
Queries grounded through associated products
Evidence pooled
Cross-merchant interactions among semantically related products
Transitions across lexical variants of the same concept
Hierarchy use
L1–L3 prefixes used jointly as ranking features
L2 transitions for navigation; L3 descent for refinement
Task-specific signals
Affinity, recency, product performance, and interaction sequences
Transition strength, query evidence, and concept popularity
Context restored
Consumer, candidate-product, and regional context
BV, source query, and merchant assortment
Appendix
Table 7. How personalized ranking and query reformulation reuse the shared SID hierarchy.
Semantic retrieval in e-commerce must handle short, noisy, and colloquial queries over large product catalogs with fine-grained attribute distinctions. We present a Siamese LLM dual-encoder trained through a two-stage pipeline: contrastive learning with a false-negative margin mask to prevent penalization of near-duplicate products, followed by Relative Odds Alignment for Retrieval (ROAR), a preference optimization objective that extends Bradley-Terry to variable-sized graded relevance groups via consecutive odds-ratio margins. The training corpus mirrors this progression - substitute query-product pairs provide coarse semantic supervision in Stage 1 and graded relevance annotations drive fine-grained ranking in Stage 2. The resulting system accurately retrieves exact matches while correctly ordering substitutes and complementary products, with gains confirmed across query-frequency strata and business verticals, and statistical significance validated through live A/B deployment at scale.
E-commerce search requires distinguishing products that are merely related to a query from those that directly satisfy the user's shopping intent. We augment query-product pairs with structured LLM-generated query and product attributes and human-validated relevance, explanations, and centrality judgments, and evaluate these signals using a simple dual-encoder retriever and MLP re-ranker. On an augmented subset of ESCI, a human-feature oracle reaches 0.9382 nDCG@10, while a human-free trained Q+P configuration reaches 0.9258. Synthetic approximations of the human signals reach 0.9150 overall but provide substantial gains for difficult, low-performing queries. Ablations show that most of the oracle improvement comes from post-edited explanations and annotator comments rather than the scalar centrality feature, suggesting that LLMs are most useful for exposing and approximating structured semantic supervision rather than replacing human judgment directly.
Girish A. Koushik, Swapnil Bhosale, Samarth Agrawal +4
Computer Science, University of Surrey, Guildford, United Kingdom · People-Centred AI, University of Surrey, Guildford, United Kingdom · eBay USA +2
Despite rapid progress of continuous embeddings for e-commerce search relevance, a long-standing open problem is the difficulty in capturing fine-grained attribute distinctions. While discrete Semantic Identifiers (SIDs) have been widely adopted as a promising alternative, existing SID generation methods rely heavily on unsupervised quantization. In realistic scenarios, the lack of explicit supervision often makes it more difficult to dictate which items should share an SID, resulting in limited capability for query-dependent ranking. To address the issue of unsupervised SIDs, we propose to explicitly model discrete relevance features and develop a Discrete Semantic Identifier Relevance Model (DSIRM). Specifically, we present a query-bridged contrastive quantization approach on the item side, injecting query-item interaction supervision into Residual Quantization to actively learn relevance-aware semantic partitions. On the other hand, we explore generative LLMs on the query side to explicitly predict item SIDs from text, resolving tail queries and intent ambiguity. Hierarchical prefix matching between query and item SIDs yields discriminative features that perfectly complement dense signals. Extensive experimental results on Tmall's production data show that our proposed approach has achieved better results, improving offline AUC by +1.54%. Deployed via an efficient hybrid architecture, it achieves significant online lifts (+0.13% UCTR, +0.25% UCTCVR), proving its massive industrial value.