Semantic ID tokenizers assign code sequences to catalog items, which generative recommenders predict and resolve back to items. Reusing a released mapping requires checking its code organization, item resolution, and compatibility with an existing model after a refresh. SIDScope provides these checks through source traced adapters, mapping diagnostics, candidate retrieval tests, and generated trace analysis. Its inventory contains nine exports from seven tokenizer families across Amazon and Yelp, including eight executable routes and one auditable snapshot. The analyses establish three distinctions. First, code allocation and item addressability differ: in GRID, tail items occupy 95.7 distinct codes per 100 items, yet only 82.2 percent have globally unique codes. Second, prefix alignment calibrates retrieval operations that use prefixes. With fixed candidate budgets and ranking, original assignments on Beauty and Instruments expose more targets than controls preserving code capacity and coarse popularity. Resolving identical generated paths also shows that an explicit item selection rule recovers part of the gap between target path survival and singleton credit. Third, a released refresh repairs missing codes, while model adaptation separately restores new item recall. SIDScope supplies code, diagnostic summaries, and verification tools to compare mappings, evaluate their use in retrieval and generation, and test model reuse after changes.
Figures & tables
Figure 1: SIDScope connects source-traced inputs to mapping diagnostics, retrieval and generation tests, and reproducible outputs. D1 measures code-space use, D2 shared-address risk, D3 prefix–behavior alignment, D4 popularity allocation, D5 trie structure, D6 mapping change, and D7 path-to-item resolution. Source, configuration, and license records bind these measurements to the mapping being tested.
Question
SIDInspector V0
SIDScope extension
Where shown
What is inspected?
Static item-to-code mappings
Artifact states spanning mappings, exposure, traces, and package records
Figure 1 ; Sections 3 and 7
Which artifacts enter?
Reconstructed controls and snapshots
Source-traced named routes separated from stress and control rows
Tables 2 and 10
What can D3 support?
D1–D5 diagnostic tables
Prefix calibration, fixed-budget assignment controls, and measurement checks
Tables 4 , 5 , and 11
What do traces add?
D7 input hook
Constraint-aware labels and resolver comparisons on identical generated paths
Tables 6 and 7 ; Figure 4
How broad is the matrix?
Musical worked example
Nine named exports, seven families, two data ecosystems, controls, and trained traces
Tables 2 , 3 , and 6
What changes over time?
Mapping snapshots
Paired D6 refresh audit, mapping repair, and generator handoff
Table 8
Table 1: Scientific and resource extension beyond SIDInspector V0. The final column points to the evidence introduced and evaluated in this article.
Figure 2: One mapping, distinct diagnostic questions. Items i1 and i2 share full code ⟨1,1,1⟩ , creating a D2 collision. Items i4 and i5 co-occur and share prefix ⟨2,1⟩ but have different leaves, illustrating the relation measured by D3. All values are illustrative.
Tokenizer export
Data source
Source evidence
Mapped items
SID depth
Source-traced named tokenizer exports
ReSID
Amazon / Musical
Tracked snapshot
23,742
3
ReSID-GAOQ
Amazon / Video Games
Code-derived export
24,685
3
GRID
Amazon / Beauty (P5)
Rebuilt tokenizer
12,101
3
CARD
Amazon / Beauty (P5)
Code-derived export
12,101
4
DIGER
Amazon / Beauty (P5)
Checkpoint export
12,101
3
Table 2: Artifact inventory. Eight executable tokenizer routes pass C0–C5; ReSID/Musical is a source-traced auditable snapshot. Reference and control rows calibrate probe behavior but are not counted as named tokenizer coverage. Data-source labels normalize the ecosystem and subset name; each mapped-item count refers to that export’s pinned item universe. Source evidence records the route used to inspect or reconstruct the mapping.
Route
Items
Depth
D1 symbols
D2 collision
D3@1
D4 tail code ratio
D5 leaves
ReSID-GAOQ / Amazon Video
24,685
3
32
0.000
0.213
1.000
24,685
GRID / Amazon Beauty
12,101
3
256
0.205
0.038
0.957
10,619
CARD / Amazon Beauty
12,101
4
256
<0.001
0.023
1.000
12,098
DIGER / Amazon Beauty
12,101
3
256
0.070
0.028
0.982
11,581
DIGER / Yelp Business
20,033
3
256
0.102
0.010
0.941
18,705
ReSOT / Amazon Instruments
6,250
4
83
0.000
0.063
1.000
6,250
Table 3: D1–D5 interface profiles for eight executable source-traced routes. Each row uses its own catalog and reference interactions. D1 is the active first-level code-symbol count; D3@1 is the fraction of retained neighbor edges sharing a depth-1 prefix under πC3 ; D5 is the number of occupied full-code leaves; and D2–D4 are rates. Per-level count sequences are released in the deterministic table snapshot. Cross-route differences describe interface states under each route’s declared catalog and protocol.
Figure 3: Mapping structure and prefix-candidate exposure. Candidate recall tests whether a held-out target shares a prefix with an item in the user’s training history. Panel A expands the source-traced and stress/reference rows, panel B shows deterministic category controls on their own scale, and marker area encodes D2 full-code aliasing. Panel C shows that associations with candidate recall change with prefix depth. Category-labeled squares are interpretation controls and are excluded from source-traced tokenizer coverage. Labels spell out the Amazon catalog used by each plotted route.
Evaluation protocol
Measured endpoint
Reporting unit
Spearman ρ
Interval / exact test
A. Same prefix construction
Per-artifact aggregate
Prefix-candidate recall
12 artifacts
0.958
[0.778, 1.000]
Controls and local references removed
Prefix-candidate recall
8 tokenizer exports
0.976
exact p=0.0004
Collapsed by shared catalog
Prefix-candidate recall
5 catalogs
0.900
exact p=0.083
B. Held-out users and fixed reranking
Per-artifact aggregate
Held-out candidate recall
8 artifacts
0.976
[0.793, 1.000]
Table 4: Evidence ladder for D3 calibration reach. Block A uses the same prefix construction for D3 and candidate recall; Block B separates the users used to construct D3 from held-out exposure and ranking; Block C samples candidates without prefix retrieval and varies the scorer or negative pool. Each row reports a Spearman association with the named endpoint. Brackets are 95% percentile bootstraps over the listed unit; rows labeled “exact” use two-sided permutation tests.
Candidate Recall@100 (%)
NDCG@20 ( ×103 )
Mapping
Original
Permuted
Popularity
Original
Permuted
Popularity
GRID / Beauty
13.20
1.91
6.85
21.18
3.16
9.41
CARD / Beauty
9.70
1.81
6.85
15.80
2.83
9.41
ReSOT / Instruments
18.75
6.24
25.25
65.09
18.64
66.98
Table 5: Fixed-budget prefix consumption on Beauty and Instruments, with 2,000 validation users per mapping and exactly 100 candidates per row. Original and permuted assignments use the same candidate constructor and ranker; permutation values average 19 fixed controls. Catalog popularity retrieves without SID prefixes. Recall is a percentage; NDCG is scaled by 103 .
Trace route
Split
Targets
Beam rows
Ambiguous rows
Path survives
Singleton credit
Gap
GRID/P5
0
500
25,000
34.0%
6.4%
3.4%
3.0 pp
GRID/P5
1
500
25,000
37.1%
5.0%
3.0%
2.0 pp
DIGER sensitivity
0
500
25,000
10.1%
6.6%
5.4%
1.2 pp
Table 6: Trained constrained-beam traces. Ambiguous-row rates use beam rows; target-path survival, singleton credit, and their difference use target traces. Singleton credit requires the target leaf to resolve to the target alone. GRID/P5 uses two disjoint explicit-target splits; DIGER is a last-event portability check. All rates are percentages and the final column is a percentage-point difference.
Figure 4: Path-to-item accounting in trained trie-constrained beams. (a) A target SID path may survive decoding yet resolve ambiguously rather than to the target item alone. (b) Target-path survival minus singleton credit is 2.0–3.0 percentage points in the two GRID/P5 folds and 1.2 points in the DIGER sensitivity route. Points show target-level rates; bars are 95% user-cluster bootstrap intervals over 1,000 resamples. DIGER uses a different last-event split and serves as a portability check.
Case
Path
Singleton
Uniform
Popularity
Gain interval
GRID / split 0
6.40
3.40
4.45
5.80
[1.20, 3.80]
GRID / split 1
5.00
3.00
3.61
3.60
[0.00, 1.40]
DIGER / last event
6.60
5.40
5.82
6.00
[0.00, 1.40]
Table 7: Resolution policies on identical top-50 paths (500 target traces per case). Hit rates are percentages. Path credit is an upper reference; singleton credit counts targets resolved without a choice. The last column is the 95% paired user-bootstrap interval for popularity resolution minus singleton credit, in percentage points, from 1,999 resamples.
Mapping and model
Items without SID
Code churn vs. 0.6
Existing-item path NDCG@20 ( ×103 )
New-item Recall@20 (%)
Handoff gate
0.6 / released model
275
0.0%
13.94 [11.53, 16.50]
–
Baseline
0.7 / released model
0
23.6%
13.84 [11.43, 16.47]
0.0 [0.0, 0.0]
Fail
0.7 / adapted, seed 2025
0
23.6%
21.00 [17.95, 24.12]
12.0 [9.4, 14.7]
Pass
0.7 / adapted, seed 2026
0
23.6%
20.57 [17.70, 23.45]
12.7 [10.1, 15.4]
Pass
0.7 / adapted, seed 2027
0
23.6%
20.26 [17.25, 23.24]
12.7 [10.1, 15.4]
Pass
Table 8: DACT mapping repair and generator handoff on one fixed 5,364-target test universe. D6 churn is measured against mapping 0.6. The preregistered gate requires common-item path NDCG@20 of at least 0.01393 (13.93 in the displayed units) and nonzero new-item Recall@20. Brackets are 95% paired user-bootstrap intervals from 4,999 stratified resamples. Path and singleton credit coincide for every evaluated target in all five states. NDCG is scaled by 103 ; new-item Recall is a percentage.
Resource surface
Status
Evidence record
Verification gate
Code and examples
Yes
Adapters, CLI, quickstart, failure fixture
Verify package contents
Route conformance
Yes
Eight C0–C5 reports; one additional auditable snapshot
Verify route admission and evidence role
Published paper evidence
Yes
Eight table snapshots, four figure records
Rebuild tables and check claims
Additional analyses
Supplement
Measurement, budget, and resolver results (Tables 5 , 7 , 11 , and 12 )
Recompute estimates and intervals; rebuild tables
D7 trace labels
Yes
125,000 deidentified labeled rows
Verify released trace rows
Upstream raw inputs
Source-bound
Inventory, hashes, regeneration notes
Check source inventory
Table 9: Resource contents, supplementary evidence, and verification entry points. Upstream raw artifacts remain at their sources.
Method, catalog, revision, config, depth, license, role
Keep as reference/control
Appendix
Table 10: Adapter admission protocol. A named route enters the shared matrix only when all six gates pass.
Mapping
D3 edge-micro
D3 frequency-weighted
Tail allocation
Tail singleton
CARD / Beauty
0.0229
0.0228
100.0
100.0
DIGER / Beauty
0.0283
0.0284
98.2
94.2
DIGER / Yelp
0.0097
0.0101
94.1
87.5
GRID / Beauty
0.0384
0.0380
95.7
82.2
LC-Rec / Instruments
0.0550
0.0678
99.8
99.5
LETTER / Instruments
0.1079
0.1281
99.8
99.5
Appendix
Table 11: Measurement checks on the eight executable exports. D3 columns are depth-1 recalls under the same C3 graph. Tail allocation is distinct codes per tail item; tail singleton is the fraction of tail items whose full code is unique across the catalog. Tail columns are percentages.
Mapping
Budget
Recall
Recall gain [interval]
NDCG gain [interval]
GRID / Beauty
50
9.05
7.77 [6.56, 8.99]
18.45 [14.18, 23.06]
GRID / Beauty
100
13.20
11.29 [9.82, 12.80]
18.02 [13.74, 22.65]
GRID / Beauty
200
17.70
14.67 [13.00, 16.37]
14.27 [10.67, 18.41]
CARD / Beauty
50
7.00
5.81 [4.73, 6.95]
13.39 [9.65, 17.71]
CARD / Beauty
100
9.70
7.89 [6.63, 9.17]
12.98 [9.36, 17.23]
CARD / Beauty
200
13.20
9.99 [8.50, 11.43]
10.53 [7.08, 14.43]
Appendix
Table 12: Budget sensitivity for the fixed prefix consumer across Beauty and Instruments. Candidate recall is a percentage; its gain is in percentage points. NDCG gains are scaled by 103 . Gains compare each original assignment with its mean permutation control; brackets are paired-user 95% intervals.
Semantic IDs (SIDs) are now a central component of generative recommendation. Current SID-based systems assign three roles to the same token sequence. Shared prefixes are intended to organize related items, the complete SID identifies an individual item, and each generated token narrows the items that can still be returned. We systematically investigate SIDs from item encoding and SID construction to autoregressive generation and final recommendation. We examine how SID construction changes item representations and how those changes affect generation. Across three Amazon domains and eight SID constructions, SID neighborhoods recover only 32.2% of the encoder's ten nearest neighbors on average. Alternative item descriptions still retrieve the corresponding item first in 99.57% of controlled cases, yet change 38.4% of exact SIDs. These results show that SIDs retain broad organization but lose much of the encoder's fine local structure, while their exact tokens are not determined by item meaning alone. This loss becomes consequential during generation. After the final semantic token, TIGER retains only 29.9% of held-out targets that were plausible recommendations before SID filtering. Motivated by these findings, we propose Item-Supported Decoding (ISD), a lightweight inference-time method that allows a user-specific item ranking to support corresponding SID prefixes before beam search discards them. The same ranking then orders the generated items. ISD requires no additional parameters or retraining of the SID constructor or decoder. We empirically show that ISD improves NDCG@10 over the corresponding SID backbone in every evaluated setting, with relative gains of up to 31.2%. Our results show that SIDs provide useful coarse item organization, but their fine boundaries should not alone determine which items remain available during generation.
Junting Wang, Xinrui He, Yunzhe Li +1
University of Illinois Urbana-Champaign Urbana, Illinois, USA
Semantic-ID-based generative recommendation represents items as sequences of shared semantic tokens, enabling token recombination beyond isolated item IDs. However, closed-world recombination does not necessarily imply temporal open-token cold-start induction, where new items enter the item catalog with unseen atomic tokens or weakly supported SID paths. In this work, we revisit SID-based generative recommendation under an absolute-time temporal protocol that separates seen and unseen targets and diagnoses the cold item reachability at the token level. Through seen/unseen-hit analysis, coldness taxonomy, and oracle-prefix probing, we show that current SID-based models can occasionally reach future items supported by observed tokens and prefixes, but struggle with unseen atomic tokens and unsupported SID paths. We further explain this boundary by interpreting SID generation as hierarchical semantic bucketing: early tokens select coarse semantic regions, while later tokens refine item-specific paths. These findings show that SID generation is compositional but not fully open-ended, and suggest future directions in more independent SID spaces, scoring-based interfaces, and dynamic textual context.
Jie Peng, Yanping Zheng, Zhewei Zhe +3
Renmin University of China · Beijing, China · Alibaba
Generative recommenders increasingly emit semantic IDs (SIDs): each item is a short sequence of hierarchical discrete codes from a residual quantizer, decoded autoregressively. Before spending scarce A/B-test, a team may decide offline which decoder or reranking variants are worth testing - a job for off-policy evaluation (OPE). We ask a simple question: can the model's own SID tree serve as the action abstraction for that OPE? Our answer has three parts. (i) Under the near-argmax logging real recommenders use, per-item OPE is hopeless - as item-level effective sample size is usually small on production logs - but marginalizing items to code-prefix clusters restores estimable support and cuts error. (ii) This gain is thanks to coarsening, not to the hierarchy specifically; but the SID tree is what makes coarsening feasible in a generative system - each cluster's mass is exactly and cheaply returned by the decoder, whereas flat clustering requires enumerating item/leaf masses that a code-only decoder does not directly expose. (iii) Resolution depth is the operative knob - coarser under scarce support - and a conditional bias bound links the coarsening bias to the quantizer's worst-case reconstruction residual and the target-logging divergence.