SIDScope: A Diagnostic Resource for Semantic-ID Interfaces in Generative Recommendation

Jiandong Ding; Huijie Qin; Tiandeng Wu; Yi Cao

cs.IRAug 19, 2026

SIDScope: A Diagnostic Resource for Semantic-ID Interfaces in Generative Recommendation

Authors: Jiandong Ding, Huijie Qin, Tiandeng Wu, Yi Cao

Organizations: Huawei Technologies Co., Ltd., Shanghai, China

Abstract

Semantic ID tokenizers assign code sequences to catalog items, which generative recommenders predict and resolve back to items. Reusing a released mapping requires checking its code organization, item resolution, and compatibility with an existing model after a refresh. SIDScope provides these checks through source traced adapters, mapping diagnostics, candidate retrieval tests, and generated trace analysis. Its inventory contains nine exports from seven tokenizer families across Amazon and Yelp, including eight executable routes and one auditable snapshot. The analyses establish three distinctions. First, code allocation and item addressability differ: in GRID, tail items occupy 95.7 distinct codes per 100 items, yet only 82.2 percent have globally unique codes. Second, prefix alignment calibrates retrieval operations that use prefixes. With fixed candidate budgets and ranking, original assignments on Beauty and Instruments expose more targets than controls preserving code capacity and coarse popularity. Resolving identical generated paths also shows that an explicit item selection rule recovers part of the gap between target path survival and singleton credit. Third, a released refresh repairs missing codes, while model adaptation separately restores new item recall. SIDScope supplies code, diagnostic summaries, and verification tools to compare mappings, evaluate their use in retrieval and generation, and test model reuse after changes.

Figures & tables

Figure 1: SIDScope connects source-traced inputs to mapping diagnostics, retrieval and generation tests, and reproducible outputs. D1 measures code-space use, D2 shared-address risk, D3 prefix–behavior alignment, D4 popularity allocation, D5 trie structure, D6 mapping change, and D7 path-to-item resolution. Source, configuration, and license records bind these measurements to the mapping being tested.

Question	SIDInspector V0	SIDScope extension	Where shown
What is inspected?	Static item-to-code mappings	Artifact states spanning mappings, exposure, traces, and package records	Figure 1 ; Sections 3 and 7
Which artifacts enter?	Reconstructed controls and snapshots	Source-traced named routes separated from stress and control rows	Tables 2 and 10
What can D3 support?	D1–D5 diagnostic tables	Prefix calibration, fixed-budget assignment controls, and measurement checks	Tables 4 , 5 , and 11
What do traces add?	D7 input hook	Constraint-aware labels and resolver comparisons on identical generated paths	Tables 6 and 7 ; Figure 4
How broad is the matrix?	Musical worked example	Nine named exports, seven families, two data ecosystems, controls, and trained traces	Tables 2 , 3 , and 6
What changes over time?	Mapping snapshots	Paired D6 refresh audit, mapping repair, and generator handoff	Table 8

Table 1: Scientific and resource extension beyond SIDInspector V0. The final column points to the evidence introduced and evaluated in this article.

Figure 2: One mapping, distinct diagnostic questions. Items $i_{1}$ and $i_{2}$ share full code $\langle 1,1,1\rangle$ , creating a D2 collision. Items $i_{4}$ and $i_{5}$ co-occur and share prefix $\langle 2,1\rangle$ but have different leaves, illustrating the relation measured by D3. All values are illustrative.

Tokenizer export	Data source	Source evidence	Mapped items	SID depth
Source-traced named tokenizer exports
ReSID	Amazon / Musical	Tracked snapshot	23,742	3
ReSID-GAOQ	Amazon / Video Games	Code-derived export	24,685	3
GRID	Amazon / Beauty (P5)	Rebuilt tokenizer	12,101	3
CARD	Amazon / Beauty (P5)	Code-derived export	12,101	4
DIGER	Amazon / Beauty (P5)	Checkpoint export	12,101	3

Table 2: Artifact inventory. Eight executable tokenizer routes pass C0–C5; ReSID/Musical is a source-traced auditable snapshot. Reference and control rows calibrate probe behavior but are not counted as named tokenizer coverage. Data-source labels normalize the ecosystem and subset name; each mapped-item count refers to that export’s pinned item universe. Source evidence records the route used to inspect or reconstruct the mapping.

Route	Items	Depth	D1 symbols	D2 collision	D3@1	D4 tail code ratio	D5 leaves
ReSID-GAOQ / Amazon Video	24,685	3	32	0.000	0.213	1.000	24,685
GRID / Amazon Beauty	12,101	3	256	0.205	0.038	0.957	10,619
CARD / Amazon Beauty	12,101	4	256	$<0.001$	0.023	1.000	12,098
DIGER / Amazon Beauty	12,101	3	256	0.070	0.028	0.982	11,581
DIGER / Yelp Business	20,033	3	256	0.102	0.010	0.941	18,705
ReSOT / Amazon Instruments	6,250	4	83	0.000	0.063	1.000	6,250

Table 3: D1–D5 interface profiles for eight executable source-traced routes. Each row uses its own catalog and reference interactions. D1 is the active first-level code-symbol count; D3@1 is the fraction of retained neighbor edges sharing a depth-1 prefix under

\pi_{\mathrm{C3}}

; D5 is the number of occupied full-code leaves; and D2–D4 are rates. Per-level count sequences are released in the deterministic table snapshot. Cross-route differences describe interface states under each route’s declared catalog and protocol.

Figure 3: Mapping structure and prefix-candidate exposure. Candidate recall tests whether a held-out target shares a prefix with an item in the user’s training history. Panel A expands the source-traced and stress/reference rows, panel B shows deterministic category controls on their own scale, and marker area encodes D2 full-code aliasing. Panel C shows that associations with candidate recall change with prefix depth. Category-labeled squares are interpretation controls and are excluded from source-traced tokenizer coverage. Labels spell out the Amazon catalog used by each plotted route.

Evaluation protocol	Measured endpoint	Reporting unit	Spearman $\rho$	Interval / exact test
A. Same prefix construction
Per-artifact aggregate	Prefix-candidate recall	12 artifacts	0.958	[0.778, 1.000]
Controls and local references removed	Prefix-candidate recall	8 tokenizer exports	0.976	exact $p=0.0004$
Collapsed by shared catalog	Prefix-candidate recall	5 catalogs	0.900	exact $p=0.083$
B. Held-out users and fixed reranking
Per-artifact aggregate	Held-out candidate recall	8 artifacts	0.976	[0.793, 1.000]

Table 4: Evidence ladder for D3 calibration reach. Block A uses the same prefix construction for D3 and candidate recall; Block B separates the users used to construct D3 from held-out exposure and ranking; Block C samples candidates without prefix retrieval and varies the scorer or negative pool. Each row reports a Spearman association with the named endpoint. Brackets are 95% percentile bootstraps over the listed unit; rows labeled “exact” use two-sided permutation tests.

	Candidate Recall@100 (%)			NDCG@20 ( $\times 10^{3}$ )
Mapping	Original	Permuted	Popularity	Original	Permuted	Popularity
GRID / Beauty	13.20	1.91	6.85	21.18	3.16	9.41
CARD / Beauty	9.70	1.81	6.85	15.80	2.83	9.41
ReSOT / Instruments	18.75	6.24	25.25	65.09	18.64	66.98

Table 5: Fixed-budget prefix consumption on Beauty and Instruments, with 2,000 validation users per mapping and exactly 100 candidates per row. Original and permuted assignments use the same candidate constructor and ranker; permutation values average 19 fixed controls. Catalog popularity retrieves without SID prefixes. Recall is a percentage; NDCG is scaled by

10^{3}

Trace route	Split	Targets	Beam rows	Ambiguous rows	Path survives	Singleton credit	Gap
GRID/P5	0	500	25,000	34.0%	6.4%	3.4%	3.0 pp
GRID/P5	1	500	25,000	37.1%	5.0%	3.0%	2.0 pp
DIGER sensitivity	0	500	25,000	10.1%	6.6%	5.4%	1.2 pp

Table 6: Trained constrained-beam traces. Ambiguous-row rates use beam rows; target-path survival, singleton credit, and their difference use target traces. Singleton credit requires the target leaf to resolve to the target alone. GRID/P5 uses two disjoint explicit-target splits; DIGER is a last-event portability check. All rates are percentages and the final column is a percentage-point difference.

Figure 4: Path-to-item accounting in trained trie-constrained beams. (a) A target SID path may survive decoding yet resolve ambiguously rather than to the target item alone. (b) Target-path survival minus singleton credit is 2.0–3.0 percentage points in the two GRID/P5 folds and 1.2 points in the DIGER sensitivity route. Points show target-level rates; bars are 95% user-cluster bootstrap intervals over 1,000 resamples. DIGER uses a different last-event split and serves as a portability check.

Case	Path	Singleton	Uniform	Popularity	Gain interval
GRID / split 0	6.40	3.40	4.45	5.80	[1.20, 3.80]
GRID / split 1	5.00	3.00	3.61	3.60	[0.00, 1.40]
DIGER / last event	6.60	5.40	5.82	6.00	[0.00, 1.40]

Table 7: Resolution policies on identical top-50 paths (500 target traces per case). Hit rates are percentages. Path credit is an upper reference; singleton credit counts targets resolved without a choice. The last column is the 95% paired user-bootstrap interval for popularity resolution minus singleton credit, in percentage points, from 1,999 resamples.

Mapping and model	Items without SID	Code churn vs. 0.6	Existing-item path NDCG@20 ( $\times 10^{3}$ )	New-item Recall@20 (%)	Handoff gate
0.6 / released model	275	0.0%	13.94 [11.53, 16.50]	–	Baseline
0.7 / released model	0	23.6%	13.84 [11.43, 16.47]	0.0 [0.0, 0.0]	Fail
0.7 / adapted, seed 2025	0	23.6%	21.00 [17.95, 24.12]	12.0 [9.4, 14.7]	Pass
0.7 / adapted, seed 2026	0	23.6%	20.57 [17.70, 23.45]	12.7 [10.1, 15.4]	Pass
0.7 / adapted, seed 2027	0	23.6%	20.26 [17.25, 23.24]	12.7 [10.1, 15.4]	Pass

Table 8: DACT mapping repair and generator handoff on one fixed 5,364-target test universe. D6 churn is measured against mapping 0.6. The preregistered gate requires common-item path NDCG@20 of at least 0.01393 (13.93 in the displayed units) and nonzero new-item Recall@20. Brackets are 95% paired user-bootstrap intervals from 4,999 stratified resamples. Path and singleton credit coincide for every evaluated target in all five states. NDCG is scaled by

10^{3}

; new-item Recall is a percentage.

Resource surface	Status	Evidence record	Verification gate
Code and examples	Yes	Adapters, CLI, quickstart, failure fixture	Verify package contents
Route conformance	Yes	Eight C0–C5 reports; one additional auditable snapshot	Verify route admission and evidence role
Published paper evidence	Yes	Eight table snapshots, four figure records	Rebuild tables and check claims
Additional analyses	Supplement	Measurement, budget, and resolver results (Tables 5 , 7 , 11 , and 12 )	Recompute estimates and intervals; rebuild tables
D7 trace labels	Yes	125,000 deidentified labeled rows	Verify released trace rows
Upstream raw inputs	Source-bound	Inventory, hashes, regeneration notes	Check source inventory

Table 9: Resource contents, supplementary evidence, and verification entry points. Upstream raw artifacts remain at their sources.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Gate	What must be true	Evidence checked	Failure action
C0	Source identity and reuse terms are explicit	URL, revision, derivation path, license, redistribution policy	Hold as discovered source
C1	SID tables are internally coherent	Required columns, level reconstruction, dataset, depth, item count	Reject normalized mapping
C2	Mapping joins metadata and interactions	Joined-item counts and missing-coverage report	Hold as partial route
C3	Bounded diagnostics execute	D1–D5 outputs with declared limits	Exclude from metric matrix
C4	Inputs are pinned and replayable	Command, runtime, evidence paths, SHA-256 hashes	Exclude from release claim
C5	Inventory identity matches the matrix role	Method, catalog, revision, config, depth, license, role	Keep as reference/control

Appendix

Table 10: Adapter admission protocol. A named route enters the shared matrix only when all six gates pass.

Mapping	D3 edge-micro	D3 frequency-weighted	Tail allocation	Tail singleton
CARD / Beauty	0.0229	0.0228	100.0	100.0
DIGER / Beauty	0.0283	0.0284	98.2	94.2
DIGER / Yelp	0.0097	0.0101	94.1	87.5
GRID / Beauty	0.0384	0.0380	95.7	82.2
LC-Rec / Instruments	0.0550	0.0678	99.8	99.5
LETTER / Instruments	0.1079	0.1281	99.8	99.5

Appendix

Table 11: Measurement checks on the eight executable exports. D3 columns are depth-1 recalls under the same C3 graph. Tail allocation is distinct codes per tail item; tail singleton is the fraction of tail items whose full code is unique across the catalog. Tail columns are percentages.

Mapping	Budget	Recall	Recall gain [interval]	NDCG gain [interval]
GRID / Beauty	50	9.05	7.77 [6.56, 8.99]	18.45 [14.18, 23.06]
GRID / Beauty	100	13.20	11.29 [9.82, 12.80]	18.02 [13.74, 22.65]
GRID / Beauty	200	17.70	14.67 [13.00, 16.37]	14.27 [10.67, 18.41]
CARD / Beauty	50	7.00	5.81 [4.73, 6.95]	13.39 [9.65, 17.71]
CARD / Beauty	100	9.70	7.89 [6.63, 9.17]	12.98 [9.36, 17.23]
CARD / Beauty	200	13.20	9.99 [8.50, 11.43]	10.53 [7.08, 14.43]

Appendix

Table 12: Budget sensitivity for the fixed prefix consumer across Beauty and Instruments. Candidate recall is a percentage; its gain is in percentage points. NDCG gains are scaled by

10^{3}

. Gains compare each original assignment with its mean permutation control; brackets are paired-user 95% intervals.

Semantic ID Recommendation

Semantic identifier recommendation uses discrete, content-aware item codes to retrieve and rank items, especially in generative systems.

See all 38 papers

Understanding Semantic IDs: From Item Representation to Item Selection in Generative Recommendation
Jul 27, 2026Junting Wang, Xinrui He, Yunzhe Li +1Semantic ID Recommendation Language Model Decoding
University of Illinois Urbana-Champaign Urbana, Illinois, USA
Can Generative Recommendation Reach Cold Items? A Temporal Perspective on Semantic-ID Generation
Jul 23, 2026Jie Peng, Yanping Zheng, Zhewei Zhe +3Semantic ID Recommendation Sequential Recommendation
Renmin University of China · Beijing, China · Alibaba
Off-Policy Evaluation for Semantic ID Recommenders: Does the Model's Own Code Hierarchy Help?
Aug 28, 2026Artem BetleiOff-Policy Evaluation Semantic ID Recommendation
Criteo AI Lab Grenoble, France

Jul 27, 2026J/K move · Enter open · S save

Understanding Semantic IDs: From Item Representation to Item Selection in Generative Recommendation

Junting Wang, Xinrui He, Yunzhe Li, Hari Sundaram

University of Illinois Urbana-Champaign Urbana, Illinois, USA

Semantic ID Recommendation Language Model Decoding Generative Recommendation

Open paper AskPDF arXiv

Semantic IDs (SIDs) are now a central component of generative recommendation. Current SID-based systems assign three roles to the same token sequence. Shared prefixes are intended to organize related items, the complete SID identifies an individual item, and each generated token narrows the items that can still be returned. We systematically investigate SIDs from item encoding and SID construction to autoregressive generation and final recommendation. We examine how SID construction changes item representations and how those changes affect generation. Across three Amazon domains and eight SID constructions, SID neighborhoods recover only 32.2% of the encoder's ten nearest neighbors on average. Alternative item descriptions still retrieve the corresponding item first in 99.57% of controlled cases, yet change 38.4% of exact SIDs. These results show that SIDs retain broad organization but lose much of the encoder's fine local structure, while their exact tokens are not determined by item meaning alone. This loss becomes consequential during generation. After the final semantic token, TIGER retains only 29.9% of held-out targets that were plausible recommendations before SID filtering. Motivated by these findings, we propose Item-Supported Decoding (ISD), a lightweight inference-time method that allows a user-specific item ranking to support corresponding SID prefixes before beam search discards them. The same ranking then orders the generated items. ISD requires no additional parameters or retraining of the SID constructor or decoder. We empirically show that ISD improves NDCG@10 over the corresponding SID backbone in every evaluated setting, with relative gains of up to 31.2%. Our results show that SIDs provide useful coarse item organization, but their fine boundaries should not alone determine which items remain available during generation.

Abstract

Semantic ID Recommendation

Explore similar work

Understanding Semantic IDs: From Item Representation to Item Selection in Generative Recommendation

Can Generative Recommendation Reach Cold Items? A Temporal Perspective on Semantic-ID Generation

Off-Policy Evaluation for Semantic ID Recommenders: Does the Model's Own Code Hierarchy Help?