Generative recommendation promises to simplify conventional multi-stage recommender systems by generating item Semantic IDs (SIDs) autoregressively. However, two obstacles limit its effective- ness in industrial settings: interaction-only supervision underuses the semantic knowledge encoded in pretrained language models, while beam-search likelihood ranks token sequences rather than directly estimating item relevance. To address these limitations, we propose HiGenTrans, a three-level framework that introduces LLM knowledge at the input, representation, and decision levels. First, Global Semantic Priming (GSP) adds an LLM-derived user representation to the beginning-of-sequence embedding, provid- ing a semantic prior before SID generation. Second, Hierarchi- cal Representation Transfer (HRT) aligns intermediate LLM representations with depth-matched GenRec layers through mean- centered projections and gated residual connections. Finally, Multi- Signal Decision Calibration (MDC) uses a lightweight mixture- of-experts calibrator to combine semantic relevance, generation confidence, and candidate-set context when reranking beam-search outputs. All semantic features are computed only from the observ- able interaction prefix, preventing target leakage, and can be cached offline. Experiments on three public benchmarks show that HiGen- Trans improves Recall@10 by up to 12.9% and NDCG@10 by up to 33.7% over the strongest baselines. These results demonstrate the value of transferring LLM knowledge at complementary stages of the generative recommendation pipeline.
Figures & tables
Figure 1. Motivation and overview of HiGenTrans. Existing GenRec and LLM4Rec methods suffer from shallow semantic fusion and misalignment between generation likelihood and item relevance. HiGenTrans addresses these limitations through Global Semantic Prior (GSP), Hierarchical Representation Transfer (HRT), and Multi-signal Decision Calibration (MDC), enabling richer semantic integration and more accurate recommendation ranking.
Figure 2. Overview of \ourname , a three-level semantic knowledge transfer framework for generative recommendation. \MODa (GSP) incorporates global user semantics into the BOS token. \MODb (HRT) injects multi-level LLM representations into the generative backbone. \MODc (MDC) employs MoE-based multi-signal calibration to refine candidate rankings.
Figure 3. SID-based compression for hierarchical transfer. Multi-level LLM embeddings are compressed into SIDs and reconstructed for stage-wise semantic transfer.
Dataset
#Users
#Items
#Interactions
Density (%)
Avg. Len.
Beauty
22,363
12,101
198,502
0.073
8.88
Sports
35,598
18,357
296,337
0.045
8.32
Toys
19,412
11,924
167,597
0.072
8.63
Table 1. Statistics of the datasets used in our experiments.
Beauty
Sports
Toys
Paradigm
Model
R@1
R@5
R@10
N@5
N@10
R@1
R@5
R@10
N@5
N@10
R@1
R@5
R@10
N@5
N@10
Recurrent
GRU4Rec
.0125
.0343
.0512
.0235
.0290
.0043
.0160
.0262
.0102
.0134
.0091
.0239
.0358
.0167
.0205
Attention-based
SASRec
.0077
.0566
.0880
.0327
.0428
.0046
.0307
.0492
.0180
.0239
.0059
.0620
.0927
.0352
.0450
HSTU
.0118
.0563
.0868
.0346
.0445
.0076
.0260
.0427
.0167
.0220
.0094
.0630
.0919
.0370
.0464
BERT4Rec
.0050
.0230
.0413
.0138
.0197
.0033
.0147
.0259
.0090
.0126
.0043
.0241
.0414
.0139
.0194
Generative Rec.
TIGER
.0113
.0391
.0626
.0253
.0329
.0077
.0250
.0409
.0164
.0216
.0128
.0360
.0565
.0245
.0311
Table 2. Overall performance comparison on three Amazon datasets. Best results are highlighted in bold , and second-best results are underlined .
Method
R@1
R@5
N@10
R@10
Base LLM4GenRec
.0229
.0581
.0480
.0803
+Global Semantic Priming
.0269
.0696
.0569
.0958
+Hierarchical Transfer
.0269
.0707
.0577
.0967
+Decision Calibration
.0319
.0753
.0628
.1010
Full Model
.0372
.0818
.0674
.1047
Table 3. Ablation study of HiGenTrans on the Toys dataset.
Candidate-ranking AUC
Stage-wise Gain
Overall
Dataset
S1
S2
S3
S4
Δ2−1
Δ3−2
Δ4−3
Δ4−1
Toys
0.5788
0.5863
0.5956
0.5953
+0.75
+0.93
-0.03
+1.65
Beauty
0.5434
0.5414
0.5475
0.5530
-0.20
+0.61
+0.55
+0.96
Sports
0.5302
0.5335
0.5481
0.5440
+0.33
+1.46
-0.41
+1.38
Table 4. Stage-wise discriminative ability of LLM representations measured by candidate-ranking AUC. Δ indicates absolute AUC changes in percentage points.
Users
Hit@10
Gain (%)
Range (%)
Δ100 (pp)
100%
0.0917
+8.86
—
—
95%
0.0908
+7.84
7.17–8.44
− 0.09 ↓
90%
0.0906
+7.59
7.48–7.70
− 0.11 ↓
80%
0.0903
+7.24
6.58–8.01
− 0.14 ↓
Table 5. Robustness to reduced training data on Beauty.
Popularity
Hit@10
NDCG@10
Base / Ours
Gain
Base / Ours
Gain
All
.0932 / .1047
+12.3%
.0544 / .0674
+23.9%
≤ 2
.0330 / .0400
+21.3%
.0239 / .0341
+42.8%
3–5
.0591 / .0770
+30.4%
.0327 / .0536
+64.1%
6–15
.0964 / .1097
+13.7%
.0575 / .0690
+20.2%
≥ 16
.1341 / .1412
+5.2%
.0768 / .0867
+13.0%
Table 6. Performance across item popularity groups on Toys. Gains denote relative improvements (%) over the baseline. Darker green indicates larger gains.
History
Hit@10
NDCG@10
Base / Ours
Gain
Base / Ours
Gain
All
.0932 / .1047
+12.3%
.0544 / .0674
+23.9%
3
.1023 / .1138
+11.3%
.0612 / .0736
+20.3%
4–5
.1001 / .1108
+10.7%
.0582 / .0719
+23.6%
6–9
.0739 / .0849
+14.9%
.0419 / .0554
+32.4%
≥ 10
.0841 / .0978
+16.3%
.0479 / .0598
+25.0%
Table 7. Performance across user history lengths on Toys. Gains denote relative improvements (%) over the baseline. Darker green indicates larger gains.
Figure 4. Ground-truth ranking transitions before and after MDC. Each cell shows the transition percentage and user count. Dashed lines indicate the top-10 boundary.
Domain 1
Domain 2
Method
SID 1
SID 2
SID 3
SID 1
SID 2
SID 3
Baseline
0.1360
0.4789
0.8038
0.2149
0.4698
0.6083
Ours
0.6178
0.6155
0.8226
0.8347
0.5797
0.6383
Δ
+0.4818
+0.1366
+0.0188
+0.6198
+0.1099
+0.0300
Table 8. SID prediction performance across different hierarchical levels in two industrial cross-domain recommendation scenarios. Δ denotes absolute improvements, with darker green indicating larger gains.
Recent advances in generative recommendation have leveraged pretrained LLMs by formulating sequential recommendation as autoregressive generation over a unified token space comprising language tokens and itemic identifiers, where each item is represented by a compact sequence of discrete tokens, namely Semantic IDs (SIDs). This SID-based formulation enables efficient decoding over large-scale item corpora and provides a natural interface for LLM-based recommenders to leverage rich world knowledge. Meanwhile, breakthroughs in LLM reasoning motivate reasoning-enhanced recommendation, yet effective reasoning over SIDs remains underexplored and challenging. Itemic tokens are not natively meaningful to LLMs; moreover, recommendation-oriented SID reasoning is hard to evaluate, making high-quality supervision scarce. To address these challenges, we propose SIDReasoner, a two-stage framework that elicits reasoning over SIDs by strengthening SID--language alignment to unlock transferable LLM reasoning, rather than relying on large amounts of recommendation-specific reasoning traces. Concretely, SIDReasoner first enhances SID-language alignment via multi-task training on an enriched SID-centered corpus synthesized by a stronger teacher model, grounding itemic tokens in diverse semantic and behavioral contexts. Building on this enhanced alignment, SIDReasoner further improves recommendation reasoning through outcome-driven reinforced optimization, which guides the model toward effective reasoning trajectories without requiring explicit reasoning annotations. Extensive experiments on three real-world datasets demonstrate the effectiveness of our reasoning-augmented SID-based generative recommendation. Beyond accuracy, the results highlight the broader potential of large reasoning models for generative recommendation, including improved interpretability and cross-domain generalization.
Yingzhi He, Yan Sun, Junfei Tan +6
National University of Singapore Singapore, Singapore · University of Science and Technology of China Hefei, China · Tencent Inc. Shenzhen, China
Generative recommendation with semantic IDs represents each item as a sequence of discrete codes derived from its semantic representation and predicts these codes to generate recommendations. Existing semantic-ID recommenders are typically trained with hard labels for the observed next item. Although such supervision identifies the target, it provides no explicit distributional information about alternative items conditioned on the same user history. Sequential recommenders can supply this item-level predictive knowledge, but transferring it to semantic-ID generation is challenging because the teacher and student operate in item and code spaces, respectively. We propose two complementary knowledge-distillation objectives for parallel semantic-ID recommenders, which predict all code positions simultaneously without autoregressive prefix conditioning. Item Distribution Matching (IDM) reconstructs a catalogue-level item distribution from the student's parallel code predictions and aligns it with the teacher's item distribution. Code Marginal Matching (CMM) projects the teacher's item probabilities onto each code position using fixed item-to-code assignments, then matches the resulting code marginals to the corresponding student heads. Both objectives transfer the teacher's soft distributional knowledge while preserving the student's parallel prediction structure. Experiments on benchmark datasets show that both methods consistently outperform existing state-of-the-art approaches. On average, IDM improves Recall@10 by 15.39%, while CMM improves NDCG@10 by 8.28%. These results demonstrate that distillation in either reconstructed item space or code-marginal space can substantially strengthen hard-label supervision for parallel generative recommendation.
Yixuan Xu, Hailin Liu, Linxiang Peng +1
The Hong Kong University of Science and Technology, Hong Kong, China · University of Science and Technology of China, Hefei, China
Generative recommendation commonly represents items using fixed-length semantic identifiers (SIDs) constructed through clustering and quantization. However, these artificial codes may overcompress item semantics, remain misaligned with pretrained LLM vocabularies, and require costly autoregressive decoding. In light of this, we propose VaLiDRec, a generative recommendation framework based on variable-length, LLM-aligned semantic identifiers. VaLiDRec constructs SIDs directly from informative native LLM vocabulary tokens via token importance estimation, semantic-quality-aware pruning, and collision-aware refinement, allowing identifier lengths to adapt to item semantic complexity. To model user preferences, VaLiDRec incorporates graph-aware soft prompts and reformulates recommendation as token-set prediction with token-level item scoring, eliminating autoregressive SID generation and beam search. Experiments on four real-world datasets show that VaLiDRec consistently outperforms strong sequential and generative recommendation baselines across all evaluation metrics. It further achieves superior zero-shot item cold-start performance and 87.49× faster inference than LC-Rec. These results demonstrate that LLM-native variable-length semantic identifiers provide a more expressive and efficient paradigm for generative recommendation.
Shutong Qiao, Wei Yuan, Tong Chen +3
University of Queensland Brisbane, Australia · Computer Network Information Center, Chinese Academy of Sciences Beijing, China · Griffith University Gold Coast, Australia