The Vocabulary Is Already Here: Adapting Pretrained Interpretable Features for Generative Recommendation
Authors: Ziluowen Luo, Ruochen Liu, Jun Yin, Hao Yan, Zhengxuan Hu, Xinze Shi, Chaozhuo Li, Chengqi Zhang, Senzhang Wang
Organizations: Central South University Changsha, Hunan, China · The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China · Hong Kong Polytechnic University Hongkong, China · Central South University ChangSha, Hunan, China · Queen Mary University of London London, UK · Microsoft China · Central South University Changsha, China
Generative recommendation (GR) predicts the next item through its semantic ID (SID). Existing methods commonly fit SID vocabularies to pretrained language model (PLM) representations of item descriptions, incurring additional training costs and making vocabulary coverage dependent on the fitting domains. We observe that pretrained sparse autoencoders expose PLM features with human-interpretable semantic annotations, allowing feature indices to serve as a reusable SID vocabulary. However, reusing this vocabulary requires preserving item distinctions under compact token budgets and learning to favor target items over candidates with shared dominant features. It also requires extending feature interpretability from item content to retrieval decisions. To address these challenges, we propose API, a framework that Adapts the Pretrained Interpretable vocabulary for effective and explainable GR without additional vocabulary learning. Within API, Pretrained Interpretable IDs (PIDs) retain selected feature indices and relative activation strengths for item discrimination. With PIDs as supervision, the Semantic Prediction Adapter (SPA) learns user preferences from interaction histories through joint feature-index and strength prediction. SPA combines both predictions to score candidate items, with item-level supervision encouraging the observed next item to outrank its alternatives. API further explains candidate score differences through feature contributions paired with their pretrained semantic annotations. Extensive experiments show that API achieves competitive performance across backbones and decoding schemes, while preserving item distinctions under compact token budgets and enabling explainable retrieval.
Figures & tables
Figure 1. The vocabulary is already here. (a–c) Pretrained SAEs enable vocabulary sharing without extra training. (d) Distinct items can share the same dominant feature indices.
Figure 2. Overview of API. (a) PID construction selects pretrained features and retains their relative activation strengths and semantic annotations. (b) SPA learns user preferences through feature-index and strength prediction with item-level supervision. (c) Candidate score differences are decomposed into semantic feature contributions for retrieval explanation.
Beauty
Sports
Toys
Method
R@5
R@10
N@5
N@10
R@5
R@10
N@5
N@10
R@5
R@10
N@5
N@10
GRU4Rec
0.0433
0.0646
0.0304
0.0372
0.0222
0.0371
0.0141
0.0189
0.0384
0.0543
0.0272
0.0324
Bert4Rec
0.0281
0.0434
0.0184
0.0233
0.0129
0.0224
0.0079
0.0110
0.0201
0.0313
0.0133
0.0168
Tiger (2023)
0.0464
0.0662
0.0305
0.0369
0.0237
0.0341
0.0156
0.0189
0.0402
0.0584
0.0263
0.0322
LCRec (2024)
0.0228
0.0362
0.0159
0.0197
0.0082
0.0133
0.0073
0.0089
0.0174
0.0285
0.0119
0.0157
RPG (2025)
0.0322
0.0502
0.0210
0.0269
0.0176
0.0266
0.0124
0.0153
0.0306
0.0477
0.0192
0.0247
Table 1. Overall recommendation performance. Best and second-best results are shown in bold and underlined , respectively.
Back.
Generation
Tokenization
Beauty
Sports
Toys
R@5
R@10
N@5
N@10
R@5
R@10
N@5
N@10
R@5
R@10
N@5
N@10
T5
AR
Random
0.0310
0.0454
0.0216
0.0262
0.0082
0.0111
0.0051
0.0060
0.0188
0.0250
0.0140
0.0161
RVQ
0.0464
0.0662
0.0305
0.0369
0.0237
0.0341
0.0156
0.0189
0.0402
0.0584
0.0263
0.0322
RQVAE
0.0402
0.0602
0.0269
0.0334
0.0215
0.0310
0.0139
0.0170
0.0344
0.0492
0.0240
0.0290
RKMeans
0.0441
0.0667
0.0305
0.0378
0.0217
0.0330
0.0145
0.0181
0.0384
0.0564
0.0256
0.0314
HiD-VAE
0.0419
0.0644
0.0283
0.0355
0.0215
0.0328
0.0138
0.0175
0.0388
0.0595
0.0257
0.0324
Table 2. Recommendation performance on three datasets with T5 and GPT-2 backbones. K counts semantic tokens. R@ k and N@ k denote Recall@ k and NDCG@ k . The PID with the SPA constitutes API. For each backbone and decoding strategy, bold and underlined values indicate the best and second-best results, respectively, including ties.
Figure 3. Vocabulary reuse across domains. Collision rates and downstream Recall@5 are reported for each source–target pair; shaded columns mark the source domain of data-trained tokenizers.
Dataset
Selection
K=4
K=8
K=16
K=24
K=32
Beauty
Native SAE
77.37
42.72
11.11
2.86
1.23
Random ID
2.50
0.00
0.00
0.00
1.23
PID
15.16
3.33
1.26
1.19
1.23
Sports
Native SAE
79.96
51.77
19.98
6.30
3.44
Random ID
3.67
0.00
0.00
0.00
3.44
PID
21.48
7.71
3.43
3.20
3.44
Table 3. Collision-affected items (%) across token budgets.
Figure 4. Interpretability and controlled steering with P-SAE features. Left: The labeled f4260 by the LLMs, title matches to the meanings and top-activating items. K is the PID budget; lift uses the catalog baseline ( ≈12.5% ). Right: changes in target items exposure@10 under activation steering, relative to the non-target. Shaded bands show the min–max range over three runs.
Figure 5. Content alignment across semantic-token budgets, measured by nearest-neighbor overlap@20 against title-based and PLM-based references. Higher values indicate greater agreement with the reference neighborhoods.
Table 5. Supervision ablation of SPA under T5-style parallel prediction with K=8 features.
Figure 6. Retrieval explanation and tokenization efficiency. (a) A Toys case showing candidate score-gap contributions grouped by PID position. (b) Projected umulative tokenization time for three tasks, counting UTGRec pretraining once.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Users
Items
Interactions
Train items
Beauty
22,363
12,101
198,502
12,068
Sports
35,598
18,357
296,337
18,284
Toys
19,412
11,924
167,597
11,876
Appendix
Table 6. Dataset statistics after pre-processing.
Setting
T5 AR/PR
GPT-2 PR
GPT-2 AR
Layers (E/D)
4/4
0/4
0/4
Width / heads
128/6
448/8
128/8
K/V per head
64
56
16
FFN widths
1,024/1,024
1,792
1,024
Activation
ReLU
SiLU
GELU
Dropout
0.15
0.15
0.15
Appendix
Table 7. Recommendation backbones. E/D: encoder/decoder layers. GPT-2 AR and PR are separate capacity cohorts.
Figure 7. Vocabulary reuse from Sports. Collision-affected items (%) are reported; shaded columns mark the source domain used to fit learned tokenizers. PID reuses the frozen P-SAE dictionary with target-catalog IDF.
Figure 8. Collision-affected items (%) across semantic-token budgets on Beauty, Sports, and Toys. Activation–IDF SAE denotes PID; random-positive selection samples active SAE features. Strengths and disambiguation suffixes are excluded from collision statistics.
Generative recommendation models user histories and item catalogs to generate personalized items, lists, or queries rather than only scoring candidates.