MuSID-KD: Multigranularity Knowledge Distillation for Generative Recommendation with Semantic IDs

Fengshi Zhang; Xiyan Ding; Shuxiao Xie; Wei Zhang

cs.AIOct 26, 2026

MuSID-KD: Multigranularity Knowledge Distillation for Generative Recommendation with Semantic IDs

Authors: Fengshi Zhang, Xiyan Ding, Shuxiao Xie, Wei Zhang

Organizations: East China Normal University Shanghai, China · Fudan University Shanghai, China

Abstract

Generative recommendation with semantic IDs (SIDs) represents each item as a sequence of discrete tokens and autoregressively predicts the next item's SID. Despite strong recommendation performance, repeated decoding over multiple SID positions incurs substantial inference cost. Knowledge distillation transfers knowledge from a powerful generative teacher to a lightweight student, yet conventional objectives do not fully account for multi-token SID generation. We identify two limitations: item-level distribution alignment may not adequately preserve the teacher's ranking of candidate groups sharing SID prefixes, while uniform alignment across SID positions may affect final item ranking differently. We propose MuSID-KD, which combines Hierarchical Candidate Distribution Alignment (HCDA) and Ranking Impact Guided Levelwise Distillation (RIGD). HCDA aggregates item probabilities according to shared SID prefixes and aligns candidate distributions at successive levels, from candidate groups to individual items. RIGD evaluates the reduction in pairwise ranking surrogate loss when one student score contribution is replaced with the teacher's contribution, and allocates supervision accordingly. Experiments on real-world recommendation datasets using three lightweight student architectures demonstrate improved recommendation accuracy and better alignment of prefix marginal distributions and pairwise item rankings, without modifying the student's online inference architecture. Code is available at https://anonymous.4open.science/r/m7k2-review-83f9/.

Figures & tables

Figure 1. Two supervision problems in distillation with SIDs, illustrated by teacher and student predictions. (a) Teacher and student may rank candidate groups formed by shared SID prefixes differently. (b) For a fixed item pair, differences between teacher and student pairwise score contributions may have opposite signs across SID positions. Two panel diagram comparing teacher and student candidate group orderings and score contributions at individual SID positions of a pair of video game items.

Figure 2. Candidate group misordering rates at intermediate SID positions when distillation aligns only candidate item probabilities. For each comparison, four groups share their preceding prefix and differ at the current token. A comparison is incorrect when the student reverses any strictly teacher ordered group pair or predicts a tie. Grouped bars reporting candidate group misordering at the first three intermediate SID levels on three recommendation datasets.

Figure 3. Proportion of item pairs ordered by the teacher whose residual

r_{d}

is positive at some SID positions and negative at others, measured for pairs crossing the top

K

recommendation boundary. Three groups of bars showing the fraction of item pairs with residuals of opposite signs across SID levels.

Figure 4. Overview of MuSID-KD. HCDA aligns distributions over candidate groups induced by shared SID prefixes, culminating in the candidate item distribution. RIGD estimates the reduction in pairwise ranking loss when replacing each student score contribution with the teacher’s value and uses positive reductions to weight Huber alignment. Both components are absent during online inference. Framework with teacher and student candidate generation on the left, hierarchical candidate distribution alignment in the center, and ranking impact guided levelwise distillation on the right.

Beauty
Category	Method	MLP				GRU				CNN
		R@5	N@5	R@10	N@10	R@5	N@5	R@10	N@10	R@5	N@5	R@10	N@10
Teacher reference	TIGER (Teacher)	0.0428	0.0283	0.0677	0.0363	0.0428	0.0283	0.0677	0.0363	0.0428	0.0283	0.0677	0.0363
	RD	0.0423	0.0274	0.0656	0.0350	0.0433 0.0433	0.0284	0.0671 0.0671	0.0362 0.0362	0.0418	0.0274	0.0647	0.0348
	RRD	0.0431	0.0285	0.0658	0.0354	0.0422	0.0275	0.0663	0.0354	0.0420 0.0420	0.0281 0.0281	0.0649	0.0352
Traditional Recommendation KD	RCE-KD	0.0433	0.0280	0.0664	0.0361	0.0431	0.0279	0.0669	0.0356	0.0408	0.0276	0.0651 0.0651	0.0354 0.0354

Table 1. Amazon results (0–1 scale; mean over seeds 42–44). R@K/N@K denote Recall@K/NDCG@K. Gain vs. Base and Gain vs. best are relative percentage improvements over SID-MLP (Base KD) and the strongest other student, respectively, computed from the displayed means; TIGER is a teacher reference and excluded from student comparisons. SD is the standard deviation of MuSID-KD over three seeds. Bold/light-red denote the best student; gray denotes the strongest other student (including ties). VG is Video Games.

	Beauty						Toys						Video Games
Variant	MLP		GRU		CNN		MLP		GRU		CNN		MLP		GRU		CNN
	R	N	R	N	R	N	R	N	R	N	R	N	R	N	R	N	R	N
CE-only	0.0660	0.0359	0.0388	0.0215	0.0288	0.0161	0.0591	0.0303	0.0333	0.0189	0.0261	0.0140	0.0925	0.0492	0.0646	0.0342	0.0375	0.0196
Base KD	0.0660	0.0356	0.0656	0.0358	0.0580	0.0319	0.0626	0.0315	0.0565	0.0296	0.0533	0.0276	0.0922	0.0488	0.0935	0.0499	0.0870	0.0460
HCDA-only	0.0666	0.0361	0.0666	0.0361	0.0609	0.0335	0.0631	0.0321	0.0594	0.0309	0.0565	0.0291	0.0929	0.0493	0.0939	0.0503	0.0895	0.0477
RIGD-only	0.0667	0.0360	0.0667	0.0362	0.0617	0.0336	0.0632	0.0320	0.0596	0.0308	0.0573	0.0296	0.0928	0.0495	0.0939	0.0504	0.0894	0.0476

Table 2. Amazon ablations (R@10 and N@10). HCDA-only and RIGD-only isolate each loss while retaining cross entropy. Bold: MuSID-KD; gray: strongest competing variant.

Figure 5. Nine diagnostics of candidate distribution alignment and item ranking across Amazon Beauty, Toys, and Video Games with MLP, GRU, and CNN. H1/H2 denote HDrift-1/2; TGD1/TGD2 denote TGD-1/2; Pair denotes Pair Softplus; and Hub-D/Hub-I denote Huber-D/I. Each metric is normalized within each dataset and student setting; darker colors indicate lower error. Nine heatmaps across Beauty, Toys, and Video Games and MLP, GRU, and CNN, with seven training objectives and nine diagnostic metrics each. Darker colors indicate lower error within each dataset, student, and metric.

Figure 6. Results using residual K-means (R-KMeans) SIDs instead of the main RQ-VAE tokenizer. Teacher and students use the same R-KMeans SIDs. Values are Recall@10 and NDCG@10 in percent; bold marks the best student per dataset and architecture. Two heatmaps of Recall at 10 and NDCG at 10 on Steam, Beauty, Toys, and Video Games for MLP, GRU, and CNN students. Columns are Base KD, SmartGR, and MuSID-KD. MuSID-KD is highest in all rows for both metrics.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Method	MLP				GRU				CNN
		R@5	N@5	R@10	N@10	R@5	N@5	R@10	N@10	R@5	N@5	R@10	N@10
Teacher	TIGER (Teacher)	4.25	2.43	6.15	3.03	4.25	2.43	6.15	3.03	4.25	2.43	6.15	3.03
	RD	4.17	2.36	5.59	2.86	3.93	2.32	5.58	2.82	3.68	2.14	5.34	2.64
	RRD	4.15	2.39	5.65	2.84	4.05	2.34	5.54	2.83	3.72	2.13	5.32	2.64
Traditional Recommendation KD	RCE-KD	4.01	2.33	5.61	2.83	3.92	2.32	5.50	2.78	3.74	2.17	5.40	2.65
	GKD	3.82	2.26	5.62	2.80	3.81	2.32	5.48	2.81	3.69	2.12	5.38	2.62

Appendix

Table S1 . Steam-100K baseline comparison (%). TIGER is a teacher reference and is excluded from student best/second-best highlighting. Bold and gray indicate the best and second-best student results, including ties.

Figure A1 . Beauty–MLP hyperparameter sensitivity: relative Recall@10 and NDCG@10 gains (%) over the MuSID-KD reference result; dashed horizontal lines mark zero relative change. Four line charts of Recall at 10 and NDCG at 10 against candidate count M, RIGD weight lambda R, rank cutoff B, and temperature tau.

Setting	MLP				GRU				CNN
	R@5	N@5	R@10	N@10	R@5	N@5	R@10	N@10	R@5	N@5	R@10	N@10
CE-only	3.62	2.26	5.54	2.82	3.68	2.27	5.46	2.79	3.51	2.11	5.39	2.69
Base KD	3.80	2.25	5.66	2.84	3.72	2.27	5.49	2.83	3.45	2.08	5.31	2.69
Leaf-only	4.03	2.35	5.55	2.86	3.93	2.30	5.60	2.82	3.71	2.13	5.28	2.67
Leaf+Pair	4.11	2.38	5.56	2.80	3.92	2.30	5.51	2.76	3.64	2.12	5.41	2.70
Leaf+RIGD	4.12	2.37	5.63	2.85	4.10	2.33	5.52	2.75	3.71	2.15	5.37	2.63

Appendix

Table S2 . Steam-100K ablations on all four metrics (%). Base KD and MuSID-KD provide reference results for the ablation comparisons. Bold indicates best student results and gray denotes second-best (ties included).

Setting	Beauty			Toys			Video Games
	MLP	GRU	CNN	MLP	GRU	CNN	MLP	GRU	CNN
CE-only	4.34/2.87	2.53/1.72	1.78/1.26	3.70/2.32	2.19/1.52	1.72/1.11	5.87/3.83	4.05/2.65	2.32/1.50
Base KD	4.29/2.82	4.16/2.81	3.80/2.54	3.74/2.35	3.67/2.33	3.28/2.10	5.88/3.83	5.91/3.89	5.53/3.58
Leaf-only	4.31/2.85	4.23/2.83	3.97/2.65	3.82/2.41	3.73/2.36	3.52/2.22	5.91/3.87	5.93/3.93	5.70/3.71
HCDA-only	4.34/2.83	4.24/2.84	3.96/2.63	3.85/2.43	3.73/2.37	3.51/2.21	5.90/3.87	5.97/3.90	5.71/3.71
Leaf+Pair	4.35/2.83	4.25/2.85	4.05/2.67	3.85/2.42	3.75/2.42	3.54/2.23	5.93/3.88	5.96/3.90	5.72/3.73

Appendix

Table S3 . Complementary Amazon ablations at @5 (%). Each entry reports Recall@5 / NDCG@5 ; the corresponding @10 results are in the main text. Bold denotes MuSID-KD.

Run	200 epochs min	Peak GPU GB
MLP Base KD	27	7.4
MLP MuSID-KD	49	9.2–9.5
CNN MuSID-KD	65	–

Appendix

Table B1 . Training, GPU memory, cache export, and serving costs. Times are minutes or ms/request (serving); training includes beam validation.

Dataset	Student	Method	Item KL	HDrift-1	HDrift-2	TGD-1	TGD-2	POV	Pair Softplus	Huber-D	Huber-I
Beauty	MLP	Base KD	0.100670	0.045033	0.082126	16.79	35.75	10.93	0.393655	0.076414	0.053440
		Leaf-only	0.051456	0.021539	0.042083	12.46	26.80	7.26	0.391118	0.050309	0.032503
		HCDA-only	0.045387	0.018498	0.036893	11.83	25.88	6.76	0.395989	0.051863	0.032359
		HCDA+Uniform	0.044023	0.019050	0.036987	11.93	26.05	6.33	0.388316	0.037520	0.023753
		HCDA+Pair	0.045503	0.018693	0.037182	11.89	26.35	6.54	0.395344	0.054633	0.033297
		RIGD-only	0.070771	0.031490	0.058785	14.42	29.17	6.68	0.343984	0.042486	0.026364

Appendix

Table D1 . Beauty diagnostics (lower is better). TGD and POV are percentages; other metrics are unscaled. Bold marks the minimum; definitions are in Appendix B.2 .

Knowledge Distillation

Knowledge distillation transfers useful behavior or representations from larger or specialized models into more compact or adaptable learners.

See all 821 papers

Knowledge Distillation across Item and Code Spaces for Parallel Generative Recommendation
Oct 23, 2026Yixuan Xu, Hailin Liu, Linxiang Peng +1Semantic ID Recommendation Generative Recommendation
The Hong Kong University of Science and Technology, Hong Kong, China · University of Science and Technology of China, Hefei, China
Understanding Semantic IDs: From Item Representation to Item Selection in Generative Recommendation
Jul 27, 2026Junting Wang, Xinrui He, Yunzhe Li +1Semantic ID Recommendation Language Model Decoding
University of Illinois Urbana-Champaign Urbana, Illinois, USA
EPIC: Explicit Posterior Item Conditioning for Semantic ID Diffusion Recommendation
Sep 3, 2026Tuan-Binh Tran, Thanh Tam Nguyen, Quoc Viet Hung Nguyen +3Semantic ID Recommendation Sequential Recommendation
College of Engineering and Computer Science VinUniversity Hanoi, Vietnam · School of Information and Communication Technology Griffith University Gold Coast, Queensland, Australia · Department of Computer Science Aalborg University Copenhagen, Denmark

Oct 23, 2026J/K move · Enter open · S save

Knowledge Distillation across Item and Code Spaces for Parallel Generative Recommendation

Yixuan Xu, Hailin Liu, Linxiang Peng, Can Yang

The Hong Kong University of Science and Technology, Hong Kong, China · University of Science and Technology of China, Hefei, China

Semantic ID Recommendation Generative Recommendation Knowledge Distillation

Open paper AskPDF arXiv

Generative recommendation with semantic IDs represents each item as a sequence of discrete codes derived from its semantic representation and predicts these codes to generate recommendations. Existing semantic-ID recommenders are typically trained with hard labels for the observed next item. Although such supervision identifies the target, it provides no explicit distributional information about alternative items conditioned on the same user history. Sequential recommenders can supply this item-level predictive knowledge, but transferring it to semantic-ID generation is challenging because the teacher and student operate in item and code spaces, respectively. We propose two complementary knowledge-distillation objectives for parallel semantic-ID recommenders, which predict all code positions simultaneously without autoregressive prefix conditioning. Item Distribution Matching (IDM) reconstructs a catalogue-level item distribution from the student's parallel code predictions and aligns it with the teacher's item distribution. Code Marginal Matching (CMM) projects the teacher's item probabilities onto each code position using fixed item-to-code assignments, then matches the resulting code marginals to the corresponding student heads. Both objectives transfer the teacher's soft distributional knowledge while preserving the student's parallel prediction structure. Experiments on benchmark datasets show that both methods consistently outperform existing state-of-the-art approaches. On average, IDM improves Recall@10 by 15.39%, while CMM improves NDCG@10 by 8.28%. These results demonstrate that distillation in either reconstructed item space or code-marginal space can substantially strengthen hard-label supervision for parallel generative recommendation.

Abstract

Knowledge Distillation

Explore similar work

Knowledge Distillation across Item and Code Spaces for Parallel Generative Recommendation

Understanding Semantic IDs: From Item Representation to Item Selection in Generative Recommendation

EPIC: Explicit Posterior Item Conditioning for Semantic ID Diffusion Recommendation