Search-Aware On-Policy Self-Distillation with Sibling Alignment for Generative Recommendation
Organizations: Kuaishou Technology, Beijing, China · University of Amsterdam, Amsterdam, Netherlands · Huazhong University of Science and Technology, Wuhan, China
Abstract
Generative recommendation retrieves items by autoregressively generating semantic IDs, typically via beam search. On-policy self-distillation (OPSD) enhances training supervision by transferring privileged self-teacher knowledge at student-generated states. However, beam-search retention depends on competition among prefixes: visited states differ in importance, and initially reliable decisions may approach the pruning boundary as training evolves. Moreover, conventional distribution matching couples target-token confidence with preferences among competing continuations, potentially weakening relational supervision when the self-teacher strongly favors the target. To address these challenges, we propose Search-Aware On-Policy Self-Distillation (SA-OPSD), a framework that transfers privileged self-teacher knowledge to the states visited during the student's own beam search. Specifically, we introduce Evolving-Boundary Sampling, which prioritizes search-critical states by considering both current pruning-boundary proximity and initial-reference retention-margin decline under a controlled distillation budget. Furthermore, we develop Confidence-Decoupled Branch Distillation, which separates target-probability calibration from informative sibling relations to better preserve the teacher's preferences among competing branches. Experiments on Industrial, Office, and Instruments demonstrate strong performance for SA-OPSD.
Figures & tables
| Statistic | Industrial | Office | Instruments |
|---|---|---|---|
| #Users | 7,694 | 8,328 | 7,537 |
| #Items | 3,685 | 3,459 | 3,399 |
| #Interactions | 50,354 | 54,226 | 57,482 |
| Train samples | 36,259 | 38,924 | 39,956 |
| Valid samples | 4,532 | 4,866 | 4,994 |
| Test samples | 4,533 | 4,866 | 4,995 |
| Method | Industrial | Office | Instruments | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HR@5 | NDCG@5 | HR@10 | NDCG@10 | HR@5 | NDCG@5 | HR@10 | NDCG@10 | HR@5 | NDCG@5 | HR@10 | NDCG@10 | |
| Sequential recommenders | ||||||||||||
| GRU4Rec | 0.0774 | 0.0598 | 0.0999 | 0.0669 | 0.0789 | 0.0595 | 0.1019 | 0.0669 | 0.0661 | 0.0538 | 0.0871 | 0.0605 |
| Caser | 0.0717 | 0.0555 | 0.0942 | 0.0628 | 0.0865 | 0.0664 | 0.1093 | 0.0737 | 0.0628 | 0.0510 | 0.0845 | 0.0580 |
| SASRec | 0.0909 | 0.0748 | 0.1088 | 0.0806 | 0.0949 | 0.0805 | 0.1120 | 0.0858 | 0.0793 | 0.0675 | 0.0971 | 0.0733 |
| BERT4Rec | 0.0869 | 0.0709 | 0.1110 | 0.0784 | 0.0912 | 0.0753 | 0.1173 | 0.0836 | 0.0759 | 0.0652 | 0.0980 | 0.0725 |
| Strategy | Industrial | Office | Instruments |
|---|---|---|---|
| Single Rollout | 0.1168 | 0.1233 | 0.1093 |
| Multi Rollout | 0.1198 | 0.1266 | 0.1122 |
| Dense Beam (Full KL) | 0.1251 | 0.1277 | 0.1149 |
| SA-OPSD (Ours) | 0.1263 | 0.1334 | 0.1192 |
| Sampling Strategy | NDCG@10 | Regression (%) |
|---|---|---|
| Uniform sampling | 0.1216 | 16.1% |
| Current boundary only | 0.1240 | 14.5% |
| Boundary + shuffled reference declines | 0.1248 | 14.4% |
| EBS (Ours) | 0.1263 | 10.7% |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Generative Recommendation
Generative recommendation models user histories and item catalogs to generate personalized items, lists, or queries rather than only scoring candidates.
See all 120 papers