Efficient Personalized Reranking with Semi-Autoregressive Generation and Online Knowledge Distillation

Kai Cheng; Hao Wang; Wei Guo; Weiwen Liu; Yong Liu; Yawen Li; Enhong Chen

cs.IRMar 7, 2026

Efficient Personalized Reranking with Semi-Autoregressive Generation and Online Knowledge Distillation

Authors: Kai Cheng, Hao Wang, Wei Guo, Weiwen Liu, Yong Liu, Yawen Li, Enhong Chen

Organizations: University of Science and Technology of China, Hefei, China · Huawei, Shanghai, China · Shanghai Jiao Tong University, Shanghai, China · Beijing University of Posts and Telecommunications, Beijing, China

Abstract

Generative models offer a promising paradigm for the final stage reranking in multi-stage recommender systems, with the ability to capture inter-item dependencies within reranked lists. However, their practical deployment still faces two key challenges: (1) an inherent conflict between achieving high generation quality and ensuring low-latency inference, making it difficult to balance the two, and (2) insufficient interaction between user and item features in existing methods. To address these challenges, we propose a novel Personalized Semi-Autoregressive with online knowledge Distillation (PSAD) framework for reranking. In this framework, the teacher model adopts a semi-autoregressive generator to balance generation quality and efficiency, while its ranking knowledge is distilled online into a lightweight scoring network during joint training, enabling real-time and efficient inference. Furthermore, we propose a User Profile Network (UPN) that injects user intent and models interest dynamics, enabling deeper interactions between users and items. Extensive experiments conducted on three large-scale public datasets demonstrate that PSAD significantly outperforms state-of-the-art baselines in both ranking performance and inference efficiency.

Figures & tables

Figure 1: Two challenges faced by generative reranking methods.

Figure 2: The overall framework of PSAD and its sub-modules.

Dataset	#Users	#Items	#Records
Ad	$1.06\times 10^{6}$	$8.27\times 10^{5}$	$2.04\times 10^{6}$
PRM Public	$7.44\times 10^{5}$	$7.25\times 10^{6}$	$1.44\times 10^{7}$
Avito	$1.32\times 10^{6}$	$2.36\times 10^{7}$	$5.36\times 10^{7}$

Table 1: Dataset statistics.

Datasets	Ad				PRM Public				Avtio
Metric	N@5	N@10	M@5	M@10	N@5	N@10	M@5	M@10	N@5	M@5
miDNN	0.6819	0.6952	0.6025	0.6063	0.2620	0.3341	0.2842	0.2954	0.6737	0.6838
DLCM	0.6857	0.6989	0.6064	0.6100	0.2664	0.3395	0.2918	0.3023	0.6912	0.7057
PRM	0.6872	0.7015	0.6080	0.6117	0.2689	0.3430	0.2937	0.3042	0.7047	0.7174
Seq2Slate	0.6916	0.7062	0.6119	0.6155	0.2709	0.3449	0.2959	0.3061	0.7245	0.7360
CRUM	0.6899	0.7037	0.6084	0.6112	0.2703	0.3445	0.2952	0.3056	0.7110	0.7266

Table 2: Overall performance comparison of all methods in terms of NDCG@K and MAP@K (K=5, 10). (p-value < 0.05)

Batch Size	64		128		256
Model	Train(↓)	Inf.(↓)	Train(↓)	Inf.(↓)	Train(↓)	Inf.(↓)
miDNN	0.0895	0.0764	0.1425	0.1256	0.2132	0.1893
DLCM	0.2799	0.1497	0.4301	0.2409	0.6827	0.3940
PRM	0.1238	0.1014	0.1852	0.1360	0.2806	0.2037
Seq2Slate	0.3142	0.2176	0.5114	0.3993	0.8384	0.5873
CRUM	0.2021	0.1589	0.3185	0.2850	0.4415	0.3048

Table 3: The training and inference time comparison between PSAD and baseline methods on the PRM Public dataset. The comparison is conducted under different batch sizes, and the time is calculated by averaging the results over 10 steps. The time unit is seconds.

Datasets	Ad		PRM Public		Avito
Metric	N@10	M@10	N@10	M@10	N@5	M@5
w/o sa	0.7095	0.6192	0.3455	0.3054	0.7145	0.7295
w/o ce	0.7133	0.6220	0.3482	0.3096	0.7231	0.7360
w/o sace	0.7042	0.6139	0.3441	0.3052	0.7103	0.7217
w/o ppe	0.7168	0.6248	0.3523	0.3128	0.7384	0.7546
w/o pg	0.7136	0.6224	0.3507	0.3110	0.7402	0.7543

Table 4: Ablation analysis of PSAD-G

Figure 3: Hyper-Parameter Performance

Dataset	Ad		PRM
Teacher Model	N@10(↑)	Train(↓)	N@10(↑)	Train(↓)
AR(online)	0.7078	0.4896	0.3434	0.7372
NAR(online)	0.7046	0.3004	0.3429	0.4165
SAR(offline)	0.7083	0.6547	0.3481	0.8933
SAR(online)	0.7091	0.3752	0.3464	0.5128

Table 5: Comparison of different distillation methods in terms of performance (NDCG@10) and training time. The batch size is set to 256, and the time is reported as the average over 10 training steps. The time unit is seconds.

Figure 4: Performance comparison of item–user feature interaction mechanisms across user sequences with varying activity levels.

Online Knowledge Distillation

See all 10 papers

Dual-Rerank: Fusing Causality and Utility for Industrial Generative Reranking
Apr 8, 2026Chao Zhang, Shuai Lin, ChengLei Dai +5Learning to Rank RL for IR
Kuaishou Technology, Hangzhou, China · Kuaishou Technology, Beijing, China · Unaffiliated, Beijing, China
DeGRe: Dense-supervised Generative Reranking for Recommendation
May 25, 2026Chaotian Song, Jingyao Zhang, Chenghao Chen +6Learning to Rank Generative Recommendation
College of Software, Zhejiang University Hangzhou, China · Rajax Network Technology, Taobao Shangou of Alibaba Hangzhou, China · Rajax Network Technology, Taobao Shangou of Alibaba Beijing, China +1
GR2: Generative Reasoning Re-ranker
Feb 8, 2026Mingfu Liang, Yufei Li, Jay Xu +20RL for Language Models RL for Recommendation
Meta AI

Apr 8, 2026J/K move · Enter open · S save

Dual-Rerank: Fusing Causality and Utility for Industrial Generative Reranking

Chao Zhang, Shuai Lin, ChengLei Dai, Ye Qian+4

Kuaishou Technology, Hangzhou, China · Kuaishou Technology, Beijing, China · Unaffiliated, Beijing, China

Learning to Rank RL for IR

Open paper AskPDF arXiv

Kuaishou serves over 400 million daily active users, processing hundreds of millions of search queries daily against a repository of tens of billions of short videos. As the final decision layer, the reranking stage determines user experience by optimizing whole-page utility. While traditional score-and-sort methods fail to capture combinatorial dependencies, Generative Reranking offers a superior paradigm by directly modeling the permutation probability. However, deploying Generative Reranking in such a high-stakes environment faces a fundamental dual dilemma: 1) the structural trade-off where Autoregressive (AR) models offer superior Sequential modeling but suffer from prohibitive latency, versus Non-Autoregressive (NAR) models that enable efficiency but lack dependency capturing; 2) the optimization gap where Supervised Learning faces challenges in directly optimizing whole-page utility, while Reinforcement Learning (RL) struggles with instability in high-throughput data streams. To resolve this, we propose Dual-Rerank, a unified framework designed for industrial reranking that bridges the structural gap via Sequential Knowledge Distillation and addresses the optimization gap using List-wise Decoupled Reranking Optimization (LDRO) for stable online RL. Extensive A/B testing on production traffic demonstrates that Dual-Rerank achieves State-of-the-Art performance, significantly improving User satisfaction and Watch Time while drastically reducing inference latency compared to AR baselines.

Abstract

Online Knowledge Distillation

Explore similar work

Dual-Rerank: Fusing Causality and Utility for Industrial Generative Reranking

DeGRe: Dense-supervised Generative Reranking for Recommendation

GR2: Generative Reasoning Re-ranker