Not All Candidates are Created Equal: A Heterogeneity-Aware Approach to Pre-ranking in Recommender Systems

Pengfei Tong; Siyuan Chen; Chenwei Zhang; Bo Wang; Qi Pi; Pixun Li; Zuotao Liu

cs.IRMar 4, 2026

Not All Candidates are Created Equal: A Heterogeneity-Aware Approach to Pre-ranking in Recommender Systems

Authors: Pengfei Tong, Siyuan Chen, Chenwei Zhang, Bo Wang, Qi Pi, Pixun Li, Zuotao Liu

Organizations: ByteDance

Abstract

Most large-scale recommender systems follow a multi-stage cascade of retrieval, pre-ranking, ranking, and re-ranking. A key challenge at the pre-ranking stage arises from the heterogeneity of training instances sampled from coarse-grained retrieval results, fine-grained ranking signals, and exposure feedback. Our analysis reveals that prevailing pre-ranking methods, which indiscriminately mix heterogeneous samples, suffer from gradient conflicts: hard samples dominate training while easy ones remain underutilized, leading to suboptimal performance. We further show that the common practice of uniformly scaling model complexity across all samples is inefficient, as it overspends computation on easy cases and slows training without proportional gains. To address these limitations, this paper presents Heterogeneity-Aware Adaptive Pre-ranking (HAP), a unified framework that mitigates gradient conflicts through conflict-sensitive sampling coupled with tailored loss design, while adaptively allocating computational budgets across candidates. Specifically, HAP disentangles easy and hard samples, directing each subset along dedicated optimization paths. Building on this separation, it first applies lightweight models to all candidates for efficient coverage, and further engages stronger models on the hard ones, maintaining accuracy while reducing cost. This approach not only improves pre-ranking effectiveness but also provides a practical perspective on scaling strategies in industrial recommender systems. HAP has been deployed in the Toutiao production system for 9 months, yielding up to 0.4% improvement in user app usage duration and 0.05% in active days, without additional computational cost. We also release a large-scale industrial hybrid-sample dataset to enable the systematic study of source-driven candidate heterogeneity in pre-ranking.

Figures & tables

Figure 1. Gradient conflicts across negatives of varying difficulty, illustrating the inherent heterogeneity of candidates.

Figure 2. Architecture of HAP’s training sample collection and model training. Samples are drawn from complete requests, including exposed positives and negatives, and negatives from retrieval, pre-ranking , and ranking stages. HAP adopts the DAMR architecture: the lightweight model learns from all sample types via GHCL, scores retrieval candidates, and filters easy ones, while the expressive model trains exclusively on hard samples using a deeper, attention-based network.

Loss	Sample	Difficulty	Grad Norm	Logits	Cosine
BCE	EN	Hardest	1.101	0.322	–
	RN	Harder	0.254	0.232	0.816
	PRN	Easier	0.140	0.191	0.496
	GN	Easiest	0.060	0.173	0.237
InfoNCE	EN	Hardest	1.750	0.326	–
	RN	Harder	0.401	0.235	0.820

Table 1. Comparison of gradient characteristics across negatives of varying difficulty under BCE and InfoNCE losses. Sample types: exposed (EN), ranking (RN), pre-ranking (PRN), and global random (GN).

Figure 3. Model AUC vs. sample difficulty, with "Exposed" for exposed negatives and "Rank-

N

"/ "Prerank-

N

" for negatives ranked after position

N

; smaller

N

implies higher difficulty.

Figure 4. HAP’s online deployment in Toutiao: a two-stage pre-ranking pipeline with a lightweight model for easy candidates and an expressive model for hard ones; samples are grouped and common features are shared for efficiency.

Dataset	Multi-Stage	Complete	Multi-Type	Industrial
	Coverage	Rank Info	Feedback	-Scale
MovieLens ( Harper and Konstan, 2015 )	✗	✗	✗	✗
Amazon ( Ni et al., 2019 )	✗	✗	✗	✗
Yelp ( Asghar, 2016 )	✗	✗	✗	✗
Taobao ( Zhu et al., 2018 )	✗	✗	✓	✓
Tenrec ( Yuan et al., 2022 )	✗	✗	✓	✓

Table 2. Comparisons of ToutiaoRec with existing datasets.

Model	TEN	TRN	TPRN	TGN	THard	TEasy
DSSM	0.6901	0.8247	0.9008	0.9785	0.7527	0.9257
COLD	0.6936	0.8303	0.9038	0.9803	0.7601	0.9261
COPR	0.7221	0.8533	0.9172	0.9871	0.7824	0.9402
HCCP	0.7244	0.8653	0.9218	0.9874	0.7917	0.9441
HAP	0.7377	0.8693	0.9236	0.9878	0.8023	0.9468
$\text{HAP}_{\text{lite}}$	0.7331	0.8666	0.9225	0.9872	0.7998	0.9452

Table 3. Offline AUC comparison of SoTA pre-ranking models on various test sets. The best result is in bold and the second-best result is underline , respectively.

NegType	Model	TEN	TRN	TPRN	TGN	THard	TEasy
SingleNeg	$\text{BCE}_{\text{GN}}$	0.5930	0.6859	0.7717	0.9992	0.6391	0.8680
	$\text{BCE}_{\text{PRN}}$	0.6186	0.7732	0.9616	0.9406	0.6955	0.9502
	$\text{BCE}_{\text{RN}}$	0.6238	0.9017	0.8595	0.8861	0.7623	0.8728
	$\text{BCE}_{\text{EN}}$	0.7591	0.6815	0.7121	0.7815	0.7207	0.7484
	$\text{InfoNCE}_{\text{GN}}$	0.5941	0.6834	0.7677	0.9991	0.6378	0.8634
	$\text{InfoNCE}_{\text{PRN}}$	0.6148	0.7564	0.9531	0.9348	0.6845	0.9487

Table 4. Comparison of negative sampling strategies. For each negative type (GN, PRN, RN, EN), SingleNeg evaluates BCE loss versus the combined InfoNCE+BCE objective. MultiNeg combines all four. Best results in bold and underline .

Figure 5. Ablations of GHCL in HAP: removing GHCL leads to a notable AUC decrease across all test sets.

Model	THard a	TEasy a	CTR c	Compute Cost b	FLOPs ( $10^{6}$ )	Latency ( $ms$ )	Cost Ratio c
Unified Large Model	0.8158	0.9586	+0.55%	38.1	70	225	+15%
HAP	0.8197	0.9579	+3.22%	39.2	89	203	-6%

Table 5. Ablations of DAMR in HAP: Online A/B test results on Toutiao, trained on the large-scale logs.

Panel of Figure 6 — Figure 6. Gradient statistics of heterogeneous negatives. GHCL stabilizes directions and magnitudes, mitigating conflicts compared to existing methods.

Figure 7. Online performance–efficiency trade-off of HAP under varying gating thresholds.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Table 6. Performance and Efficiency Trade-offs of Models with Different Complexities

	Model	TEN	TRN	TPRN	TGN	THard	TEasy
AUC $\uparrow$	DSSM	0.6901	0.8247	0.9008	0.9785	0.7527	0.9257
	COLD	0.6936	0.8303	0.9038	0.9803	0.7601	0.9261
	COPR	0.7221	0.8533	0.9172	0.9871	0.7824	0.9402
	HCCP	0.7244	0.8653	0.9218	0.9874	0.7917	0.9441
	HAP	0.7377	0.8693	0.9236	0.9878	0.8023	0.9468
	$\text{HAP}_{\text{lite}}$	0.7331	0.8666	0.9225	0.9872	0.7998	0.9452

Appendix

Table 7. Comparative Evaluation of HAP Against Industrial Recommender Baselines. The best results are highlighted in bold and underline . Metrics with

\uparrow

indicate higher is better, while

\downarrow

indicates lower is better.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 508 papers

Generative Pseudo-Labeling for Pre-Ranking with LLMs
Feb 24, 2026Junyu Bi, Xinting Niu, Daixuan Cheng +4Pseudo-Labeling Recommender Systems
Alibaba Group Beijing, China · Renmin University Beijing, China
Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study
Jul 30, 2026Di Bai, Jintao Liu, Zhenwei Tang +3Learning to Rank Multi-Task Learning
Google LLC Mountain View, CA, USA · Google LLC Toronto, ON, Canada · Google LLC New York, NY, USA +1
Recommendation Retrievers Need Verifiers: Universal Generative Reranking for Sequential Recommendations
Sep 10, 2026Benyu Zhang, Qiang Zhang, Rui Li +3Sequential Recommendation Recommender Systems
Meta MRS

Feb 24, 2026J/K move · Enter open · S save

Generative Pseudo-Labeling for Pre-Ranking with LLMs

Junyu Bi, Xinting Niu, Daixuan Cheng, Kun Yuan+3

Alibaba Group Beijing, China · Renmin University Beijing, China

Pseudo-Labeling Recommender Systems

Open paper AskPDF arXiv

Pre-ranking is a critical stage in industrial recommendation systems, tasked with efficiently scoring thousands of recalled items for downstream ranking. A key challenge is the train-serving discrepancy: pre-ranking models are trained only on exposed interactions, yet must score all recalled candidates -- including unexposed items -- during online serving. This mismatch not only induces severe sample selection bias but also degrades generalization, especially for long-tail content. Existing debiasing approaches typically rely on heuristics (e.g., negative sampling) or distillation from biased rankers, which either mislabel plausible unexposed items as negatives or propagate exposure bias into pseudo-labels. In this work, we propose Generative Pseudo-Labeling (GPL), a framework that leverages large language models (LLMs) to generate unbiased, content-aware pseudo-labels for unexposed items, explicitly aligning the training distribution with the online serving space. By offline generating user-specific interest anchors and matching them with candidates in a frozen semantic space, GPL provides high-quality supervision without adding online latency. Deployed in a large-scale production system, GPL improves click-through rate by 3.07%, while significantly enhancing recommendation diversity and long-tail item discovery.

Abstract

Recommender Systems

Explore similar work

Generative Pseudo-Labeling for Pre-Ranking with LLMs

Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study

Recommendation Retrievers Need Verifiers: Universal Generative Reranking for Sequential Recommendations