We present DRanker (Disney+ Ranker), a transformer-based multitask ranking model serving personalized recommendations on the Disney+ homepage. We describe the encoder-only architecture over enriched engagement-history sequences paired with a lightweight causal debiasing mechanism, and the inference stack that enables transformer-based ranking at scale. Through iterative development, we systematically evaluated more complex alternatives and found that simpler designs consistently outperformed them on offline metrics, qualitative behavioral evaluation, and in production. Deployed globally, DRanker drives over 1% lift in our key engagement metric and 0.5% lift in daily active users across all experiences powered by the ranker. We share practical lessons on data engineering, debiasing, and architectural choices that shaped our ranking system at scale.
Figures & tables
Figure 1. Training data construction funnel. Each stage progressively filters and refines the raw viewing signal into high-quality training samples. Four-stage funnel diagram. Top layer: All Engagement. Second: Qualified by Engagement Length. Third: First-Time Interactions. Bottom: Stratified Sampling. Each stage is narrower than the previous, with annotations on the right explaining the filtering rationale.
Figure 2. DRanker model architecture. The transformer encoder processes enriched engagement-history sequences. The shared MLP backbone receives embeddings from the sequential encoder, candidate item, contextual features, and causal tower. Two task-specific heads predict engagement probability and expected engagement length. The causal tower’s pop/rel output heads are discarded at inference; only its learned embeddings are retained. Model architecture diagram. Bottom: input features per sequence title (content embedding, contextual features, engagement features) feed into a transformer encoder with self-attention and mean pooling. Middle: candidate embedding, sequence embedding, session/contextual features, and causal tower embeddings feed into a shared ReLU MLP backbone. Top: two task-specific heads output engagement probability and engagement length. A separate causal tower on the left produces popularity and relevance embeddings with gradient isolation.
Experiment
Attention
Causal
MTL
Result
DRanker (final)
Self-attn
Yes
YT
Best overall
Base (no causal)
Self-attn
No
YT
Lower entity coverage
Weighted loss
Self-attn
No
YT
Unstable training loss
Causal + weighted
Self-attn
Yes
YT
No gain over causal-only
Cross-attention
Cross-attn
Yes
YT
Contextual features dominated
PLE
Self-attn
Yes
PLE
Gating saturation
Table 1. Summary of architectural experiments. All experiments use LLM-derived + ID-based content embeddings, a fixed positive-to-negative ratio, and a short-term sequence length. “YT” denotes YouTube-style stacked ReLU heads. Best configuration in bold.
Variant
AP@10
Top-50 Overlap
Self-Attn + No Causal + YT
∼ 0.035
∼ 10%
Self-Attn + Causal + YT
∼ 0.028
∼ 5%
Self-Attn + Causal + PLE
∼ 0.018
∼ 1%
Cross-Attn + Causal + YT
∼ 0.015
∼ 4%
Table 2. Offline evaluation — accuracy vs. popularity concentration. AP@10 and Top-50 Overlap (fraction of a user’s top-100 recommendations appearing in the platform’s daily top-50 titles). Evaluated on 100k daily interactions (US, 10-day holdout). Missing interactions are imputed as not relevant. Higher AP@10 is better; lower Top-50 Overlap indicates less popularity bias. The same variant ranking held for NDCG@k and Recall@k across all cutoffs.
Base
Base
Tower
Tower
Metric
(prob)
(combined)
(prob)
(combined)
Engagement/User
+0.43
+0.41
+0.35
+0.48 †
Title Diversity
+1.02 †
+0.50
+0.84
+0.32
DAU
+0.29 †
+0.13
+0.20
+0.24
% Sessions w/ Search
− 35.1
− 49.3
− 41.8
− 61.6 †
Table 3. Online A/B results — relative lift (%) over control. Bold = statistically significant ( p<0.01 ). Best per metric highlighted with † .
Figure 3. Multi-user profile test. Top: synthetic shared engagement history with two distinct patterns (children’s content during daytime on mobile, general entertainment during evening on TV). Bottom: the model correctly surfaces children’s content for daytime/mobile sessions and general entertainment for evening/TV sessions. Two-panel figure. Top panel: a synthetic shared engagement history with two distinct patterns, children's content engaged during daytime on mobile and general entertainment content engaged during evening on TV. Bottom panel: model recommendations split by session context, surfacing children's content for daytime/mobile sessions and general entertainment for evening/TV sessions.
Figure 4. Relative model scores by local hour of day. Bluey (children’s) peaks during daytime; Andor (mature) peaks in the evening, matching real engagement patterns. Line chart with two curves plotted against local hour of day. Bluey (children's title) scores peak during midday hours. Andor (mature title) scores peak during evening hours.
Scaling Transformers has driven large gains in language modeling, but transplanting this to behavior-sequence modeling in production ranking is challenging: recommendation differs in signal quality, where behavior sequences are noisy, temporally irregular, and sparsely supervised, and in computation asymmetry, where each request scores many candidates against one shared user history under tight latency budgets. We propose ReST, a recommendation-native Transformer scaling framework. For signal quality, it introduces a sequence encoder with dual-gated attention, rotary positional and temporal embedding, stabilized residual normalization, and training-only auxiliary objectives. For computation asymmetry, it factorizes ranking into a heavy reusable encoder and a lightweight cross decoder with projection-free KV attention and token-specific parameterization, coupling user-level shared-prefix training with shared-prefix serving for compute-once, decode-many-times ranking. Across industrial and public benchmarks, ReST achieves higher accuracy and scales more consistently along sequence length, depth, and width, where LLM-style Transformer blocks saturate. A one-week online A/B test on a production advertising platform improves online AUC by 1.31% and lifts a core revenue metric by 11.93% within a 50 ms P99 budget; ReST has since been fully deployed in production, showing that behavior-sequence scaling remains a promising, under-exploited axis for production ranking.
Modern recommendation systems predominantly train retrieval and ranking as separate models despite both increasingly relying on large transformers encoding the same user behavior data, duplicating parameters, compute, and serving cost. Prior work unifies the model architecture but not the full pipeline: input formats, training procedures, and serving stacks remain fragmented across stages. We present UniPinRec, which achieves full-stack unification of retrieval and ranking at Pinterest: one input format, one model, one training stage, deployed within existing serving infrastructure. A shared transformer encodes the user action sequence into candidate-independent representations that branch into retrieval (ANN dot-product) and ranking (cross-attention) via task-specific heads. Three ideas make this work: (1) Masked Action Modeling (MAM) eliminates interleaving, enabling weight sharing without doubling context length; (2) Blended training examples pair action sequences with feedview impression slates to satisfy both objectives jointly; (3) Cross-stage KV cache sharing reuses user-history computation from retrieval for ranking, reducing total FLOPs versus serving two independent models. Deployed in the Pinterest core surfaces, UniPinRec delivers approximately +1% online engagement lift while cutting end-to-end serving latency by 11.1% and lifting QPS by 63.6%. To our knowledge, this is the first full-stack unification of retrieval and ranking, covering inputs, model, training and serving, deployed in a production recommendation system.
Modern recommendation systems rank candidates by aggregating multiple behavioral signals through a value model. However, many commonly used signals are inherently affected by heterogeneous biases. For example, watch time naturally favors long-form content, loop rate favors short - form content, and comment probability favors videos over images. Such biases introduce two critical issues: (1) value model scores may be systematically misaligned with users' relative preferences - for instance, a seemingly low absolute like probability may represent exceptionally strong interest for a user who rarely engages; and (2) changes in value modeling rules can trigger abrupt and undesirable ecosystem shifts. In this work, we ask a fundamental question: can biased behavioral signals be systematically transformed into unbiased signals, under a user - defined notion of ``unbiasedness'', that are both personalized and adaptive? We propose a general, model-based debiasing (MBD) framework that addresses this challenge by augmenting it with distributional modeling. By conditioning on a flexible subset of features (partial feature set), we explicitly estimate the contextual mean and variance of the engagement distribution for arbitrary cohorts (e.g., specific video lengths or user regions) directly alongside the main prediction. This integration allows the framework to convert biased raw signals into unbiased representations, enabling the construction of higher-level, calibrated signals (such as percentiles or z - scores) suitable for the value model. Importantly, the definition of unbiasedness is flexible and controllable, allowing the system to adapt to different personalization objectives and modeling preferences. Crucially, this is implemented as a lightweight, built-in branch of the existing MTML ranking model, requiring no separate serving infrastructure.