MBD: A Model-Based Debiasing Framework Across User, Content, and Model Dimensions

Yuantong Li; Lei Yuan; Zhihao Zheng; Weimiao Wu; Songbin Liu; Jeong Min Lee; Ali Selman Aydin; Shaofeng Deng

cs.LGMar 15, 2026

MBD: A Model-Based Debiasing Framework Across User, Content, and Model Dimensions

Authors: Yuantong Li, Lei Yuan, Zhihao Zheng, Weimiao Wu, Songbin Liu, Jeong Min Lee, Ali Selman Aydin, Shaofeng Deng,

Organizations: Meta AI

Abstract

Modern recommendation systems rank candidates by aggregating multiple behavioral signals through a value model. However, many commonly used signals are inherently affected by heterogeneous biases. For example, watch time naturally favors long-form content, loop rate favors short - form content, and comment probability favors videos over images. Such biases introduce two critical issues: (1) value model scores may be systematically misaligned with users' relative preferences - for instance, a seemingly low absolute like probability may represent exceptionally strong interest for a user who rarely engages; and (2) changes in value modeling rules can trigger abrupt and undesirable ecosystem shifts. In this work, we ask a fundamental question: can biased behavioral signals be systematically transformed into unbiased signals, under a user - defined notion of ``unbiasedness'', that are both personalized and adaptive? We propose a general, model-based debiasing (MBD) framework that addresses this challenge by augmenting it with distributional modeling. By conditioning on a flexible subset of features (partial feature set), we explicitly estimate the contextual mean and variance of the engagement distribution for arbitrary cohorts (e.g., specific video lengths or user regions) directly alongside the main prediction. This integration allows the framework to convert biased raw signals into unbiased representations, enabling the construction of higher-level, calibrated signals (such as percentiles or z - scores) suitable for the value model. Importantly, the definition of unbiasedness is flexible and controllable, allowing the system to adapt to different personalization objectives and modeling preferences. Crucially, this is implemented as a lightweight, built-in branch of the existing MTML ranking model, requiring no separate serving infrastructure.

Figures & tables

Figure 1: The MBD framework augments the MTML model with an additional branch that estimates contextual distribution statistics

(\mu,\sigma^{2})

through a partial feature set.

Figure 2: Transformed watch time (

P_{50}

) trend over one month bucketed by video length.

Task	Bias		NLL		Correlation ( $\rho$ )
Task	( $p,y$ )	( $\mu_{\text{{{MBD}}}},p$ )	$\sigma_{cluster}$	$\sigma_{\text{\text{{MBD}}}}$	( $p,\mu_{\text{\text{{MBD}}}}$ )	( $\sigma_{\text{\text{{MBD}}}},\sigma_{\text{cluster}}$ )
Watch Time	0.0014	-0.0060	0.989	0.442	0.815	0.690
Plike (Logit)	0.0018	-0.0026	5.863	1.752	0.863	0.625

Table 1: Contextual distribution estimation quality measured across accuracy (bias), distributional fit (NLL), and alignment (correlation).

Task: Watch Time (Positive Bias)		Task: Loop Rate (Negative Bias)
Method	Correlation ( $\rho$ )	Method	Correlation ( $\rho$ )
$y$	0.350	$y$	-0.13
Log( $y$ )	0.222
$p(\mathbf{x})$	0.514	$p(\mathbf{x})$	-0.13
VVP95	-0.043
VVP95 $\times$ NormTS	-0.228

Table 2: Correlation analysis between signals and video duration.

Application	Task	Metric Lift (%)
Application	Task	Session	View	Watch Time	Share	Like	Breakout
a. Media Length Debias	Watch Time	0.002%	0.003%	0.198 %	0.44%	0.173 %	-
b. Content Format Debias	Long Time Spent	0.006%	-0.011%	0.058 %	-0.071%	-0.119 %	-
	Like	-0.001%	0.040 %	-0.018%	0.035%	0.421 %	-
	Click	0.018 %	0.023%	0.034 %	-0.040%	0.007%	-
c. Content Cold Start Debias	Watch Time	0.011 %	0.135 %	0.072%	0.049%	0.823%	0.190 %

Table 3: Online A/B testing results across case studies. Note: A boldface means a statistically significant result (p-value< 5%).

Video Length	Short-Form (0s - 30s)				Mid-Length (30s - 3m)				Long-Form (3m - 30m)			Ultra-Long (30m+)
Bucket	0 $\sim$ 5s	5 $\sim$ 10s	10 $\sim$ 15s	15 $\sim$ 30s	30 $\sim$ 45s	45 $\sim$ 60s	60 $\sim$ 90s	90 $\sim$ 180s	3 $\sim$ 5m	5 $\sim$ 10m	10 $\sim$ 30m	30 $\sim$ 60m	60m+
Watch Time Shift	-0.64%	-0.75%	-0.62%	-0.19%	0.43%	0.60%	0.77%	0.85%	1.05%	0.73%	0.31%	0.53%	-0.65%
Video Views Shift	-0.83%	-0.80%	-0.77%	-0.40%	0.12%	0.27%	0.39%	0.41%	0.55%	0.13%	-0.23%	0.25%	-0.46%
Efficiency Ratio	77%	94%	81%	47%	350%	222%	198%	209%	191%	562%	-135%	>200%	143%
Traffic Trend	$\searrow$	$\searrow$	$\searrow$	$\searrow$	$\nearrow$	$\nearrow$	$\nearrow$	$\nearrow$	$\nearrow$	$\nearrow$	$\searrow$	$\nearrow$	$\searrow$
Impact	Pruned	Pruned	Pruned	Pruned	Promoted	Promoted	Promoted	Promoted	Promoted	Promoted	Optimized	Promoted	Pruned

Table 4: Engagement efficiency analysis by video length buckets. The table illustrates the traffic redistribution driven by the MBD model. Efficiency Ratio (

\frac{\%\Delta\text{WT}}{\%\Delta\text{VV}}

) quantifies the quality of the shift: a ratio

<100\%

during pruning (

\searrow

) indicates the removal of low-value views, while a ratio

>100\%

during promotion (

\nearrow

) indicates the surfacing of high-retention content.

Figure 3: User-level transformed time spent mean and variance distribution stratified by stories length.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Analysis of MBD predictions across duration buckets: (a) The predicted mean $\mu_{\text{{MBD}}}$ closely tracks the empirical distribution $p(\mathbf{x})$ ; (b) The predicted variance $\sigma_{\text{{MBD}}}^{2}$ effectively captures the uncertainty within each bucket.

Panel of Figure 5 — Figure 5: Analysis of MBD predictions for Likes in logit space: (a) The predicted mean $\mu_{\text{{MBD}}}$ accurately tracks the main model’s logit prediction $p(\mathbf{x})$ ; (b) The predicted variance $\sigma_{\text{{MBD}}}^{2}$ quantifies the uncertainty of user preference intensity.

Debiased ML

Debiased machine learning develops methods to reduce unfair or spurious model behavior and statistical bias in learned estimates.

See all 56 papers

May 20, 2026

PEARL: Unbiased Percentile Estimation via Contrastive Learning for Industrial-Scale Livestream Recommendation

Recommender systems trained on user interaction data are susceptible to behavioral intensity imbalance--a systematic distortion arising from heterogeneous engagement patterns across users. This imbalance skews feedback signals such that observed interactions no longer faithfully reflect true preferences, causing models to disproportionately amplify signals from highly active users while underrepresenting others, which ultimately degrades recommendation quality and robustness at scale. To address this issue, we propose a nonparametric contrastive percentile approximation framework, PEARL, that models relative preference signals instead of absolute engagement magnitudes. Building upon relative advantage debiasing, PEARL leverages real contrastive interaction samples to approximate percentile relationships directly, without relying on auxiliary distribution estimation models. We provide theoretical justification demonstrating that such pairwise comparisons yield unbiased estimates of percentile-based preference signals. For broader applicability, we introduce a prediction-based bootstrapping mechanism for percentile smoothing to handle sparse and discrete feedback, alongside a generalized value-weighted formulation and a co-training strategy to enhance both modeling flexibility and representation learning. Extensive offline experiments demonstrate that PEARL effectively mitigates behavioral bias and consistently improves recommendation performance across multiple ranking targets. Deployed in a production livestream platform with a combined user base of billions, online A/B testing confirms substantial real-world gains: +2.10% Watch Duration, +0.80% Consumption Amount, +1.49% Interaction Rate, and -6.91% Report Rate.

Blake Gella, Wei Wu, Yuhao Yin +6

TikTok · San Jose, California, USA · ByteDance

Contrastive Learning Learning to Rank Recommender Systems

Ask PDF

Mar 25, 2026

Exploring How Fair Model Representations Relate to Fair Recommendations

One of the many fairness definitions pursued in recent recommender system research targets mitigating demographic information encoded in model representations. Models optimized for this definition are typically evaluated on how well demographic attributes can be classified given model representations, with the (implicit) assumption that this measure accurately reflects recommendation parity, i.e., how similar recommendations given to different users are. We challenge this assumption by comparing the amount of demographic information encoded in representations with various measures of how the recommendations differ. We propose two new approaches for measuring how well demographic information can be classified given ranked recommendations. Our results from extensive testing of multiple models on one real and multiple synthetically generated datasets indicate that optimizing for fair representations positively affects recommendation parity, but also that evaluation at the representation level is not a good proxy for measuring this effect when comparing models. We also provide extensive insight into how recommendation-level fairness metrics behave for various models by evaluating their performances on numerous generated datasets with different properties.

Bjørnar Vassøy, Benjamin Kille, Helge Langseth

Norwegian University of Science and Technology, Trondheim, Norway

Fairness in Recommender Systems Algorithmic Fairness Recommender Systems Group Fairness Representation Learning

Ask PDF

Oct 11, 2026

DRanker: A Transformer-Based Multi-Task Ranking Model for the Disney+ Homepage

We present DRanker (Disney+ Ranker), a transformer-based multitask ranking model serving personalized recommendations on the Disney+ homepage. We describe the encoder-only architecture over enriched engagement-history sequences paired with a lightweight causal debiasing mechanism, and the inference stack that enables transformer-based ranking at scale. Through iterative development, we systematically evaluated more complex alternatives and found that simpler designs consistently outperformed them on offline metrics, qualitative behavioral evaluation, and in production. Deployed globally, DRanker drives over 1% lift in our key engagement metric and 0.5% lift in daily active users across all experiences powered by the ranker. We share practical lessons on data engineering, debiasing, and architectural choices that shaped our ranking system at scale.

Priya Nirmal Singh Khokher, Daniel Nemirovsky, Benjamin P. Whitesell +4

The Walt Disney Company United States

Recommender Systems Learning to Rank User Preference Modeling

Ask PDF

Abstract

Debiased ML

Explore similar work

PEARL: Unbiased Percentile Estimation via Contrastive Learning for Industrial-Scale Livestream Recommendation

Exploring How Fair Model Representations Relate to Fair Recommendations

DRanker: A Transformer-Based Multi-Task Ranking Model for the Disney+ Homepage