MBD: A Model-Based Debiasing Framework Across User, Content, and Model Dimensions

Yuantong Li; Lei Yuan; Zhihao Zheng; Weimiao Wu; Songbin Liu; Jeong Min Lee; Ali Selman Aydin; Shaofeng Deng

cs.LGMar 15, 2026

MBD: A Model-Based Debiasing Framework Across User, Content, and Model Dimensions

Authors: Yuantong Li, Lei Yuan, Zhihao Zheng, Weimiao Wu, Songbin Liu, Jeong Min Lee, Ali Selman Aydin, Shaofeng Deng,

Organizations: Meta AI

Abstract

Modern recommendation systems rank candidates by aggregating multiple behavioral signals through a value model. However, many commonly used signals are inherently affected by heterogeneous biases. For example, watch time naturally favors long-form content, loop rate favors short - form content, and comment probability favors videos over images. Such biases introduce two critical issues: (1) value model scores may be systematically misaligned with users' relative preferences - for instance, a seemingly low absolute like probability may represent exceptionally strong interest for a user who rarely engages; and (2) changes in value modeling rules can trigger abrupt and undesirable ecosystem shifts. In this work, we ask a fundamental question: can biased behavioral signals be systematically transformed into unbiased signals, under a user - defined notion of ``unbiasedness'', that are both personalized and adaptive? We propose a general, model-based debiasing (MBD) framework that addresses this challenge by augmenting it with distributional modeling. By conditioning on a flexible subset of features (partial feature set), we explicitly estimate the contextual mean and variance of the engagement distribution for arbitrary cohorts (e.g., specific video lengths or user regions) directly alongside the main prediction. This integration allows the framework to convert biased raw signals into unbiased representations, enabling the construction of higher-level, calibrated signals (such as percentiles or z - scores) suitable for the value model. Importantly, the definition of unbiasedness is flexible and controllable, allowing the system to adapt to different personalization objectives and modeling preferences. Crucially, this is implemented as a lightweight, built-in branch of the existing MTML ranking model, requiring no separate serving infrastructure.

Figures & tables

Figure 1: The MBD framework augments the MTML model with an additional branch that estimates contextual distribution statistics

(\mu,\sigma^{2})

through a partial feature set.

Figure 2: Transformed watch time (

P_{50}

) trend over one month bucketed by video length.

Task	Bias		NLL		Correlation ( $\rho$ )
Task	( $p,y$ )	( $\mu_{\text{{{MBD}}}},p$ )	$\sigma_{cluster}$	$\sigma_{\text{\text{{MBD}}}}$	( $p,\mu_{\text{\text{{MBD}}}}$ )	( $\sigma_{\text{\text{{MBD}}}},\sigma_{\text{cluster}}$ )
Watch Time	0.0014	-0.0060	0.989	0.442	0.815	0.690
Plike (Logit)	0.0018	-0.0026	5.863	1.752	0.863	0.625

Table 1: Contextual distribution estimation quality measured across accuracy (bias), distributional fit (NLL), and alignment (correlation).

Task: Watch Time (Positive Bias)		Task: Loop Rate (Negative Bias)
Method	Correlation ( $\rho$ )	Method	Correlation ( $\rho$ )
$y$	0.350	$y$	-0.13
Log( $y$ )	0.222
$p(\mathbf{x})$	0.514	$p(\mathbf{x})$	-0.13
VVP95	-0.043
VVP95 $\times$ NormTS	-0.228

Table 2: Correlation analysis between signals and video duration.

Application	Task	Metric Lift (%)
Application	Task	Session	View	Watch Time	Share	Like	Breakout
a. Media Length Debias	Watch Time	0.002%	0.003%	0.198 %	0.44%	0.173 %	-
b. Content Format Debias	Long Time Spent	0.006%	-0.011%	0.058 %	-0.071%	-0.119 %	-
	Like	-0.001%	0.040 %	-0.018%	0.035%	0.421 %	-
	Click	0.018 %	0.023%	0.034 %	-0.040%	0.007%	-
c. Content Cold Start Debias	Watch Time	0.011 %	0.135 %	0.072%	0.049%	0.823%	0.190 %

Table 3: Online A/B testing results across case studies. Note: A boldface means a statistically significant result (p-value< 5%).

Video Length	Short-Form (0s - 30s)				Mid-Length (30s - 3m)				Long-Form (3m - 30m)			Ultra-Long (30m+)
Bucket	0 $\sim$ 5s	5 $\sim$ 10s	10 $\sim$ 15s	15 $\sim$ 30s	30 $\sim$ 45s	45 $\sim$ 60s	60 $\sim$ 90s	90 $\sim$ 180s	3 $\sim$ 5m	5 $\sim$ 10m	10 $\sim$ 30m	30 $\sim$ 60m	60m+
Watch Time Shift	-0.64%	-0.75%	-0.62%	-0.19%	0.43%	0.60%	0.77%	0.85%	1.05%	0.73%	0.31%	0.53%	-0.65%
Video Views Shift	-0.83%	-0.80%	-0.77%	-0.40%	0.12%	0.27%	0.39%	0.41%	0.55%	0.13%	-0.23%	0.25%	-0.46%
Efficiency Ratio	77%	94%	81%	47%	350%	222%	198%	209%	191%	562%	-135%	>200%	143%
Traffic Trend	$\searrow$	$\searrow$	$\searrow$	$\searrow$	$\nearrow$	$\nearrow$	$\nearrow$	$\nearrow$	$\nearrow$	$\nearrow$	$\searrow$	$\nearrow$	$\searrow$
Impact	Pruned	Pruned	Pruned	Pruned	Promoted	Promoted	Promoted	Promoted	Promoted	Promoted	Optimized	Promoted	Pruned

Table 4: Engagement efficiency analysis by video length buckets. The table illustrates the traffic redistribution driven by the MBD model. Efficiency Ratio (

\frac{\%\Delta\text{WT}}{\%\Delta\text{VV}}

) quantifies the quality of the shift: a ratio

<100\%

during pruning (

\searrow

) indicates the removal of low-value views, while a ratio

>100\%

during promotion (

\nearrow

) indicates the surfacing of high-retention content.

Figure 3: User-level transformed time spent mean and variance distribution stratified by stories length.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Analysis of MBD predictions across duration buckets: (a) The predicted mean $\mu_{\text{{MBD}}}$ closely tracks the empirical distribution $p(\mathbf{x})$ ; (b) The predicted variance $\sigma_{\text{{MBD}}}^{2}$ effectively captures the uncertainty within each bucket.

Panel of Figure 5 — Figure 5: Analysis of MBD predictions for Likes in logit space: (a) The predicted mean $\mu_{\text{{MBD}}}$ accurately tracks the main model’s logit prediction $p(\mathbf{x})$ ; (b) The predicted variance $\sigma_{\text{{MBD}}}^{2}$ quantifies the uncertainty of user preference intensity.

Debiased ML

Debiased machine learning develops methods to reduce unfair or spurious model behavior and statistical bias in learned estimates.

See all 56 papers

PEARL: Unbiased Percentile Estimation via Contrastive Learning for Industrial-Scale Livestream Recommendation
May 20, 2026Blake Gella, Wei Wu, Yuhao Yin +6Contrastive Learning Learning to Rank
TikTok · San Jose, California, USA · ByteDance
Exploring How Fair Model Representations Relate to Fair Recommendations
Mar 25, 2026Bjørnar Vassøy, Benjamin Kille, Helge LangsethFairness in Recommender Systems Algorithmic Fairness
Norwegian University of Science and Technology, Trondheim, Norway
DRanker: A Transformer-Based Multi-Task Ranking Model for the Disney+ Homepage
Oct 11, 2026Priya Nirmal Singh Khokher, Daniel Nemirovsky, Benjamin P. Whitesell +4Recommender Systems Learning to Rank
The Walt Disney Company United States

May 20, 2026J/K move · Enter open · S save

PEARL: Unbiased Percentile Estimation via Contrastive Learning for Industrial-Scale Livestream Recommendation

Blake Gella, Wei Wu, Yuhao Yin, Zexi Huang+5

TikTok · San Jose, California, USA · ByteDance

Contrastive Learning Learning to Rank Recommender Systems

Open paper AskPDF arXiv

Recommender systems trained on user interaction data are susceptible to behavioral intensity imbalance--a systematic distortion arising from heterogeneous engagement patterns across users. This imbalance skews feedback signals such that observed interactions no longer faithfully reflect true preferences, causing models to disproportionately amplify signals from highly active users while underrepresenting others, which ultimately degrades recommendation quality and robustness at scale. To address this issue, we propose a nonparametric contrastive percentile approximation framework, PEARL, that models relative preference signals instead of absolute engagement magnitudes. Building upon relative advantage debiasing, PEARL leverages real contrastive interaction samples to approximate percentile relationships directly, without relying on auxiliary distribution estimation models. We provide theoretical justification demonstrating that such pairwise comparisons yield unbiased estimates of percentile-based preference signals. For broader applicability, we introduce a prediction-based bootstrapping mechanism for percentile smoothing to handle sparse and discrete feedback, alongside a generalized value-weighted formulation and a co-training strategy to enhance both modeling flexibility and representation learning. Extensive offline experiments demonstrate that PEARL effectively mitigates behavioral bias and consistently improves recommendation performance across multiple ranking targets. Deployed in a production livestream platform with a combined user base of billions, online A/B testing confirms substantial real-world gains: +2.10% Watch Duration, +0.80% Consumption Amount, +1.49% Interaction Rate, and -6.91% Report Rate.

Abstract

Debiased ML

Explore similar work

PEARL: Unbiased Percentile Estimation via Contrastive Learning for Industrial-Scale Livestream Recommendation

Exploring How Fair Model Representations Relate to Fair Recommendations

DRanker: A Transformer-Based Multi-Task Ranking Model for the Disney+ Homepage