A Fairness Audit of the Duckworth-Lewis-Stern Method: Format-Specific and Gender-Differential Bias, with an Interpretable Calibration Layer for Cricket Target Revision

Soumyadeep Roy

cs.LGSep 4, 2026

A Fairness Audit of the Duckworth-Lewis-Stern Method: Format-Specific and Gender-Differential Bias, with an Interpretable Calibration Layer for Cricket Target Revision

Authors: Soumyadeep Roy

Organizations: St. Xavier’s College (Autonomous), Kolkata, India

Abstract

The Duckworth-Lewis-Stern (DLS) method has been the international standard for revising target scores in rain-interrupted limited-overs cricket since 1999. Despite over two decades of operational use, no large-scale empirical audit of its prediction bias has been published. We conduct such an audit on 8,150 international matches (3,095 ODIs, 5,055 T20Is) from Cricsheet, generating 233,550 synthetic interruption scenarios with temporal splits. We document two structured biases. First, DLS prediction error spans a 137-run range across (overs-remaining, wickets-lost) match-state buckets. Second, DLS exhibits a gender-differential bias on ODIs that has not previously been quantified: on the training split, mean over-prediction is +1.51 runs for men but +7.63 runs for women, a gap of +6.13 runs (F = 195.16, p < 10^-43). We benchmark DLS against five modern alternatives: Bi-LSTM, XGBoost, an enriched XGBoost variant, a deep context-aware model, and a stacking ensemble, and propose DLS-Cal, a lightweight interpretable calibration layer (27K parameters) outputting a state-conditioned correction added to DLS. DLS-Cal reduces absolute bias by 31% on ODI and 19% on T20I, and a gender-aware variant reduces women's ODI residual bias from +6.19 to +0.65 runs while leaving men's calibration unchanged. We release code, models, and data.

Figures & tables

Term	Definition
Prediction error $e_{i}$	$\hat{r}_{i}-r_{i}$ (predicted minus actual remaining runs)
Signed mean bias	$\tfrac{1}{N}\sum_{i}e_{i}$ (may cancel across states)
Mean absolute bias	$\tfrac{1}{N}\sum_{i}\|e_{i}\|$
Per-bucket mean bias	signed mean of $e_{i}$ within an (overs, wickets) cell
Residual bias	signed mean of $e_{i}$ after applying a correction (e.g. DLS-Cal)
Gender gap	(female mean bias) $-$ (male mean bias), at matched states

Table 1: Terminology for error and bias used throughout.

Figure 1: DLS systematic bias map on the training split, by format, overs remaining, and wickets lost. Cell value is mean DLS prediction error (predicted $-$ actual) in runs; red indicates over-prediction, blue indicates under-prediction. Cells with fewer than 30 examples are not shown. Both panels reveal a structured, format-specific bias pattern.

Format	Gender	N	Mean bias	RMSE	Mean $\|$ bias $\|$
ODI	male	60,660	$+1.51$	41.82	32.87
ODI	female	10,350	$+7.63$	38.49	30.60
T20	male	46,740	$+7.07$	25.51	20.50
T20	female	22,410	$+7.58$	23.06	18.04

Table 2: DLS bias on the training split , stratified by format and gender. Test-split numbers (Section 5.5 ) follow the same qualitative pattern but differ in absolute magnitude due to scoring-rate evolution between the training and test periods.

Figure 2: DLS bias by format and gender on the training split. Each row pairs male (left) and female (right) bias maps for the same format; color scale is matched within each row for direct comparison. Cells with fewer than 30 examples are not shown. ODI bias structures differ qualitatively between genders; T20 patterns are similar.

Method	Type	RMSE	MAE	Bias	RMSE ODI	RMSE T20	WFR 5 ODI	WFR 5 T20
DLS (original)	baseline	37.41	30.54	$+17.40$	43.25	36.25	92.0%	90.0%
DLS (recalibrated)	baseline	36.84	30.01	$+16.38$	43.42	35.51	93.0%	89.0%
DLS (canonical)	baseline	28.92	22.61	$+7.57$	42.84	25.57	91.0%	85.0%
Simple LSTM	deep	24.42	18.22	$+1.96$	38.92	20.70	88.0%	80.0%
XGBoost	tree	22.20	16.77	$+1.84$	31.00	20.19	84.0%	79.0%
XGBoost-Plus	tree	23.46	17.57	$-0.69$	32.33	21.45	84.0%	79.0%

Table 3: Master results table on the test split (

N=69{,}090

). Lower is better for all metrics. Best non-baseline value per column shown in bold . Gender-aware DLS-Cal results stratified by gender are reported in Section 5.5 (Table 5 ).

Correction scheme	RMSE	MAE	Bias
DLS (uncorrected)	28.88	22.58	$+7.41$
Bucket-mean lookup	37.20	29.79	$+14.58$
Linear (full features)	22.80	17.31	$+2.21$
MLP, no bucket prior	22.02	16.52	$+0.46$
MLP, full (DLS-Cal)	22.09	16.57	$+0.56$

Table 4: Ablation ladder for the calibration layer (test split, ODI+T20I). Lower is better.

Figure 3: DLS-Cal correction structure on the test split. Top row: ODI; bottom row: T20I. Left column: DLS bias by (overs remaining, wickets lost). Middle column: DLS-Cal’s mean correction added. Right column: residual bias after applying DLS-Cal. Cells with fewer than 30 examples are not shown.

Method	Gender	N	RMSE	MAE	Bias [95% CI]
DLS canonical	male	40,530	30.28	23.64	$+8.55$ $[+8.27,+8.83]$
DLS canonical	female	28,560	26.88	21.15	$+6.19$ $[+5.89,+6.49]$
DLS-Cal (gender-blind)	male	40,530	24.27	18.23	$-3.53$ $[-3.76,-3.30]$
DLS-Cal (gender-blind)	female	28,560	22.92	17.00	$+1.09$ $[+0.82,+1.36]$
DLS-Cal (gender-aware)	male	40,530	24.50	18.41	$-3.47$ $[-3.71,-3.23]$
DLS-Cal (gender-aware)	female	28,560	22.11	16.53	$\mathbf{+0.65}$ $\mathbf{[+0.39,+0.91]}$

Table 5: Test-split performance stratified by gender. Confidence intervals are computed by match-level block bootstrap (2,000 resamples of whole matches) so that widths reflect the effective number of matches rather than synthetic examples; the same clustering is used for the gender-gap intervals in Section 3.3 . The gender-aware DLS-Cal is trained separately on men’s and women’s training data; at inference, examples are routed to the matching model.

Figure 4: Calibration / reliability diagram for the four most informative methods, by format. Each marker is a quantile bin of predicted remaining runs; perfect calibration lies on the dashed diagonal.

Algorithmic Bias

Algorithmic bias studies how automated systems produce or amplify unfair outcomes across demographic groups, and how to measure and reduce them.

See all 101 papers

Human Responses to Algorithmic Correction: Evidence from Centre Court
Jan 30, 2024David Almog, Romain Gauriot, Lionel Page +1Human-AI Decision Making Human Oversight of AI
Kellogg School of Management, Northwestern University · Deakin University · University of Queensland +1
Cheap, Consistent, Calibrated, but Biased? Typed Decision Models as Judges
Oct 13, 2026Fabian Lukassen, Christoph Weisser, Alexander SilbersdorffBias in LLM-as-a-Judge LLM-as-a-Judge
University of Göttingen · Bielefeld University of Applied Sciences and Arts (HSBI)
From Score Matrices to Football-Aware Match-State Simulation: An Auditable LLM Harness for Exact-Score Reranking
Aug 5, 2026Shaopeng LiangProbabilistic Forecasting LLM Reranking
Independent Researcher

Jan 30, 2024J/K move · Enter open · S save

Human Responses to Algorithmic Correction: Evidence from Centre Court

David Almog, Romain Gauriot, Lionel Page, Daniel Martin

Kellogg School of Management, Northwestern University · Deakin University · University of Queensland+1

Human-AI Decision Making Human Oversight of AI AI-Assisted Decision Making

Open paper AskPDF arXiv

We study how algorithmic review changes the initial decisions that a system is designed to correct. Using Hawk-Eye data from professional tennis, we compare umpire decisions with and without review against a common accuracy benchmark. Under review, initial errors were 8.6 percentage points higher just outside the line and lower for comparably close balls inside. We find no evidence that average initial accuracy deteriorated. Within matches, reversals were followed by larger adjustments than upheld calls, consistent with adaptation to correction. Overall, correction offsets worse initial decisions for incorrect in calls and reinforces better initial decisions for incorrect out calls. Whether review lowers organizational costs depends jointly on these changes and the costs of remedying corrected errors. A conditional illustration shows how utilization spreads fixed review costs across calls.

Abstract

Algorithmic Bias

Explore similar work

Human Responses to Algorithmic Correction: Evidence from Centre Court

Cheap, Consistent, Calibrated, but Biased? Typed Decision Models as Judges

From Score Matrices to Football-Aware Match-State Simulation: An Auditable LLM Harness for Exact-Score Reranking