Fairness constraints in predict-then-optimize pipelines are necessarily enforced on predicted outcomes y^, since true outcomes y are unobserved at decision time. We study whether a demographic parity constraint enforced this way continues to hold against y, and how the resulting violation scales with prediction error. Using a budget-constrained allocation problem solved as a linear program, we show empirically that the true-outcome fairness gap grows with prediction error σ and saturates, well fit by c(1−e−σ/τ) with c=0.1545±0.0103 and τ=3.9045±0.3745; its mean is invariant to sample size while its variance decays at the standard O(1/n) rate. We derive a two-part theoretical bound: a closed-form local sensitivity result showing the gap's first-order response to prediction error depends on only two individuals per group, validated against the solver to within three significant figures; and a universal, error-independent ceiling following from the boundedness of the decision variable, confirmed consistent with but looser than the empirical saturation level. On US Census income data, enforcing demographic parity on a trained predictor's output reduces an unconstrained disparity in true income to roughly one-third of its original size, leaving 34.4% ± 3.3% (California) and 35.7% ± 1.8% (Texas) of the disparity intact and undetectable from the predictions alone -- consistent not only in mechanism but in approximate magnitude across two states with different demographics and income distributions, and stable across ten independent train/test splits in each. A fairness constraint that appears fully satisfied at decision time can mask a substantial, systematic violation of the outcome it was meant to protect.
Figures & tables
Symbol
Meaning
n
Population size
ai∈{0,1}
Group membership of individual i
G0,G1
The two groups, Gg={i:ai=g}
n0,n1
Group sizes, ng=∣Gg∣
yi∈R
True outcome of individual i
y^i∈R
Predicted outcome of individual i
Table 1: Notation used throughout the paper.
Figure 1: True-outcome fairness gap as a function of prediction error σ ( n=4000 , 60–100 seeds per point, error bars show ±1 SE), with the fitted curve of Eq. ( 5 ).
California
Texas
Group sizes ( n0 , n1 )
121,006 , 8,557
105,835 , 12,529
Predictor R2
0.407
0.417
Unconstrained gap
-\19{,}112\pm$872$
-\16{,}357\pm$826$
Constrained true gap
-\6{,}577\pm$738$
-\5{,}846\pm$431$
Fraction of disparity retained
34.4%±3.3%
35.7%±1.8%
Table 2: Real-data results, California vs. Texas (10-split mean ± std).
Algorithmic fairness studies how automated decisions distribute errors, opportunities, and harms across groups, and how to measure and reduce disparities.
Exact demographic parity requires the same prediction distribution across sensitive groups. In regression, this can be too restrictive and reduce predictive accuracy. We instead bound every pairwise Wasserstein distance between group-specific prediction distributions by a chosen tolerance, allowing the groups to differ while keeping fairness tunable. On the real line, the prediction and constrained transport problems have the same infimum. If the transport problem attains this value and suitable Monge maps exist, a deterministic transport-form predictor realizes it. Under a standard one-dimensional condition, the problem becomes a bounded quantile optimization that is convex when the conditional expected loss is convex in prediction. Under squared loss with two positively weighted groups, the W1, W2, and W∞ constraints yield soft-thresholding, shrinkage, and clipping. Under the stated conditions, we establish rates for the known-cost plug-in estimator. When the cost is unavailable, we describe an observed-loss computational proxy. Synthetic and California 2018 ACS studies illustrate the tunable fairness-accuracy trade-off and distinguish calibration feasibility from held-out fairness and accuracy.
Demographic parity (DP) is a widely used group fairness criterion requiring predictive distributions to be invariant across sensitive groups. While natural in classification, full distributional DP is often overly restrictive in regression and can lead to substantial accuracy loss. We propose a relaxation of DP tailored to regression, enforcing parity only at a finite set of quantile levels and/or score thresholds. Concretely, we introduce a novel (ℓ, Z)-fair predictor, which imposes groupwise CDF constraints of the form F f |S=s (z m ) = ℓ m for prescribed pairs (ℓ m , z m ). For this setting, we derive closed-form characterizations of the optimal fair discretized predictor via a Lagrangian dual formulation and quantify the discretization cost, showing that the risk gap to the continuous optimum vanishes as the grid is refined. We further develop a model-agnostic post-processing algorithm based on two samples (labeled for learning a base regressor and unlabeled for calibration), and establish finite-sample guarantees on constraint violation and excess penalized risk. In addition, we introduce two alternative frameworks where we match group and marginal CDF values at selected score thresholds. In both settings, we provide closed-form solutions for the optimal fair discretized predictor. Experiments on synthetic and real datasets illustrate an interpretable fairness-accuracy trade-off, enabling targeted corrections at decision-relevant quantiles or thresholds while preserving predictive performance.
Arthur Charpentier, Christophe Denis, Romuald Elie +2
UQAM · Université du Québec à Montréal, UQAM, Canada · Kyoto University, Japan +7
Many real-world systems rely on predictive models to inform decisions, and fairness concerns arise in both the prediction and decision stages. We introduce end-to-end fairness optimization (E2EFO) as a unifying framework that integrates fairness across the prediction-to-decision pipeline. We focus on resource allocation with group-based fairness: the prediction task estimates allocation impacts while limiting accuracy disparity across groups, and the decision task distributes those impacts equitably by optimizing a group-based alpha-fairness measure. Within this framework, we propose fair decision-focused learning (FDFL), a training paradigm that jointly accounts for prediction accuracy, prediction fairness, and decision regret -- the loss in decision fairness due to imperfect predictions. FDFL trains the predictor by gradient descent, combining the objective gradients through multi-task learning techniques. The core computational challenge is the decision Jacobian with respect to the predictor parameters: we derive exact closed-form formulas for a tractable class of fair allocation and apply a differentiable optimization layer in the general case. We further establish a finite-sample generalization bound for the scalarized FDFL objective. Numerical experiments on a healthcare-based single resource allocation and a synthetic multiple resource allocation illustrate the value of jointly accounting for prediction fairness and decision fairness in prediction-informed decision-making.
Yu Wang, Violet Xinying Chen
School of Business Stevens Institute of Technology