Organizations: Instituto de Computação, Universidade Estadual de Campinas, Campinas, Brazil · Department of Computer Science, Saarland University, Saarbrücken, Germany
Fair machine learning typically evaluates the societal costs of a decision policy -- the loss incurred by the decision-maker (DM) and the unfair distribution of costs among demographic groups of decision-subjects (DS) -- on static data, where reducing one often comes at the expense of the other. Yet decisions change the populations on which they act, and prior work has shown that fairness interventions designed on static data may fail to benefit, or even harm, the disadvantaged group over time. We investigate a complementary question: When can a policy that is Pareto-optimal on pre-deployment data be outperformed by a fairer policy once each is evaluated on the population it induces, reducing costs for one stakeholder without increasing them for the other? In a setting where qualification scores evolve in response to binary decisions and their outcomes, we show that under Rawlsian fairness, which never lowers any group's acceptance rate, placing sufficiently high weight on fairness reduces societal costs whenever accepting disadvantaged DS at the DM's break-even point leaves them in a state less costly to the DM than rejecting them. Under egalitarian fairness, which may lower the advantaged group's acceptance rate, this additionally requires the reduction in DM loss on the disadvantaged group to outweigh the increase on the advantaged group. Under both fairness notions, however, when DM loss is accuracy-based, improvements in the post-deployment population may arise from declining DS qualification, revealing that gains for the DM do not necessarily translate into better outcomes for DS. Our results show that policy-induced population dynamics can overturn apparent performance-fairness trade-offs, allowing fairer policies to reduce the delayed societal costs of decision-making.
Figures & tables
Figure 1: Illustration of the decision-making process in which stakeholders (DM and DS) assess the optimal trade-offs between performance and fairness, agree on a policy θ(λ) , and deploy it. Once deployed, the policy supports decisions that will reshape the population it acts on. Over time, policies incur costs from the updated distribution and may fail to remain optimal compared to other pre-deployment Pareto-optimal solutions.
Figure 2: DM loss and fairness of pre-deployment PO policies θ(λ) evaluated on pre- and post-deployment distributions. Points in purple are non-dominated , pink displays dominated policies.
Figure 3: Changes (indicated by arrows) of a DS from the disadvantaged group (marked by the black dot ∙ ) that is rejected by the pre-deployment DM optimal policy θ(0) but accepted by another policy θ(λ) with λ>0 . Their score updates to a new value based on acceptance/rejection and repayment/default. Left: increasing λ moves the threshold from θ(0) to θ(λ) , accepting the DS in the shaded band. Center: the DM’s cost of the best decision for any DS (x,g) , which is independent of the marginal distribution of scores. A DS in the band who repays moves to a higher score with less risk for the DM; ΔΓ is the DM’s expected gain from accepting rather than rejecting. Right: the DS cost, which is non-null with rejection. A DS that repays increases their score and remains accepted, while one that defaults drops below the threshold θ(λ) and gets rejected, as it already was under θ(0) .
Figure 4: Effects of fairness weight on threshold and losses with selective costs for the DM and rejection costs for the DS, under Rawlsian fairness (first row) and egalitarian fairness (second row). The last two columns show the change in post-deployment losses relative to θ(0) , with the brown dashed line displaying the other fairness notion.
Figure 5: Post-deployment DM loss and fairness of policies from the pre-deployment PO set S for three configurations of the ratio b+/(−b−) , with selective DM costs and rejection DS costs. Right: per-group thresholds, which do not depend on the dynamics. In all cases, fairness weights below λmin are post-deployment dominated, and λmin increases with the positive effect of acceptance b+/(−b−) .
Figure 6: Difference in post-deployment dominance between selective costs and accuracy costs when scores cannot increase and will only decrease with default. Values are displayed as the change in comparison to θ(0) . The two cost matrices yield the same policies, so fairness and qualification coincide; accuracy improves while qualification for Black applicants falls.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Empirical distribution of scores pg(x) and conditional probability of repayment α(x,g) estimated from FICO data to define the pre-deployment distribution Dpre .
Algorithmic fairness studies how automated decisions distribute errors, opportunities, and harms across groups, and how to measure and reduce disparities.
Algorithmic decision-making in high-stakes settings can have profound impacts on individuals and populations. While much prior work studies fairness in static settings, recent results show that enforcing static fairness constraints may exacerbate long-run disparities. Motivated by this tension, we study a stylized sequential selection problem in which a decision-maker repeatedly selects individuals, affecting both immediate utility and the population distribution over time. We introduce notions of group fairness for both the short and long term and theoretically analyze the trade-off between fairness and utility via the Price of Fairness (PoF). We characterize optimal and fair policies in the short term and show that the PoF can be large even when group distributions are nearly identical. In contrast, we show that long-term disparities can vanish under simple investment policies that achieve a low PoF. We also empirically validate these theoretical observations using both synthetic and real datasets.
Shahin Jabbari, Chen Wang
Drexel University · Rensselaer Polytechnic Institute
As machine learning (ML) systems increasingly shape access to credit, jobs, and other opportunities, the fairness of algorithmic decisions has become a central concern. Yet it remains unclear when enforcing fairness constraints in these systems genuinely improves outcomes for affected groups or instead leads to "leveling down," making one or both groups worse off. We address this question in a unified, population-level (Bayes) framework for binary classification under prevalent group fairness notions. Our Bayes approach is distribution-free and algorithm-agnostic, isolating the intrinsic effect of fairness requirements from finite-sample noise and from training and intervention specifics. We analyze two deployment regimes for ML classifiers under common legal and governance constraints: attribute-aware decision-making (sensitive attributes available at decision time) and attribute-blind decision-making (sensitive attributes excluded from prediction). We show that, in the attribute-aware regime, fair ML necessarily (weakly) improves outcomes for the disadvantaged group and (weakly) worsens outcomes for the advantaged group. In contrast, in the attribute-blind regime, the impact of fairness is distribution-dependent: fairness can benefit or harm either group and may shift both groups' outcomes in the same direction, leading to either leveling up or leveling down. We characterize the conditions under which these patterns arise and highlight the role of "masked" candidates in driving them. Overall, our results provide structural guidance on when pursuing algorithmic fairness is likely to improve group outcomes and when it risks systemic leveling down, informing fair ML design and deployment choices.
Yi Yang, Xiangyu Chang, Pei-yu Chen
Department of Information Systems, Arizona State University · Department of Information Systems and Intelligent Business, Xi’an Jiaotong University
Designing fair algorithmic decision systems requires balancing model performance with fairness toward affected individuals: More fairness might require sacrificing some performance and vice versa, yet the space of possible trade-offs is still poorly understood. We investigate fairness in binary prediction-based decision problems by conceptualizing decision making as a multi-objective optimization problem that simultaneously considers decision-maker utility and group fairness. We investigate the set of Pareto-optimal decision rules for arbitrary utility functions for decision maker, arbitrary population distributions, and a wide range of group fairness metrics. We find that the Pareto frontier consists of deterministic, group-specific threshold rules applied to individuals' success probability. This complements existing optimality theorems from literature which, for specific fairness constraints, posit lower-bound threshold rules only. However we also show that, depending on the used fairness metric, the Pareto frontier may include upper-bound threshold rules, thus preferring individuals with lower success probabilities. We show that the location of the Pareto frontier depends only on population characteristics, utility functions and fairness score, but not on the technical design of the algorithm - our findings hold for pre-, in-, and post-processing approaches alike. Our results generalize existing optimality theorems for fairness-constrained classification and extend them to generalized fairness metrics and fairness principles, and to partial fairness regimes. This paper connects formal fairness research with legal and ethical requirements to search for less discriminatory alternatives, offering a principled foundation for evaluating and comparing algorithmic decision systems.
Mieke Wilms, Christoph Heitz
Zurich University of Applied Sciences / University of Zurich, Switzerland · Zurich University of Applied Sciences, Switzerland