Scoring Is Not Enough: Addressing Gaps in Utility-fairness Trade-offs for Ranking

Fairness in decision-making processes is often quantified using probabilistic metrics. However, these metrics need not reflect the consequences of decisions for the affected individuals and groups. We develop a utility-based framework that incorporates these consequences into the assessment of fairness. Our main result shows that a decision-making process can satisfy $\varepsilon$-fairness while nevertheless being maximally unfair once the utilities associated with its outcomes are taken into account. To address applications in which information on false negatives is unavailable, we also formulate a reduced setting that retains the essential elements of the utility-based fairness assessment. We illustrate the framework through two applications: college admissions and credit-risk assessment. In both cases, probabilistic metrics may classify a decision-making process as approximately fair even though the corresponding utility outcomes are highly unequal. In the college-admissions example, our analysis shows that improving completion rates is necessary to achieve equality of utility across groups, while in the mortgage example, mitigating unfairness requires not only adjusting approval rates but also reducing the adverse consequences of default. These findings demonstrate that fairness assessments should account not only for the probabilities of different decisions but also for the consequences of those decisions.

Many fairness criteria constrain the policy or choice of predictors, which can have unwanted consequences, in particular, when optimizing the policy under such constraints. Here, we in- stead suggest that fairness can be directly analyzed as a property of the utility function. Instead of imposing fairness constraints on the policy, we suggest to simply maximize a utility function satisfying certain fairness properties. Concretely, we define value of information fairness, which prescribes that there must not be an incentive to infer the protected attribute. This principle sug- gests modifying utility functions such that they satisfy value of information fairness. We describe how such modifications can be achieved and discuss consequences for the corresponding optimal policies. We apply our framework to thought experiments and the COMPAS data, demonstrating that focusing on utility functions sometimes provides answers that better align with intuitive judg- ments about what is fair. Moreover, we are not aware of any intuitively fair policy that violates value of information fairness; and when we find that value of information fairness recommends an intuitively unfair policy, no realizable policy is intuitively fair.

Individual fairness, the notion that "similar individuals should be treated similarly," provides a strong and flexible fairness guarantee for algorithmic decision makers. However, a barrier to implementing individual fairness in practice is the difficulty of learning the similarity metric over individuals. In this work, we present an algorithm for learning a Mahalanobis similarity metric from triplet queries of the form "is individual $i$ more similar to individual $j$ or $k$?" We work in the standard Bradley-Terry model for pairwise comparisons. Our algorithm consists of a spectral initialization step followed by gradient descent. We provide extensive theoretical guarantees on our algorithm, showing that it converges quickly to the ground truth metric despite the non-convexity of the loss in our model. Because our focus is on fairness, we also show that individual fairness with respect to an estimated metric is sufficient to achieve similar fairness with respect to the true metric. We also discuss potential applications of our work to AI model tuning. Finally, we present experimental results that demonstrate the convergence of our algorithm and the fairness performance of downstream fair predictors trained on our estimated metric.