Risk screening and recommendation systems pass or block accounts that generate both value and loss. Under a blocked-value budget, a cap on expected value forgone through blocking, the optimal ranking orders accounts by expected value per unit expected loss, a ratio of conditional expectations. This rule can conflict with accepted harm or net-harm criteria, blocking accounts that another criterion would pass at the same budget. We call these cases account-level objections and define group complaint disparity as the difference between groups' normalized realized objections. Weighted binary cross-entropy (W-BCE) implements the value-per-loss order with one score. Objection penalties modify the ranking, while group multipliers scale those penalties differently by group, tracing tradeoffs between loss capture and complaint disparity. This preserves the value-budgeted ranking structure and makes the cost of reducing contestable blocks explicit. Across public recommendation, insurance, credit, and mortgage datasets, we compare undamped W-BCE with ordinary BCE using yield Gini, an area summary of loss capture versus blocked value. Logarithmic weight damping is evaluated separately. Held-out evaluations measure capture-disparity tradeoffs. Validation-selected group-aware complaint penalties improve complaint closure in some cohorts. A corrected comparison normalizes and enlarges candidate policy banks using validation-only price sweeps and retains all ten applicable sources. The source-level results do not establish a statistically significant average advantage over group-blind pricing at the fixed minimum retained loss capture. We also compare with blocked-count, blocked-value, and protected-loss parity postprocessors, which do not directly constrain realized objections, and measure objections in candidate-count and loss units.
Figures & tables
Figure 1. Value-weighted yield ranking on realized accounts. Each bar is an account with realized value ai and loss bi , and bars are sorted by realized value per unit loss ai/bi , from high to low. A vertical rank cut passes accounts to its left and blocks accounts to its right; equivalently, it blocks accounts at or below the cut value on the realized value-per-loss scale. Moving the cut from right to left blocks more accounts. The curves show, for each possible cut, the blocked value and loss prevented by the blocked suffix. The group-1 orange outline shows how lower-value accounts can be blocked more often under a common value-per-loss rule; the inset reports the group block counts for the shown cut, with G0 and G1 denoting groups 0 and 1. Markers 1 and 2 identify the similar-loss, different-value pair used in Figure 2 . An illustrative account ranking ordered by value per unit loss, with curves for blocked value and captured loss and an inset comparing group block counts. Lower value can lead to more blocking despite similar loss. Markers 1 and 2 identify the example pair in the objection-penalty figure.
Figure 2. Objection penalties for accounts 1 and 2 in Figure 1 . Subtracting penalties from loss changes the value-per-loss scores and the account order. Account 1 still passes; account 2 moves from block to pass. Dashed marks show the original loss on the priced bars. Raw and penalized value-and-loss bars for the two marked accounts in Figure 1, followed by their positions in the baseline and penalized rankings. Account 1 remains passed, and account 2 moves out of the blocked suffix.
Figure 3. Raw W-BCE minus BCE on ten datasets. Each pair shows yield-Gini and loss-capture deltas; whiskers are 95% paired Student t intervals over ten splits. Positive values favor W-BCE. Dataset suffixes describe capture: [V] validation-frozen threshold; [T] retrospective test-curve interpolation. A grouped horizontal bar chart pairs yield-Gini and loss-capture differences for each of ten datasets. Both metrics retain their paired confidence intervals and use the same dataset order. Capture protocol labels distinguish validation-frozen thresholds from retrospective test-curve interpolation.
Figure 4. Account-objection trade-offs relative to fitted W-BCE. Green points are nondominated priced policies. Gray shading is dominated by unpriced baselines; green shading shows the additional dominated region. Markers show W-BCE, BCE, and plug-in ratios with 95% paired-split intervals. Policies use validation-frozen thresholds and thinning at a 10% value cap. The test-mean frontier is descriptive; higher and farther right is better. Three panels show the measured Pareto trade-offs between loss capture and account-objection removal. In each panel, evaluated price policies dominate the W-BCE, BCE, and plug-in baseline points on test means. All baseline points are visible, including KuaiRand capture ratios above one.
Floor
Seven mean
p
Ten mean
p
0.90
+0.0391
0.8426
-0.0372
0.8008
0.95
+0.2185
0.3071
+0.0863
0.6007
0.98
+0.1969
0.3214
+0.0735
0.6333
1.00
+0.1093
0.5482
+0.0472
0.7127
Table 1. Corrected validation-selected value-unit comparison. Each mean averages source medians of paired held-out closure lift. The seven-source scope and all ten recorded sources are reported separately; p is a two-sided source t test. Prices are normalized and extended using validation only.
Source
Median lift
Australian MTPL
+0.5393
Belgian MTPL
+0.4448
Bondora
+0.4892
CMS provider fraud
-0.6403
EU health insurance
+0.4899
French portfolio
-0.0027
Table 2. Corrected value-unit group-aware versus group-blind complaint closure lift at the .95 capture floor. Each row is one source’s median paired held-out lift. Negative and zero effects are retained.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Sample size
Split
KuaiRand
1,186,059
64/16/20
Australian MTPL
67,856
64/16/20
freMTPL2
100,000
80/0/20
Belgian MTPL
100,000
80/0/20
Lending Club
100,000
80/0/20
Prosper
55,051
80/0/20
Appendix
Table 3. Score samples and training/validation/test percentages. Each dataset uses ten paired splits; SBA denotes the Small Business Administration.
Source
Cohorts
Test rows
Groups
Australian MTPL
2
13,571
Female/Male
Belgian MTPL
3
10,000–32,642
Female/Male
French portfolio
9
3,200–7,660
Female/Male
Bondora
1
10,006
Female/Male
EU health
1
35,207
F/M
Norwegian auto
1
10,000
0/1
Appendix
Table 4. Group-evaluation cohorts and held-out rows per split. Each fitted cohort has ten paired seeds; multiple fitted banks from one source remain one source for inference. F/M means Female/Male; these labels include their recorded case variants.
Cohort
Policy
L/L0
1−O/O0
V
Australian MTPL
W-BCE
1.000±0.000
0.000±0.000
0.100±0.002
Priced endpoint
0.599±0.125
1.000±0.000
0.098±0.004
Belgian MTPL
W-BCE
1.000±0.000
0.000±0.000
0.101±0.001
Priced endpoint
0.820±0.074
1.000±0.000
0.100±0.002
KuaiRand
W-BCE
1.000±0.000
0.000±0.000
0.101±0.002
Priced endpoint
1.769±0.258
1.000±0.000
0.086±0.004
Appendix
Table 5. Natural-cohort endpoints from the same corrected policy bank as the account-objection frontier figure. Entries are means ± 95% paired-split intervals. L/L0 is loss capture relative to fitted W-BCE, 1−O/O0 is its count-objection share removed, and V is held-out blocked value divided by total value. Each priced endpoint has zero objection in every test split and maximizes mean test capture among such evaluated policies. This endpoint selection is descriptive, not validation-selected performance. All families use the same validation cap and frozen thinning.
Figure 5. Denominator choice for group complaint disparity. The orange outline marks group 1, as in Figure 1 . Panel A shows the same global value-per-loss threshold applied to both groups. Panel B aggregates the resulting objections in value units. DgA counts blocked objection-bearing value and divides by all value in group g . Dividing instead by all objection-bearing value would report only the conditional blocking rate among accounts that could object. An illustrative account table grouped by protected group, followed by bars for total group value, objection-bearing value, and blocked objection-bearing value. The complaint rate divides the last quantity by total group value.
Source
Group gap
LP gap
Lift
Australian MTPL
0.0043
0.0132
+0.3291
Belgian MTPL
0.0019
0.0055
+0.1903
Bondora
0.0276
0.0508
+0.4322
CMS provider fraud
0.0264
0.0093
-0.7152
EU health
0.0026
0.0068
+0.3010
French portfolio
0.0079
0.0060
-0.0373
Appendix
Table 6. Group-aware prices versus linear-program (LP) mixtures of the group-blind account-price bank at the .95 capture floor. Gaps and lift Cgroup−CLP are separately computed source medians over held-out cells; CLP is the mixture complaint closure and positive lift favors group pricing. Both families use the same validation constraints and the enlarged normalized price axis. All ten recorded sources are retained.
Unit
Floor
Seven medians
Seven means
Ten medians
Ten means
Value
0.90
+0.0391
-0.1533
-0.0372
-0.5979
Value
0.95
+0.2185
+0.0977
+0.0863
-0.3979
Value
0.98
+0.1969
+0.1127
+0.0735
-0.3605
Value
1.00
+0.1093
+0.0158
+0.0472
-0.4358
Count
0.90
-0.0514
-0.2949
-0.0699
-0.2966
Count
0.95
+0.0995
-0.2123
+0.0321
-0.2080
Appendix
Table 7. Corrected closure-lift sensitivity across measurement units and capture floors. Each cell averages one source summary: medians or means. The seven-source and full ten-source scopes are distinct. Negative results are retained; all source test statistics and exact sign-flip results are in the frozen artifact.
Algorithmic fairness studies how automated decisions distribute errors, opportunities, and harms across groups, and how to measure and reduce disparities.
We study the problem of fair ranking in recommender systems. We provide a formal probabilistic framework in which ranking policies are distributions over permutations, and the utility of a user is the exposure given to the items they interact with, relative to that of an ideal ranking. We consider utility parity across user groups, as well as its category-aware refinement, conditional utility parity, which requires parity within each category of items. Following a recent view of fairness as distributional stability, we show that both criteria are equivalent to the stability of the utility under changes in the composition of user groups. Optimizing over mixtures of ranking policies leads to a second-order cone program, for which we provide generalization bounds. A numerical study validates our theoretical findings.
Gayane Taturyan, Charlotte Laclau, Stephan Clémençon
LTCI, Télécom Paris Institut Polytechnique de Paris, France
Scoring functions are used to represent the relevance of individual documents. In modern information retrieval or recommendation systems, they are often learned from data and play a pivotal role in ranking sets of documents or items in a way that maximizes utility to a query or user. With the recent interest in algorithmic fairness, the success of scoring has naturally led to methods that learn scores that simultaneously trade off fairness and utility. In this work, we show that in stark contrast with utility-centric objectives, scoring is sub-optimal in achieving all utility-fairness trade-offs. We establish this with a series of counter-examples with a generic fairness formulation. We show that the issue persists whether we have a deterministic scoring function or a randomized one, or whether we measure fairness at the scope of a single query or across multiple queries. On the positive side, we empirically demonstrate that semi-greedy post-processing has the potential to achieve much better trade-offs, often approaching the ideal of exhaustive post-processing in a tractable way.
Shubham Singh, Ian A. Kash, Mesrob I. Ohannessian
Department of Computer Science University of Illinois Chicago Chicago, IL 60607 · Department of Electrical and Computer Engineering University of Illinois Chicago Chicago, IL 60607
One of the many fairness definitions pursued in recent recommender system research targets mitigating demographic information encoded in model representations. Models optimized for this definition are typically evaluated on how well demographic attributes can be classified given model representations, with the (implicit) assumption that this measure accurately reflects recommendation parity, i.e., how similar recommendations given to different users are. We challenge this assumption by comparing the amount of demographic information encoded in representations with various measures of how the recommendations differ. We propose two new approaches for measuring how well demographic information can be classified given ranked recommendations. Our results from extensive testing of multiple models on one real and multiple synthetically generated datasets indicate that optimizing for fair representations positively affects recommendation parity, but also that evaluation at the representation level is not a good proxy for measuring this effect when comparing models. We also provide extensive insight into how recommendation-level fairness metrics behave for various models by evaluating their performances on numerous generated datasets with different properties.
Bjørnar Vassøy, Benjamin Kille, Helge Langseth
Norwegian University of Science and Technology, Trondheim, Norway