Overcoming the Resolution Limit: Significance-Aware Regularization for Intersectional Fairness
Organizations: Intesa Sanpaolo AI Research, Turin, Italy
Abstract
Intersectional fairness requires evaluating models across fine-grained subgroups. However, as group definitions become more granular, the sample size per subgroup shrinks, causing fairness estimates to become increasingly unstable. Existing in-processing methods are certainty-blind: they penalize observed disparities without distinguishing statistically certified unfairness from sampling noise. This leads to fairness overfitting, where models sacrifice utility to "fix" spurious disparities in sparse regimes. To address this, we introduce the significant-excess metric family, a continuous, uncertainty-aware fairness score that measures only the portion of a subgroup's disparity exceeding what sampling variation can explain at a specific confidence level. Building on this metric, we propose SAFER (Significance-Aware Fairness Regularizer), a differentiable in-processing method that gates fairness penalties using a soft-count likelihood-ratio evidence statistic. SAFER satisfies a provable safety property: when every subgroup disparity lies well below the gate threshold, the unconstrained optimum is nearly stationary for SAFER. At that optimum, the back-propagated contribution of its significance-gated component is exponentially small, and the additional fairness contribution comes from a small ungated term with a fixed weight. Mechanism ablations across three benchmark datasets and purpose-built stress variants show that removing any component of SAFER (threshold smoothing, the significance gate, or a dense-tail term) produces the failure the theory predicts on at least one stress variant. In sparse intersectional settings, SAFER achieves a better fairness-utility Pareto frontier than the baseline methods, and it preserves utility when certified unfairness is absent.
Figures & tables
| Dataset | Level | Split | Targeted mechanism | |||
|---|---|---|---|---|---|---|
| Adult Income | 4 | 10/90 | 4,522 | 228 | 19.9 / 4.9 | threshold smoothing: high intersectionality, low budget |
| COMPAS | 3 | 20/80 | 993 | 35 | 28.4 / 14.2 | dense tail: small training set, strong regularization |
| German Credit | 3 | 50/50 | 500 | 32 | 15.6 / 10.9 | significance gate: safety regime |
| ACS Coverage | 3 | 50/50 | 25,000 | 71 | 352.1 / 103.5 | higher-sample ACS benchmark |
| Taiwan Credit | 4 | 50/50 | 15,000 | 285 | 52.6 / 11.6 | high-intersectionality credit risk |
| Adult Income | COMPAS | German Credit | Taiwan | ACS | |
| SAFER | |||||
| EO-Penalty | |||||
| FAIRRET Norm 1 | |||||
| FAIRRET Norm 2 | |||||
| AdvDebiasing | |||||
| ExponGrad |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Experiment | Method | Grid |
|---|---|---|
| Adult L4 10/90 main | SAFER | |
| Adult L4 10/90 main | EO-Penalty | |
| Adult L4 10/90 main | FAIRRET Norm 2 | |
| Adult L4 10/90 ablation | SAFER / NoThresh / NoTail | |
| Adult L4 10/90 ablation | NoSelectivity | |
| COMPAS L3 50/50 main | SAFER |
| Experiment | Method | jobs | Split seed runs per job | Estimated wall time |
|---|---|---|---|---|
| Adult L4 10/90 main | SAFER | 14 | 3–6 h | |
| Adult L4 10/90 main | EO-Penalty | 9 | 1–1.5 h | |
| Adult L4 10/90 main | FAIRRET Norm 1 / Norm 2 | 12–13 | 20–45 min | |
| COMPAS L3 50/50 main | SAFER | 9 | 20–40 min | |
| COMPAS L3 50/50 main | EO-Penalty | 14 | 15–30 min | |
| COMPAS L3 50/50 main | FAIRRET Norm 1 / Norm 2 | 9–13 | 5–10 min |
| Method | Parameter | Eligible | Log loss | AUC | MSEE |
|---|---|---|---|---|---|
| SAFER | yes | ||||
| EO-Penalty | yes | ||||
| FAIRRET Norm 1 | yes | ||||
| FAIRRET Norm 2 | yes | ||||
| AdvDebiasing | no | ||||
| ExponGrad | no |
| Method | Parameter | Eligible | Log loss | AUC | MSEE |
|---|---|---|---|---|---|
| SAFER | yes | ||||
| EO-Penalty | yes | ||||
| FAIRRET Norm 1 | yes | ||||
| FAIRRET Norm 2 | yes | ||||
| AdvDebiasing | no | ||||
| ExponGrad | no |
| Method | Parameter | Eligible | Log loss | AUC | MSEE |
|---|---|---|---|---|---|
| SAFER | yes | ||||
| EO-Penalty | yes | ||||
| FAIRRET Norm 1 | yes | ||||
| FAIRRET Norm 2 | yes | ||||
| AdvDebiasing | no | ||||
| ExponGrad | no |
| Method | Parameter | Eligible | Log loss | AUC | MSEE |
|---|---|---|---|---|---|
| SAFER | yes | ||||
| EO-Penalty | yes | ||||
| FAIRRET Norm 1 | yes | ||||
| FAIRRET Norm 2 | yes | ||||
| AdvDebiasing | no | ||||
| ExponGrad | yes |
| Method | Parameter | Eligible | Log loss | AUC | MSEE |
|---|---|---|---|---|---|
| SAFER | yes | ||||
| EO-Penalty | yes | ||||
| fairret-norm1 | yes | ||||
| fairret-norm2 | yes | ||||
| AdvDebiasing | no | ||||
| ExponGrad | yes |
| 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | ||
|---|---|---|---|---|---|---|---|---|---|---|
| Adult | log-loss | |||||||||
| MSEE | ||||||||||
| accuracy | ||||||||||
| COMPAS | log-loss | |||||||||
| MSEE | ||||||||||
| accuracy |
| log-loss S/W | MSEE S/W | ||
|---|---|---|---|
| Adult | |||
| COMPAS |
| probe stratum | cells | planted gap | mean | certified fraction | |
|---|---|---|---|---|---|
| null | 10 | ||||
| weak evidence | 12 | ||||
| boundary | 8 | ||||
| strong evidence | 4 |
| set | arm | test log-loss | null | weak | boundary | strong | |
|---|---|---|---|---|---|---|---|
| – | ERM | ||||||
| A | SAFER | ||||||
| NS | |||||||
| EO | |||||||
| B | SAFER | ||||||
| NS |
| arm | test log-loss | null | weak | boundary | strong | |
|---|---|---|---|---|---|---|
| ERM | ||||||
| SAFER | ||||||
| SAFER | ||||||
| SAFER | ||||||
| SAFER | ||||||
| SAFER |
| test | train | |||||||||
| arm | test log-loss | null | weak | boundary | strong | null | weak | boundary | strong | |
| ERM | ||||||||||
| SAFER | ||||||||||
| SAFER | ||||||||||
| SAFER | ||||||||||
| SAFER | ||||||||||
| mean | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| certified fraction | |||||||||
| cells (all runs) |
| Ablation | Adult L4 1090 | COMPAS L3 2080 | German L3 5050 |
|---|---|---|---|
| NoThreshSmooth (Theorem 2 ) | (no traction) | ||
| NoDenseTail (Eq. ( 12 )) | at high | at high | |
| NoSelectivity (Proposition 4 ) | , harm at high | harm |
| dataset (runs) | mean : initial / final | fairness loss: initial / final |
|---|---|---|
| Adult (10) | / | / |
| COMPAS (5) | / | / |
Algorithmic Fairness
Algorithmic fairness studies how automated decisions distribute errors, opportunities, and harms across groups, and how to measure and reduce disparities.
See all 452 papers