Operationalizing Fairness: Post-Hoc Threshold Optimization Under Hard Resource Limits
Organizations: Department of Computer Science and Engineering, DUIET, Dibrugarh University, Assam, India · Department of Computer Science and Engineering, MIT, Manipur University, 795003, India
Abstract
The deployment of machine learning in high-stakes domains requires a balance between predictive safety and algorithmic fairness. However, existing fairness interventions often as- sume unconstrained resources and employ group-specific decision thresholds that violate anti- discrimination regulations. We introduce a post-hoc, model-agnostic threshold optimization framework that jointly balances safety, efficiency, and equity under strict and hard capacity constraints. To ensure legal compliance, the framework enforces a single, global decision thresh- old. We formulated a parameterized ethical loss function coupled with a bounded decision rule that mathematically prevents intervention volumes from exceeding the available resources. An- alytically, we prove the key properties of the deployed threshold, including local monotonicity with respect to ethical weighting and the formal identification of critical capacity regimes. We conducted extensive experimental evaluations on diverse high-stakes datasets. The principal re- sults demonstrate that capacity constraints dominate ethical priorities; the strict resource limit determines the final deployed threshold in over 80% of the tested configurations. Furthermore, under a restrictive 25% capacity limit, the proposed framework successfully maintains high risk identification (recall ranging from 0.409 to 0.702), whereas standard unconstrained fairness heuristics collapse to a near-zero utility. We conclude that theoretical fairness objectives must be explicitly subordinated to operational capacity limits to remain in deployment. By decou- pling predictive scoring from policy evaluation and strictly bounding intervention rates, this framework provides a practical and legally compliant mechanism for stakeholders to navigate unavoidable ethical trade-offs in resource-constrained environments.
Figures & tables
| Baseline Policy | Operational Interpretation | Relationship to Framework |
|---|---|---|
| Performance-Optimal | Maximizes aggregate accuracy | Ignores and |
| Risk-Averse | Prioritizes safety (minimizes FN) | Extreme weighting, ignores |
| Inclusion-Oriented | Pure capacity enforcement | Defaults strictly to |
| Fairness-Aware | Minimizes disparity ( ) constraint | High weighting, ignores |
| Dataset | Strategy | Recall | Disparity ( TPR) | Intervention Rate | Feasibility |
|---|---|---|---|---|---|
| ACS Income | Proposed Framework | 0.702 | 0.254 | 0.250 | Feasible |
| Unconstrained Optimization | 0.993 | 0.000 | 0.990 | Infeasible | |
| Demographic Parity | 0.181 | 0.000 | 0.172 | Feasible (low utility) | |
| Equalized Odds | 0.260 | 0.000 | 0.250 | Feasible | |
| Random Allocation | 0.250 | 0.000 | 0.250 | Feasible (no signal) | |
| COMPAS | Proposed Framework | 0.409 | 0.317 | 0.250 | Feasible |
| C | Mean Recall | Mean TPR | Constraint Activation (%) | Feasible Region (%) |
|---|---|---|---|---|
| 0.10 | 0.255 | 0.142 | 98.7 | 1.3 |
| 0.15 | 0.312 | 0.138 | 94.2 | 5.8 |
| 0.20 | 0.368 | 0.131 | 87.6 | 12.4 |
| 0.25 | 0.425 | 0.127 | 80.8 | 19.2 |
| 0.30 | 0.525 | 0.119 | 68.4 | 31.6 |
| 0.40 | 0.682 | 0.108 | 42.1 | 57.9 |
| Dataset | Model Architecture | Recall (95% CI) | Disparity (95% CI) | Constraint Active |
|---|---|---|---|---|
| ACS Income | Gradient Boosting | 0.613 [0.603, 0.623] | 0.068 [0.066, 0.071] | 86.9% |
| Logistic Regression | 0.556 [0.546, 0.566] | 0.094 [0.092, 0.097] | 82.6% | |
| Random Forest | 0.542 [0.528, 0.555] | 0.196 [0.192, 0.200] | 90.0% | |
| COMPAS | Gradient Boosting | 0.319 [0.310, 0.329] | 0.153 [0.149, 0.158] | 83.4% |
| Logistic Regression | 0.281 [0.271, 0.290] | 0.167 [0.162, 0.173] | 79.6% | |
| Random Forest | 0.275 [0.264, 0.286] | 0.204 [0.197, 0.211] | 77.3% |
| Method | Stage | Model-agnostic | Constraint Handling | Multi-obj. | |
| Budget | Hard limit | ||||
| Hardt et al. [ 15 ] | Post | ✓ | ✗ | ✗ | |
| Pleiss et al. [ 16 ] | Post | ✓ | ✗ | ✗ | ✗ |
| Zafar et al. [ 21 ] | In | ✗ | ✗ | ✗ | |
| Corbett-Davies et al. [ 28 ] | Post | ✓ | ✗ | ||
| Xian et al. [ 17 ] | Post | ✓ | ✗ | ✗ | |
Algorithmic Fairness
Algorithmic fairness studies how automated decisions distribute errors, opportunities, and harms across groups, and how to measure and reduce disparities.
See all 419 papers