CAFP: A Post-Processing Framework for Group Fairness via Counterfactual Model Averaging

Irina Arévalo; Marcos Oliva

cs.AIApr 8, 2026

CAFP: A Post-Processing Framework for Group Fairness via Counterfactual Model Averaging

Authors: Irina Arévalo, Marcos Oliva

Organizations: School of Civil Engineering, Universidad Politecnica de Madrid, Profesor Aranguren 3, Madrid, 28040, Spain. · Bain&Company, Velazquez, Madrid, 28006, Spain.

Abstract

Ensuring fairness in machine learning predictions is a critical challenge, especially when models are deployed in sensitive domains such as credit scoring, healthcare, and criminal justice. While many fairness interventions rely on data preprocessing or algorithmic constraints during training, these approaches often require full control over the model architecture and access to protected attribute information, which may not be feasible in real-world systems. In this paper, we propose Counterfactual Averaging for Fair Predictions (CAFP), a model-agnostic post-processing method that mitigates unfair influence from protected attributes without retraining or modifying the original classifier. CAFP operates by generating counterfactual versions of each input in which the sensitive attribute is flipped, and then averaging the model's predictions across factual and counterfactual instances. We provide a theoretical analysis of CAFP, showing that it eliminates direct dependence on the protected attribute, reduces mutual information between predictions and sensitive attributes, and provably bounds the distortion introduced relative to the original model. Under mild assumptions, we further show that CAFP achieves perfect demographic parity and reduces the equalized odds gap by at least half the average counterfactual bias.

Figures & tables

Method	Fairness Type	Access to Model Internals	Uses Protected Attribute at Test Time	Retraining Required	Theoretical Guarantees
Equalized Odds Post-processing Hardt et al. (2016)	Group (EO)	No	Yes	No	Partial
Reject Option Classification Kamiran et al. (2012)	Group (DP, EO)	No	Yes	No	No
Calibrated Equalized Odds Pleiss et al. (2017)	Group (EO)	No	Yes	No	Partial
FairBatch Roh et al. (2021)	Group (DP, EO)	Yes	Yes	Yes	No
Counterfactual Equalized Odds Mishler et al. (2021)	Counterfactual	No	Yes	No	Yes
Path-specific Fairness Chiappa (2019)	Counterfactual (Path-based)	Yes	Yes	Yes	Yes

Table 1: Comparison of Fairness Methods Across Key Dimensions

Property	Guarantee
Attribute Independence	If $f(x,a)$ depends on $A$ only directly and $X\perp A$ , then $\hat{f}(X)$ is independent of $A$ :
	$I(\hat{f}(X);A)=0$
Bounded Distortion	The absolute difference between the original and CAFP prediction is half the counterfactual bias:
	$\|f(x,a)-\hat{f}(x)\|=\frac{1}{2}\|f(x,a)-f(x,1-a)\|$
	The expected distortion is similarly bounded:
	$\mathbb{E}[\|f(x,a)-\hat{f}(x)\|]\leq\frac{1}{2}\mathbb{E}[\|f(x,a)-f(x,1-a)\|]$

Table 2: Summary of Theoretical Guarantees for Counterfactual Averaging for Fair Predictions (CAFP)

Model	Acc.	95% CI	SD
LR	0.8464	[0.8457, 0.8471]	0.0035
LR + Eq.Odds	0.8185	[0.8176, 0.8194]	0.0044
LR + R. O.	0.7836	[0.7807, 0.7866]	0.0148
LR + CAFP	0.8432	[0.8425, 0.8440]	0.0037
RF	0.8411	[0.8404, 0.8418]	0.0036
RF + Eq.Odds	0.8138	[0.8130, 0.8145]	0.0038

Table 3: Model accuracy on Adult dataset with 95% Confidence Intervals (CI) and Standard Deviations (SD)

Model	AOD	95% CI	SD
LR	-0.0961	[-0.0990, -0.0933]	0.0142
LR + Eq.Odds	-0.0019	[-0.0057, 0.0019]	0.0192
LR + R. O.	0.1048	[0.1020, 0.1077]	0.0145
LR + CAFP	0.0075	[0.0047, 0.0104]	0.0143
RF	-0.0847	[-0.0879, -0.0816]	0.0159
RF + Eq.Odds	-0.0025	[-0.0067, 0.0017]	0.0213

Table 4: Average Odds Difference on Adult dataset with 95% Confidence Intervals (CI) and Standard Deviations (SD)

Model	DPD	95% CI	SD
LR	-0.1868	[-0.1882, -0.1854]	0.0071
LR + Eq.Odds	-0.0909	[-0.0928, -0.0889]	0.0097
LR + R. O.	-0.0461	[-0.0488, -0.0435]	0.0134
LR + CAFP	-0.1157	[-0.1174, -0.1140]	0.0085
RF	-0.1928	[-0.1944, -0.1912]	0.0083
RF + Eq.Odds	-0.1928	[-0.1944, -0.1912]	0.0083

Table 5: Demographic Parity Difference on Adult dataset with 95% Confidence Intervals (CI) and Standard Deviations (SD)

Panel of Figure 1 — Figure 1: Accuracy on Adult dataset. Error bars indicate 95% confidence intervals.

Panel of Figure 2 — Figure 2: AOD on Adult dataset. Error bars indicate 95% confidence intervals.

Panel of Figure 3 — Figure 3: DPD on Adult dataset. Error bars indicate 95% confidence intervals.

Model	Acc.	95% CI	SD
LR	0.6662	[0.6629, 0.6695]	0.014
LR + Eq.Odds	0.6200	[0.6166, 0.6233]	0.017
LR + R. O.	0.6539	[0.6506, 0.6572]	0.017
LR + CAFP	0.6653	[0.6627, 0.6679]	0.013
RF	0.6616	[0.6589, 0.6644]	0.014
RF + Eq.Odds	0.6225	[0.6176, 0.6274]	0.025

Table 6: Model accuracy on COMPAS dataset with 95% Confidence Intervals (CI) and Standard Deviations (SD)

Model	AOD	95% CI	SD
LR	-0.2034	[-0.2119, -0.1948]	0.0431
LR + Eq.Odds	-0.0017	[-0.0117, 0.0083]	0.0503
LR + R. O.	0.0123	[-0.0018, 0.0264]	0.0709
LR + CAFP	-0.1305	[-0.1372, -0.1238]	0.0337
RF	-0.1491	[-0.1607, -0.1374]	0.0587
RF + Eq.Odds	0.0047	[-0.0049, 0.0143]	0.0484

Table 7: Average Odds Difference on COMPAS dataset with 95% Confidence Intervals (CI) and Standard Deviations (SD)

Model	DPD	95% CI	SD
LR	-0.2356	[-0.2436, -0.2276]	0.0403
LR + Eq.Odds	-0.0329	[-0.0423, -0.0236]	0.0473
LR + R. O.	-0.0220	[-0.0356, -0.0084]	0.0686
LR + CAFP	-0.1631	[-0.1692, -0.1570]	0.0308
RF	-0.1825	[-0.1933, -0.1718]	0.0542
RF + Eq.Odds	-0.0273	[-0.0367, -0.0178]	0.0476

Table 8: Demographic Parity Difference on COMPAS dataset with 95% Confidence Intervals (CI) and Standard Deviations (SD)

Panel of Figure 4 — Figure 4: Accuracy on COMPAS dataset. Error bars indicate 95% confidence intervals.

Panel of Figure 5 — Figure 5: AOD on COMPAS dataset. Error bars indicate 95% confidence intervals.

Panel of Figure 6 — Figure 6: DPD on COMPAS dataset. Error bars indicate 95% confidence intervals.

Model	Acc.	95% CI	SD
LR	0.7415	[0.7359, 0.7471]	0.0284
LR + Eq.Odds	0.7189	[0.7111, 0.7266]	0.0390
LR + R. O.	0.6842	[0.6754, 0.6929]	0.0440
LR + CAFP	0.7379	[0.7321, 0.7438]	0.0293
RF	0.7547	[0.7495, 0.7599]	0.0265
RF + Eq.Odds	0.7390	[0.7331, 0.7449]	0.0298

Table 9: Model accuracy on German Credit dataset with 95% Confidence Intervals (CI) and Standard Deviations (SD)

Model	AOD	95% CI	SD
LR	-0.0961	[-0.1142, -0.0779]	0.0915
LR + Eq.Odds	-0.0139	[-0.0323, 0.0045]	0.0929
LR + R. O.	0.0248	[0.0037, 0.0460]	0.1067
LR + CAFP	-0.0324	[-0.0480, -0.0169]	0.0786
RF	-0.0437	[-0.0580, -0.0293]	0.0724
RF + Eq.Odds	-0.0027	[-0.0210, 0.0156]	0.0923

Table 10: Average Odds Difference on German Credit dataset with 95% Confidence Intervals (CI) and Standard Deviations (SD)

Model	DPD	95% CI	SD
LR	-0.1068	[-0.1228, -0.0908]	0.0806
LR + Eq.Odds	-0.0289	[-0.0452, -0.0126]	0.0824
LR + R. O.	0.0055	[-0.0159, 0.0269]	0.1077
LR + CAFP	-0.0505	[-0.0647, -0.0364]	0.0713
RF	-0.0599	[-0.0719, -0.0480]	0.0603
RF + Eq.Odds	-0.0217	[-0.0364, -0.0070]	0.0740

Table 11: Demographic Parity Difference on German Credit dataset with 95% Confidence Intervals (CI) and Standard Deviations (SD)

Panel of Figure 7 — Figure 7: Accuracy on German Credit dataset. Error bars indicate 95% confidence intervals.

Panel of Figure 8 — Figure 8: AOD on German Credit dataset. Error bars indicate 95% confidence intervals.

Panel of Figure 9 — Figure 9: DPD on German Credit dataset. Error bars indicate 95% confidence intervals.

Panel of Figure 10 — Figure 10: Balanced Accuracy vs. DPD across thresholds - Adult dataset

Panel of Figure 11 — Figure 11: Balanced Accuracy vs. DPD across thresholds - COMPAS dataset

Panel of Figure 12 — Figure 12: Balanced Accuracy vs. DPD across thresholds - German Credit dataset

Prediction Type	Accuracy	DPD	AOD
Factual $f(x,a)$	0.8464	-0.1868	-0.0961
Counterfactual
$f(x,1-a)$	0.8463	-0.1775	-0.0912
CAFP $\hat{f}(x)$	0.8432	-0.1157	0.0075

Table 12: Ablation study comparing factual-only, counterfactual-only, and averaged predictions (Adult dataset, logistic regression).

Explore similar work

A Model Ensemble-Based Post-Processing Framework for Fairness-Aware Prediction
Mar 19, 2026Zhouting Zhao, Tin Lok James NgAlgorithmic Fairness Debiased ML
School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland.
Tuning Derivatives for Causal Fairness in Machine Learning
May 7, 2026Filip Edström, Guilherme W. F. Barros, Tetiana Gorbach +1Structural Causal Models Algorithmic Fairness
Department of Statistics, Umeå School of Business, Economics and Statistics, Umeå University, Umeå, 901 87, Sweden. · Integrated Science Lab, Department of Physics, Ume˚a University, Ume˚a, 901 87, Sweden. · Department of Statistics, Ume˚a School of Business, Economics and Statistics, Ume˚a University, Ume˚a, 901 87, Sweden.
Do Fair Models Reason Fairly? Counterfactual Explanation Consistency for Procedural Fairness in Credit Decisions
May 12, 2026Gideon Popoola, John SheppardFeature Attribution Counterfactual Explanations
Gianforte School of Computing Montana State University Bozeman, MT, USA, 59715

Mar 19, 2026J/K move · Enter open · S save

A Model Ensemble-Based Post-Processing Framework for Fairness-Aware Prediction

Zhouting Zhao, Tin Lok James Ng

School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland.

Algorithmic Fairness Debiased ML

Open paper AskPDF arXiv

Striking an optimal balance between predictive performance and fairness continues to be a fundamental challenge in machine learning. In this work, we propose a post-processing framework that facilitates fairness-aware prediction by leveraging model ensembling. Designed to operate independently of any specific model internals, our approach is widely applicable across various learning tasks, model architectures, and fairness definitions. Through extensive experiments spanning classification, regression, and survival analysis, we demonstrate that the framework effectively enhances fairness while maintaining, or only minimally affecting, predictive accuracy.

Abstract

Explore similar work

A Model Ensemble-Based Post-Processing Framework for Fairness-Aware Prediction

Tuning Derivatives for Causal Fairness in Machine Learning

Do Fair Models Reason Fairly? Counterfactual Explanation Consistency for Procedural Fairness in Credit Decisions