Machine Learning (ML) systems are increasingly used to support decision-making processes that affect individuals. However, these systems often rely on biased data, which can lead to unfair outcomes against specific groups. With the growing adoption of Automated Machine Learning (AutoML), the risk of intensifying discriminatory behaviours increases, as most frameworks primarily focus on model selection to maximise predictive performance. Previous research on fairness in AutoML had largely followed this trend, integrating fairness awareness only in the model selection or hyperparameter tuning, while neglecting other critical stages of the ML pipeline. This paper aims to study the impact of integrating fairness directly into the optimisation component of an AutoML framework that constructs complete ML pipelines, from data selection and transformations to model selection and tuning. As selecting appropriate fairness metrics remains a key challenge, our work incorporates complementary fairness metrics to capture different dimensions of fairness during the optimisation. Their integration within AutoML resulted in measurable differences compared to a baseline focused solely on predictive performance. Despite a 9.4% decrease in predictive power, the average fairness improved by 14.5%, accompanied by a 35.7% reduction in data usage. Furthermore, fairness integration produced complete yet simpler final solutions, suggesting that model complexity is not always required to achieve balanced and fair ML solutions.
Figures & tables
Table 1: Parametrisation used in the evaluation. (a) lists the parameters used to compare the two setups; (b) lists the common parameters used internally by EDCA to perform the optimisation.
Attribute
adult
credit-card
portuguese-bank-marketing (B.M.)
#Features
15
24
17
#Instances
48842
30000
45211
#Size
732630
720000
768587
Missing values
True
False
False
Negative Class Proportion
0.761
0.779
0.883
Positive Class Proportion
0.239
0.221
0.117
Table 2: Detailed description of the selected binary classification datasets. "Max Sensitive Prop." and "Min Sensitive Prop." indicate the proportion of the most represented and less represented category of any sensitive attribute, respectively.
Table 3: Average (± standard deviation) for the fairness, performance and data-related metrics, with the percentage of change (% Change) from Baseline to the Fairness-aware setup. Statistically significant differences are shown in bold. "Positive %" = proportion of the positive class, and "% Sensitives" = proportion of sensitive attributes from the total of features selected by EDCA. The arrows indicate if each metric indicate should increase ( ↑ ) or decrease ( ↓ ). (a) results after an initial average of the results achieved on 5-fold cross-validation per run. (b) only the best solution per run.
Figure 1: Relationship between fairness (average of DP [ 4 ] , EO [ 6 ] , ABROCA [ 5 ] ) and predictive performance ( MCC scaled to [0, 1] and TPR ) measured with the Pearson correlation across both setups for each dataset. Each point represents one of 150 solutions per setup and dataset.
Figure 2: Distribution of the data selection techniques selected on the final solutions across the two setups. It shows how many times each data selection technique appeared on a final solution from the total possible of 150 solutions (5 solutions from the 5-fold cross-validation across 30 seeds).
Figure 3: Frequency of each feature on the final solutions. It shows how many times each feature appeared on a final solution from the total possible of 150 solutions (5 solutions from the 5-fold cross-validation across 30 seeds). Highlighted in bold uppercase are the sensitive attributes.
Dataset
Setup
Age
Race
Gender
Education
Marital
adult
Original
0.074
0.008
0.332
-
-
Baseline
0.074±0.001
0.008±0.0
0.332±0.002
-
-
Fairness-Aware
0.074±0.002
0.008±0.001
0.332±0.004
-
-
credit-card
Original
-
-
0.396
0.0
0.002
Baseline
-
-
0.396±0.006
0.001±0.001
0.002±0.001
Fairness-Aware
-
-
0.397±0.006
0.001±0.001
0.002±0.0
Table 4: Proportion of the less represented category of each sensitive attribute. The results indicate the original proportion, and then the average (± standard deviation) proportions for each sensitive attribute category when the attribute was present on the final solutions across all seeds and folds.
Figure 4: Distribution of the classification models selected on the final solutions across the two setups. It shows how many times each model appeared on a final solution from the total possible of 150 solutions (5 solutions from the 5-fold cross-validation across 30 seeds).
Figure 5: Evolution of the performance and fairness metrics used in the Fitness Function (Equations 2 and 1 ). Each metric’s line is the average for all the runs and folds, containing the original values without the transformations applied for the fitness function. The graph was cut to the minimum of generations achieved in one of the runs.
AutoML, intended as the process of automating the application of machine learning to real-world problems, is a key step for AI popularisation. Most AutoML frameworks are not accounting for the potential lack of fairness in the training data and in the corresponding predictions. We introduce FairMind, a software prototype aiming to automatise fairness analysis at the dataset level. We achieve that by resorting to the assumptions of the standard fairness model, recently proposed by Plečko and Bareinboim. This allows for a sound fairness evaluation in terms of causal effects, based on counterfactual queries involving the target, possibly confounders and mediators, and the different values of an input feature we regard as protected. After the necessary data preprocessing, the tool implements a closed-form computation of the effects. LLMs are consequently exploited to generate accurate reports on the fairness levels detected in the training dataset. We achieve that in a zero-shot setup and show by examples the expected advantages with respect to a direct analysis performed by the LLM. To favour applications, extensions to ordinal protected variable and continuous targets and novel decomposition results are also discussed.
Alessia Berarducci, Eric Rossetto, Alessandro Antonucci +1
Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA), USI-SUPSI, Lugano, Switzerland
Striking an optimal balance between predictive performance and fairness continues to be a fundamental challenge in machine learning. In this work, we propose a post-processing framework that facilitates fairness-aware prediction by leveraging model ensembling. Designed to operate independently of any specific model internals, our approach is widely applicable across various learning tasks, model architectures, and fairness definitions. Through extensive experiments spanning classification, regression, and survival analysis, we demonstrate that the framework effectively enhances fairness while maintaining, or only minimally affecting, predictive accuracy.
Zhouting Zhao, Tin Lok James Ng
School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland.
The application of machine learning-based predictive algorithms to Anti-Money Laundering (AML) has grown rapidly, driven by the vast volume of financial transaction data available to banks. These algorithms are typically trained not only on transactional data but also on sensitive client information, which may raise fairness concerns. Despite this, AML detection systems remain largely underexplored from a fairness perspective, even though deeper analytical methods based on counterfactuals are now available. Such techniques enable the decomposition of the direct and indirect effects of potentially sensitive features on model predictions, thereby supporting the evaluation of whether their influence is acceptable from a fairness perspective. Closing this gap, we consider the synthetic IBM AMLSim transaction dataset and construct additional features of the country of an account and its average behaviour. This improves the predictive performance of diverse machine learning models, ranging from baseline decision trees to state-of-the-art graph neural networks. We assess the potential unfairness associated with these features through a counterfactual, path-specific effect analysis. This reveals that fairness violations tend to be more pronounced for models whose predictive performance benefits the most from the extended features. Such a finding highlights a concrete instance of the trade-off between predictive accuracy and fairness in AML applications, thus underscoring the urgency of a systematic fairness analysis in such critical domains.
Lea Multerer, Michele Inchingolo, David Kletz +3
IDSIA, Switzerland · UBS Switzerland AG and its affiliates, Germany