Integrating Feature Understandability into Feature Selection for Human-Centred Tabular Explanations
Authors: Nicola Rossberg, Bennett Kleinberg, Barry O'Sullivan, Luca Longo, Andrea Visentin
Organizations: School of Computer Science and Information Technology, University College Cork, Ireland · Centre for Research Training in Artificial Intelligence, University College Cork, Ireland · Department of Methodology and Statistics, Tilburg University · Insight Centre for Data Analytics, University College Cork, Ireland
Existing human-centred XAI research predominantly evaluates explanations after they have been generated. However, this fails to provide actionable insights into improving accessibility. This study incorporates human-based feature understandability into feature selection to improve the generation of human-centred explanations. Natural language explanations for machine learning models trained on tabular data often cite influential features in the input data to explain a given decision. As a consequence, users' ability to understand the produced explanation hinges on their understanding of the cited features. To quantify this natural-language feature understandability, Rossberg et al. proposed the Feature Understandability Scale, which assigns numerical `understandability scores' to features based on user feedback. To evaluate the degree to which these scores can improve machine learning explanation quality, this study collects understandability scores across two datasets, proposes an understandability-aware feature selection algorithm to select features that are both understandable and informative, and trains a random forest based on these feature subsets. The SHAP-based explanations generated by this model are evaluated using user feedback, and changes in explanation quality are assessed. The individual evaluation of the explanations showed that while the understandability-based explanations scored higher on average, the difference was not statistically significant. In contrast, the comparative evaluation found that participants preferred these explanations significantly more frequently than expected at the chance level. It is concluded that the integration of understandability can yield more understandable explanations at minimal cost to classification performance, although changes to quality require elucidation in larger cohorts.
Figures & tables
Figure 1: Graphical depiction of the theoretical basis on which the FUS implementation is expected to increase the understandability of explanations for tabular data.
Figure 2: Visualisation of the Repeated Monte Carlo Sampling, a) co-optimisation and b) informative setting of the feature selection, and machine learning model training methodology.
Figure 3: The Survey Flow of the evaluation procedure. Participants were randomly assigned to the comparative or individual evaluation, and the order of datasets was randomised for each condition. The individual setting used the explanation quality scale [ 67 ] to evaluate informative and co-optimised explanations separately, whereas the comparative setting prompted participants to select the more understandable explanation and provided prototypes of both co-optimised and informative explanations. Two or five instances were selected, respectively, and participants were asked to either use the scale to evaluate them (individual evaluation) or select the more understandable explanation (comparative evaluation).
Figure 4: Distribution of US across the two datasets
Telco Data
Mean
SD
Lower 95% CI
Higher 95% CI
Balanced Accuracy
0.733
0.009
0.731
0.736
F1
0.603
0.013
0.600
0.607
Precision
0.569
0.013
0.565
0.573
Recall
0.642
0.020
0.637
0.648
Medical Data
Table 1: Distributional Data of the Repeated Monte Carlo Sampling for both Datasets
Telco Dataset
Medical Dataset
Co-Optimised
Informative
Co-Optimised
Informative
Balanced Accuracy
75.25
75.35
75.54
75.40
F1
61.11
61.03
70.21
70.09
Precision
49.29
48.48
55.64
55.40
Recall
80.39
82.35
95.11
95.37
Mean US
4.16
3.96
3.79
3.54
Table 2: Results of the Co-Optimisation Procedure for both datasets
Telco Dataset
Medical Dataset
Co-Optimised
Informative
Co-Optimised
Informative
Internet Service, Tech Support , Total Charges, Monthly Charges, Contract
Tenure, Online Backup, Online Security, Tech Support , Contract
Table 3: Features selected for each dataset in the co-optimised and informative setting. The features are ranked by importance as determined by SHAP. Features selected by both settings are bolded.
Figure 5: Distribution of Scale ratings across the two datasets and settings.
Telco Dataset
Medical Dataset
Fixed Effects
Estimate
SE
95% CI
t (df)
p
Estimate
SE
95% CI
t (df)
p
(Intercept)
29.55
1.31
[26.95, 32.14]
22.64 (29.20)
< .001
31.16
0.99
[29.20, 33.12]
31.33 (61.86)
< .001
Exp. Type [Info]
− 1.03
0.75
[ − 2.51, 0.45]
− 1.38 (164.41)
.171
− 0.08
0.45
[ − 0.97, 0.80]
− 0.19 (167.18)
.851
Random Effects
SD (Coefficients)
SD (Int: Participant)
6.99
7.01
SD (Int: Example)
2.43
0.74
Table 4: Results of the mixed effects model analysis for the two datasets
Telco Dataset
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
Sum
Mean
SD
Informative
13
10
9
8
11
17
15
11
9
12
115
11.5
2.84
Co-Optimised
18
20
21
23
17
12
17
19
19
19
185
18.5
2.92
Medical Dataset
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
Sum
Mean
SD
Informative
5
13
7
4
15
12
17
9
9
9
100
10
4.22
Co-Optimised
26
18
23
27
11
19
15
20
20
21
200
20
4.78
Table 5: Results of the forced-choice analysis with the preferred option for each question in bold.
Telco Dataset
Medical Dataset
Fixed Effects
Estimate
SE
95% CI
z
p
Estimate
SE
95% CI
z
p
(Intercept)
0.60
0.20
[0.22, 1.02]
3.05
.002
0.92
0.28
[0.36, 1.55]
3.32
< .001
Derived Estimates
Back-transformed from the intercept
Odds ratio [95% CI]
1.82 [1.24, 2.67]
2.52 [1.46, 4.34]
P (co-optimised) [95% CI]
64.5% [55.3, 72.8]
71.6% [59.3, 81.3]
Random Effects
SD (Coefficients)
Table 6: Results of the mixed-effects logistic regression for the comparative evaluation (outcome: co-optimised explanation chosen = 1).
Number
Facet
Item
1
Actionability
I have learned something from the explanation
2
Causality rev.
The relationship between the input data and the predictions is vague
3
Comprehensibility rev.
The structure of the explanation is not clear
4
Efficiency rev.
It took me a long time to understand the explanation
5
Explicitness rev.
The explanation is incomprehensible
6
Informativeness
The explanation provides useful information
Table 7: Scale adapted from Vilone & Longo used for the individual explanation evaluation.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
I can understand the scale (units) of the feature.
x
I can easily understand if a given value of the feature is high or low.
x
I know what this feature measures.
x
I know what this feature represents.
x
I think that I can easily access a definition of the feature.
x
In my opinion, the feature should be used to predict the outcome.
x
Table 8: “Please rate the feature ‘Credit Score’ using the following questions. The scores of ‘Credit Score’ range from 18.4 to 118.65. The feature is used to predict whether or not a customer’s loan application will be approved. While rating the feature, consider if you would understand a loan decision, if it was justified using the feature. E.g. ’The customer did not get a loan because their Credit Score was 56.95’."