Interpretable and Fair Generalized Additive Neural Networks via Multi-objective Learning
Authors: Ziming Wang, Changwu Huang, Ke Tang, Yew-Soon Ong, Xin Yao
Organizations: Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China · School of AI and Liberal Arts, Beijing Normal-Hong Kong Baptist University, Zhuhai, 519087, China · Agency for Science, Technology and Research (A*STAR), 138632, Singapore · College of Computing and Data Science, Nanyang Technological University, 639798, Singapore · School of Data Science, Lingnan University, Hong Kong, China
Interpretability and fairness are two of the most emphasized dimensions in trustworthy artificial intelligence (AI). Various explainable AI methods have been introduced to improve interpretability. This paper focuses on neural network (NN)-based generalized additive models (GAMs), a class of self-interpretable models. While most existing research has prioritized improving the accuracy of NN-based GAMs, their interpretability remains largely underexplored. To address this gap, this paper introduces explicit quantitative metrics for evaluating the interpretability of NN-based GAMs, empirically examines their effectiveness, and explores strategies for improving interpretability within these models. In addition, the simultaneous and explicit optimization of both interpretability and fairness, along with their trade-offs and the underlying reasons, remains underexplored. To address this, we propose a multi-objective neural basis model (MONBM) framework based on multi-objective evolutionary learning to consider accuracy, interpretability, and fairness simultaneously. A partial retraining strategy is further developed to facilitate the practical application of evolutionary multi-objective optimization to deep model architectures. Based on MONBM, this paper reveals the complex relationships between these dimensions and the reasons behind these intricate relationships. This analysis demonstrates how multi-objective optimization can be combined with self-interpretable models to reveal relationships among trustworthiness objectives. In addition, MONBM obtains a set of models with different trade-offs between dimensions, and the competitiveness of the approach is validated by comparing it with state-of-the-art methods.
Figures & tables
Figure 1: Examples of explanations provided by GAM. For the global explanation, as shown in Fig. 1(a) , the x-axis represents the input feature values xi , while the y-axis corresponds to the values of the shape function fi(xi) . For the local explanation, as shown in Fig. 1(b) , the y-axis represents the features xi , and the x-axis represents the shape function value fi(xi(j)) for the corresponding feature xi(j) of data point x(j) .
Figure 2: NBM architecture for the binary classification task, the figure taken from [ 42 ] with slight modifications.
Method
Category
Fairness
LI
GI
GI
(Complexity)
(Smoothness)
(Monotonicity)
Spline-GAM [ 17 ]
Spline
✘
✘
✔
✘
EBM [ 40 ]
Tree
✘
✘
✘
✘
NAM [ 1 ]
NN
✘
✘
✘
✘
NBM [ 42 ]
NN
✘
✘
✘
✘
SNAM [ 52 ]
NN
✘
✔
✘
✘
Table 1: Comparison of the proposed MONBM framework with existing GAM-based approaches.
Figure 3: The computational pipeline of the MONBM instantiation.
Figure 4: Comparison of global explanations of continuous features to validate the proposed smoothness and monotonicity metrics. For each dataset, the blue, red, and green lines represent the global explanation results of the continuous features by the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. Each continuous feature has a smoothness requirement, and features highlighted in red also have a monotonicity requirement.
Adult
Bank
COMPAS
ACC ( % ) ↑
DP ↓
ACC ( % ) ↑
DP ↓
ACC ( % ) ↑
DP ↓
Best CE
86.36
0.1686 (+0.1685)
90.26
0.0357 (+0.0357)
68.13
0.2025 (+0.2023)
“Middle DP”
85.61 (-0.75)
0.0851 (+0.0850)
90.21 (-0.05)
0.0176 (+0.0176)
67.77 (-0.35)
0.1041 (+0.1039)
Best DP
76.60 (-9.76)
0.0001
88.76 (-1.49)
0.0000
53.86 (-14.27)
0.0002
Default
Dutch
LSAT
ACC ( % ) ↑
DP ↓
ACC ( % ) ↑
DP ↓
ACC ( % ) ↑
DP ↓
Table 2: Comparison of the mean values of the ACC and DP on the test set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CD instantiation over 15 trials. ACC values are expressed as percentages ( % ). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.
Figure 5: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, the blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle DP ”, and best DP points obtained by the MONBM-CD instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The sensitive attribute is highlighted in red. The Pearson correlation coefficient between each attribute and the sensitive attribute is annotated in the respective subplots.
Adult
Bank
COMPAS
ACC ( % ) ↑
LI ↓
ACC ( % ) ↑
LI ↓
ACC ( % ) ↑
LI ↓
Best CE
86.36
1.0271 (+0.9171)
90.25
1.1649 (+1.0811)
68.29
0.7289 (+0.4403)
“Middle LI”
84.81 (-1.55)
0.5720 (+0.4619)
89.75 (-0.50)
0.6292 (+0.5454)
61.51 (-6.78)
0.5102 (+0.2216)
Best LI
65.55 (-20.81)
0.1100
83.23 (-7.03)
0.0838
53.53 (-14.76)
0.2886
Default
Dutch
LSAT
ACC ( % ) ↑
LI ↓
ACC ( % ) ↑
LI ↓
ACC ( % ) ↑
LI ↓
Table 3: Comparison of the mean values of the ACC and LI on the test set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CL instantiation over 15 trials. ACC values are expressed as percentages ( % ). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.
Figure 6: Comparison of the average local explanations of the test set to assess the impact of improved LI on each feature. The three columns respectively display the average local explanation results of the model corresponding to: (1) the best CE point, (2) the “middle LI ” or “middle DP ” point, and (3) the best LI or best DP point, obtained by the MONBM-CL or MONBM-CD instantiation on the Bank dataset in one arbitrary trial. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.
Adult
Bank
COMPAS
ACC ( % ) ↑
GI ↓
ACC ( % ) ↑
GI ↓
ACC ( % ) ↑
GI ↓
Best CE
85.70
0.1651 (+0.1338)
89.47 (-0.44)
0.2212 (+0.1779)
68.51 (-0.08)
0.0129 (+0.0019)
“Middle GI”
85.57 (-0.13)
0.1105 (+0.0792)
89.91
0.0791 (+0.0358)
68.59
0.0120 (+0.0009)
Best GI
81.01 (-4.69)
0.0313
89.65 (-0.26)
0.0433
68.19 (-0.40)
0.0111
Default
Dutch
LSAT
ACC ( % ) ↑
GI ↓
ACC ( % ) ↑
GI ↓
ACC ( % ) ↑
GI ↓
Table 4: Comparison of the mean values of the ACC and GI on the test set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CG instantiation over 15 trials. ACC values are expressed as percentages ( % ). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.
Figure 7: Comparison of global explanations to assess the impact of improved GI on each feature. The blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation on the Bank dataset in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The continuous features are highlighted in red.
Figure 8: Comparison of the average local explanations of the test set to assess the impact of improved GI on each feature. The three columns represent the average local explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation on the Bank dataset in one arbitrary trial, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.
Adult
Bank
ACC ( % ) ↑
DP ↓
LI ↓
GI ↓
ACC ( % ) ↑
DP ↓
LI ↓
GI ↓
Majority Baseline
76.1
—
—
—
88.3
—
—
—
Best CE
85.7
0.174 (+0.17)
1.026 (+0.92)
0.172 (+0.14)
89.5
0.040 (+0.04)
1.160 (+1.07)
0.229 (+0.20)
Best DP
65.6 (-20.1)
0.000
0.124 (+0.02)
0.155 (+0.12)
88.3 (-1.20)
0.000
0.525 (+0.43)
0.107 (+0.08)
Best LI
65.7 (-20.0)
0.000 (+0.00)
0.108
0.167 (+0.13)
83.2 (-6.30)
0.000 (+0.00)
0.095
0.301 (+0.27)
Best GI
69.7 (-16.0)
0.008 (+0.01)
0.463 (+0.36)
0.036
88.3 (-1.20)
0.001 (+0.00)
0.751 (+0.66)
0.032
Table 5: Comparison of the mean values of the ACC , DP , LI , and GI on the test set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLG instantiation over 15 trials. ACC values are expressed as percentages ( % ). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses. The “Majority Baseline” row represents the accuracy of a constant classifier that consistently predicts the majority class for each dataset.
Figure 9: Comparison of the average local explanations of the test set to explore the interrelationships among the four objectives. For each dataset, the four columns represent the average local explanation results of the model corresponding to the best CE , best DP , best LI , and best GI points obtained by the MONBM-CDLG instantiation in one arbitrary trial, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.
Figure 10: Distribution of all MONBM-CDLG non-dominated solutions across pairs of trustworthy objectives to examine their interrelationships. Each subplot visualizes the solutions obtained from one arbitrary run of the MONBM-CDLG instantiation on a given dataset, showing pairwise relationships between DP , LI , and GI .
HV ↑
MS ↑
PD ↑
MONBM
FairEMOL
MONBM
FairEMOL
MONBM
FairEMOL
Adult
0.8849 (0.0183)
0.8678 (0.0203) -
0.2597 (0.0164)
0.2302 (0.0096) -
101910 (24450)
230.48 (27.923) -
Bank
0.9398 (0.0087)
0.8678 (0.0103) -
0.0866 (0.0058)
0.0568 (0.0029) -
26207 (7208.7)
46.04 (9.5900) -
COMPAS
0.6156 (0.0402)
0.6065 (0.0411) -
0.2458 (0.0443)
0.2938 (0.0443) +
75825 (21868)
191.89 (49.975) -
Default
0.8381 (0.0266)
0.8321 (0.0272) -
0.0457 (0.0067)
0.0465 (0.0128) ≈
19365 (4974.1)
30.611 (7.9945) -
Dutch
0.9092 (0.0136)
0.8951 (0.0160) -
0.8504 (0.3369)
0.4417 (0.0097) -
266770 (62707)
334.09 (77.768) -
Table 6: The average of 15 trials of HV , MS , and PD values for the final model set. “+/ ≈ /-” indicates that the average FairEMOL metric value is better/approximate/worse than the corresponding MONBM value on the Wilcoxon rank-sum test, with a significance level of 0.05. The best average value is marked in gray.
Fair-NBM
LFR-NBM
Re-NBM
ROC-NBM
LPP-NBM
Oracle-NBM
FRAPPÉ-NBM
FairEMOL
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Adult
0.53
0.97
0.01
0.93
0.66
0.00
0.33
0.95
0.03
1.00
0.00
0.00
0.13
0.84
0.14
0.07
0.95
0.04
0.07
0.86
0.14
0.40
0.60
0.00
Bank
0.73
0.73
0.02
0.67
0.83
0.00
0.87
0.50
0.03
1.00
0.02
0.00
0.33
0.73
0.13
0.93
0.60
0.00
0.93
0.65
0.02
0.39
0.61
0.00
COMPAS
0.53
0.74
0.00
0.93
0.50
0.00
0.73
0.69
0.05
0.87
0.93
0.00
0.40
0.71
0.19
0.87
0.68
0.02
0.73
0.75
0.01
0.17
0.83
0.00
Default
0.80
0.62
0.02
0.80
0.51
0.00
0.73
0.59
0.05
1.00
0.77
0.00
0.60
0.55
0.17
0.67
0.63
0.07
0.73
0.63
0.04
0.32
0.68
0.00
Dutch
0.00
0.96
0.04
0.80
0.82
0.01
0.40
0.98
0.01
1.00
0.75
0.00
0.20
0.94
0.05
0.00
0.96
0.04
0.00
0.94
0.06
0.13
0.87
0.00
Table 7: Dominance relationship between MONBM-CD and other methods in terms of ACC and DP metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that MONBM-CD dominates, does not dominate each other, and is dominated by the corresponding method. Dominance metrics for single-point and population-based baseline methods follow [ 58 ] and [ 47 ] , respectively.
Figure 11: Comparison of the performance of MONBM-CD and baseline methods on model accuracy and fairness metrics on one arbitrary trial.
SNAM
GAMI-Net
NPLAM
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Adult
1.000
0.788
0.000
0.573
0.914
0.316
0.933
0.775
0.000
Bank
0.967
0.541
0.000
0.433
0.793
0.108
1.000
0.412
0.000
COMPAS
0.953
0.659
0.000
0.507
0.779
0.110
0.733
0.657
0.004
Default
0.653
0.831
0.005
0.080
0.374
0.612
0.600
0.861
0.006
Dutch
0.993
0.586
0.000
0.773
0.854
0.007
1.000
0.573
0.000
Table 8: Dominance relationship between MONBM-CL and other methods in terms of ACC and LI metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that MONBM-CL dominates, does not dominate each other, and is dominated by the corresponding method. Dominance metrics for all baseline methods follow [ 58 ] .
Figure 12: Comparison of the performance of MONBM-CL and baseline methods on model accuracy and LI metrics on one arbitrary trial.
Dataset
∣D∣
∣X∣
∣Xdisc.∣
∣Xnum.∣
S
Adult
48,842
14
8
6
Sex
Bank
45,211
16
9
7
Marital
COMAPS
6172
7
3
4
Race
Default
30,000
23
9
14
Sex
Dutch
60,420
11
9
2
Sex
LSAT
20,798
11
5
6
Race
Table S-I: Six datasets used in this paper. “ ∣D∣ ”, “ ∣X∣ ”, “ ∣Xdisc.∣ ”, “ ∣Xnum.∣ ”, and “ S ” denote the number of samples, the number of features, the number of discrete features, the number of numerical features, and the sensitive attribute under consideration, respectively.
Dataset
NBM
MONBM
Adult
1,681.89
19,650.14
Bank
1,531.17
19,273.12
COMPAS
203.473
5,359.28
Default
1,515.16
20,792.25
Dutch
2,087.10
22,009.63
LSAT
863.613
10,367.60
Table S-II: Comparison of average wall-clock time (seconds) between a single NBM training run and the MONBM-CD instantiation across six datasets.
Dataset
Number of epochs
Batch size
Learning rate
Weight decay
Dropout
Basis dropout
Output penalty
Adult
1000
1024
0.02525
0.02234
0.0
0.05
0.00219
Bank
1000
1024
0.02078
0.09175
0.0
0.1
0.00803
COMPAS
1000
1024
0.01612
0.07796
0.2
0.05
0.00045
Default
1000
1024
0.00995
0.07137
0.2
0.1
0.02432
Dutch
1000
1024
0.00037
0.02814
0.2
0.0
0.01620
LSAT
1000
1024
0.01452
0.04858
0.1
0.05
0.02428
Table S-III: The parameter settings of the NBM on six datasets in this paper.
Figure S-1: Pareto front distributions of accuracy (ACC) and smoothness ( Ms ) across different scaling factors η on the Adult dataset in one arbitrary trial.
Table S-IV: The required monotonically increasing and decreasing features on each dataset in this paper.
Adult
Bank
COMPAS
ACC ( % ) ↑
DP ↓
ACC ( % ) ↑
DP ↓
ACC ( % ) ↑
DP ↓
Best CE
86.57
0.1727 (+0.1721)
90.47
0.0360 (+0.0352)
69.03
0.1964 (+0.1955)
“Middle DP”
85.83 (-0.74)
0.0865 (+0.0859)
90.44 (-0.03)
0.0171 (+0.0162)
68.51 (-0.52)
0.0998 (+0.0990)
Best DP
76.60 (-9.97)
0.0005
88.73 (-1.49)
0.0008
53.94 (-15.10)
0.0008
Default
Dutch
LSAT
ACC ( % ) ↑
DP ↓
ACC ( % ) ↑
DP ↓
ACC ( % ) ↑
DP ↓
Table S-V: Comparison of the mean values of the ACC and DP on the training set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CD instantiation over 15 trials. ACC values are expressed as percentages ( % ). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.
Adult
Bank
COMPAS
ACC ( % ) ↑
LI ↓
ACC ( % ) ↑
LI ↓
ACC ( % ) ↑
LI ↓
Best CE
86.60
1.0300 (+0.9205)
90.28
1.2031 (+1.1246)
67.41
0.7382 (+0.4508)
“Middle LI”
84.89 (-1.71)
0.5724 (+0.4629)
89.85 (-0.43)
0.6326 (+0.5541)
62.03 (-5.38)
0.5129 (+0.2255)
Best LI
65.70 (-20.90)
0.1095
83.26 (-7.02)
0.0785
54.03 (-13.38)
0.2874
Default
Dutch
LSAT
ACC ( % ) ↑
LI ↓
ACC ( % ) ↑
LI ↓
ACC ( % ) ↑
LI ↓
Table S-VI: Comparison of the mean values of the ACC and LI on the training set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CL instantiation over 15 trials. ACC values are expressed as percentages ( % ). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.
Adult
Bank
COMPAS
ACC ( % ) ↑
GI ↓
ACC ( % ) ↑
GI ↓
ACC ( % ) ↑
GI ↓
Best CE
85.94
0.1713 (+0.1411)
89.52 (-0.55)
0.2287 (+0.1864)
68.79 (-0.02)
0.0129 (+0.0019)
“Middle GI”
85.89 (-0.04)
0.0915 (+0.0612)
90.07
0.0715 (+0.0291)
68.82
0.0120 (+0.0009)
Best GI
80.62 (-5.31)
0.0303
89.70 (-0.37)
0.0423
68.42 (-0.40)
0.0111
Default
Dutch
LSAT
ACC ( % ) ↑
GI ↓
ACC ( % ) ↑
GI ↓
ACC ( % ) ↑
GI ↓
Table S-VII: Comparison of the mean values of the ACC and GI on the training set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CG instantiation over 15 trials. ACC values are expressed as percentages ( % ). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.
Adult
Bank
ACC ( % ) ↑
DP ↓
LI ↓
GI ↓
ACC ( % ) ↑
DP ↓
LI ↓
GI ↓
Majority Baseline
76.1
—
—
—
88.3
—
—
—
Best CE
85.9
0.178 (+0.18)
1.026 (+0.92)
0.159 (+0.12)
89.5
0.041 (+0.04)
1.160 (+1.07)
0.226 (+0.19)
Best DP
69.1 (-16.8)
0.000
0.130 (+0.02)
0.145 (+0.11)
88.3 (-1.20)
0.000
0.525 (+0.43)
0.111 (+0.08)
Best LI
65.7 (-20.2)
0.000 (+0.00)
0.108
0.163 (+0.13)
83.2 (-6.30)
0.000 (+0.00)
0.095
0.295 (+0.26)
Best GI
69.7 (-16.2)
0.009 (+0.01)
0.464 (+0.36)
0.036
88.3 (-1.20)
0.001 (+0.00)
0.751 (+0.66)
0.032
Table S-VIII: Comparison of the mean values of the ACC , DP , LI , and GI on the training set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLG instantiation over 15 trials. ACC values are expressed as percentages ( % ). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses. The “Majority Baseline” row represents the accuracy of a constant classifier that consistently predicts the majority class for each dataset.
Adult
Bank
COMPAS
ACC ( % ) ↑
EOD ↓
ACC ( % ) ↑
EOD ↓
ACC ( % ) ↑
EOD ↓
Best CE
86.37
0.0485 (+0.0482)
90.32
0.0181 (+0.0178)
68.60
0.1280 (+0.1280)
“Middle EOD”
85.38 (-0.99)
0.0236 (+0.0233)
90.23 (-0.09)
0.0094 (+0.0091)
68.19 (-0.41)
0.0643 (+0.0643)
Best EOD
76.07 (-10.3)
0.0003
88.62 (-1.70)
0.0003
52.94 (-15.66)
0.0000
Default
Dutch
LSAT
ACC ( % ) ↑
EOD ↓
ACC ( % ) ↑
EOD ↓
ACC ( % ) ↑
EOD ↓
Table S-IX: Comparison of the mean values of the ACC and EOD on the test set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CE instantiation over 15 trials. ACC values are expressed as percentages ( % ). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.
Figure S-2: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, the blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle EOD ”, and best EOD points obtained by the MONBM-CE instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The sensitive attribute is highlighted in red. The Pearson correlation coefficient between each attribute and the sensitive attribute is annotated in the respective subplots.
Figure S-3: Comparison of global explanations of continuous features to validate the proposed smoothness and monotonicity metrics. For each dataset, the blue, red, and green lines represent the global explanation results of the continuous features by the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. Each continuous feature has a smoothness requirement, and features highlighted in red also have a monotonicity requirement.
Figure S-4: Variation of global explanations for continuous features among non-dominated knee-point solutions. In each subfigure, the collection of curves represents the shape functions of continuous features for the set of non-dominated knee-point solutions obtained from the final generation of the MONBM-CG instantiation in one arbitrary trial. The color gradient, ranging from deep purple (High ACC , prioritizing predictive accuracy) to bright yellow (High GI , emphasizing smoothness and monotonicity), illustrates the gradual transition in the continuous shape functions across the trade-off surface.
Figure S-5: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, The blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle DP ”, and best DP points obtained by the MONBM-CD instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The sensitive attribute is highlighted in red. The Pearson correlation coefficient between each attribute and the sensitive attribute is annotated in the respective subplots.
Figure S-6: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, the blue, red, and green lines and their corresponding shaded areas represent the mean global explanation results and standard deviations of the models corresponding to the best CE , “middle DP ”, and best DP points obtained by the MONBM-CD instantiation averaged over 15 independent trials, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The sensitive attribute is highlighted in red. The Pearson correlation coefficient between each attribute and the sensitive attribute is annotated in the respective subplots.
Figure S-7: Comparison of the average local explanations to assess the impact of improved LI on each feature. For each dataset, the three columns respectively display the average local explanation results of the model corresponding to: (1) the best CE point, (2) the “middle LI ” or “middle DP ” point, and (3) the best LI or best DP point, obtained by the MONBM-CL or MONBM-CD instantiation in one arbitrary trial. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.
Figure S-8: Comparison of the average local explanations to assess the impact of improved LI on each feature. For each dataset, the three columns respectively display the mean and standard deviation of the average local explanation results of the model corresponding to: (1) the best CE point, (2) the “middle LI ” or “middle DP ” point, and (3) the best LI or best DP point, obtained by the MONBM-CL or MONBM-CG instantiation averaged over 15 independent trials. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set, with error bars representing the standard deviation across the 15 trials.
Figure S-9: Comparison of global explanations to assess the impact of improved GI on each feature. For each dataset, the blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The continuous features are highlighted in red.
Figure S-10: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, the blue, red, and green lines and their corresponding shaded areas represent the mean global explanation results and standard deviations of the models corresponding to the best CE , “middle GI ”, and best GI points obtained by the MONBM-CG instantiation averaged over 15 independent trials, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The continuous features are highlighted in red.
Figure S-11: Comparison of the average local explanations to assess the impact of improved GI on each feature. For each dataset, the three columns represent the average local explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation in one arbitrary trial, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.
Figure S-12: Comparison of the average local explanations to assess the impact of improved GI on each feature. For each dataset, the three columns represent the mean and standard deviation of the average local explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation averaged over 15 independent trials, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set, with error bars representing the standard deviation across the 15 trials.
Figure S-13: Comparison of the average local explanations to explore the interrelationships among the four objectives. For each dataset, the four columns represent the average local explanation results of the model corresponding to the best CE , best DP , best LI , and best GI points obtained by the MONBM-CDLG instantiation in one arbitrary trial, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.
Figure S-14: Comparison of the average local explanations to explore the interrelationships among the four objectives. For each dataset, the four columns represent the mean and standard deviation of the average local explanation results of the model corresponding to the best CE , best DP , best LI , and best GI points obtained by the MONBM-CDLG instantiation averaged over 15 independent trials, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set, with error bars representing the standard deviation across the 15 trials.
Figure S-15: Distribution of all MONBM-CDLG non-dominated solutions across pairs of trustworthy objectives to examine their interrelationships. Each subplot visualizes the solutions obtained from one arbitrary run of the MONBM-CDLG instantiation on a given dataset, showing pairwise relationships between DP , LI , and GI .
Figure S-16: Performance of the non-dominant knee point solutions obtained by MONBM-CG in terms of model accuracy and GI metrics on one arbitrary trial.
Figure S-17: Performance of the non-dominated knee point solutions obtained by MONBM-CDLG in terms of model accuracy, DP , LI , and GI metrics on one arbitrary trial.
Adult
Dutch
ACC ( % ) ↑
DP ↓
LI ↓
Ms↓
ACC ( % ) ↑
DP ↓
LI ↓
Ms↓
Best CE
85.8
0.178
1.029
0.384
82.4
0.328
0.700
0.388
Best DP
76.1
0.000
0.129
0.293
52.4
0.000
0.969
0.131
Best LI
76.1
0.000
0.104
0.330
66.9
0.930
0.111
0.814
Best GI
77.2
0.044
0.673
0.068
75.5
0.576
0.667
0.045
Table S-X: Comparison of the mean values of the ACC , DP , LI , and Ms on the test set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLS instantiation on the Adult and Dutch datasets over 15 trials. ACC values are expressed as percentages ( % ).
Adult
Dutch
ACC ( % ) ↑
DP ↓
LI ↓
Mm↓
ACC ( % ) ↑
DP ↓
LI ↓
Mm↓
Best CE
85.9
0.182
1.024
0.067
82.7
0.336
0.719
0.009
Best DP
76.1
0.000
0.229
0.049
52.4
0.000
0.861
0.006
Best LI
76.1
0.000
0.115
0.076
65.8
0.972
0.099
0.048
Best GI
76.1
0.022
0.498
0.003
71.7
0.054
0.781
0.000
Table S-XI: Comparison of the mean values of the ACC , DP , LI , and Mm on the test set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLM instantiation on the Adult and Dutch datasets over 15 trials. ACC values are expressed as percentages ( % ).
MONBM w/o AS
MONBM with AS
ACC ( % ) ↑
DP ↓
LI ↓
GI ↓
ACC ( % ) ↑
DP ↓
LI ↓
GI ↓
Best CE
85.7
0.174
1.026
0.172
85.9
0.174
1.027
0.170
Best DP
65.6
0.000
0.124
0.155
76.1
0.000
0.097
0.151
Best LI
65.7
0.000
0.108
0.167
76.1
0.000
0.097
0.151
Best GI
69.7
0.008
0.463
0.036
76.2
0.002
0.414
0.028
Table S-XII: Comparison of the mean values of the ACC , DP , LI , and GI on the test set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLG instantiation with and without the adaptive strategy ( MONBM with AS vs. MONBM w/o AS) on the Adult dataset over 15 trials. ACC values are expressed as percentages ( % ). For each metric, the better average between MONBM with AS and MONBM w/o AS is highlighted in bold.
Fair-NBM
LFR-NBM
Re-NBM
ROC-NBM
LPP-NBM
Oracle-NBM
FRAPPÉ-NBM
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Adult
0.13
0.99
0.00
0.00
1.00
0.00
0.27
0.99
0.00
0.73
0.85
0.00
0.13
0.99
0.00
0.00
1.00
0.00
0.13
0.99
0.00
Bank
0.07
0.99
0.00
0.07
0.98
0.00
0.13
0.96
0.00
1.00
0.48
0.00
0.27
0.49
0.00
0.27
0.77
0.00
0.73
0.63
0.00
COMPAS
0.00
1.00
0.00
0.33
0.87
0.00
0.20
0.98
0.01
0.00
1.00
0.00
0.00
1.00
0.00
0.00
1.00
0.00
0.07
0.99
0.00
Default
0.47
0.95
0.00
0.53
0.89
0.00
0.13
0.97
0.01
0.47
0.97
0.00
0.53
0.92
0.00
0.47
0.94
0.00
0.43
0.82
0.00
Dutch
0.00
0.99
0.01
0.73
0.87
0.00
0.00
1.00
0.00
0.00
1.00
0.00
0.00
1.00
0.00
0.00
1.00
0.00
0.00
1.00
0.00
Table S-XIII: Dominance relationship between MONBM-CD and other methods in terms of ACC , DP , LI , and GI metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that MONBM-CD dominates, does not dominate each other, and is dominated by the corresponding method. Dominance metrics for all baseline methods follow [ 58 ] .
SNAM
GAMI-Net
NPLAM
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Adult
0.000
1.000
0.000
0.067
0.999
0.000
0.000
1.000
0.000
Bank
0.000
1.000
0.000
0.067
0.999
0.003
0.000
1.000
0.000
COMPAS
0.000
1.000
0.000
0.067
0.999
0.000
0.015
0.985
0.000
Default
0.000
1.000
0.000
0.000
1.000
0.000
0.000
1.000
0.000
Dutch
0.000
1.000
0.000
0.000
1.000
0.000
0.002
0.998
0.000
Table S-XIV: Dominance relationship between MONBM-CL and other methods in terms of ACC , DP , LI , and GI metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that MONBM-CL dominates, does not dominate each other, and is dominated by the corresponding method. Dominance metrics for all baseline methods follow [ 58 ] .
Fair-NBM
LFR-NBM
Re-NBM
ROC-NBM
LPP-NBM
Dataset
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Do.
Non-Do.
Be-Do.
Adult
0.000
1.000
0.000
1.000
0.607
0.000
0.000
0.100
0.000
1.000
0.135
0.000
0.000
0.000
0.000
Bank
0.000
1.000
0.000
0.600
0.680
0.000
0.000
1.000
0.000
0.867
0.457
0.000
0.000
1.000
0.000
COMPAS
0.067
0.997
0.000
0.867
0.626
0.000
0.000
1.000
0.000
0.200
0.983
0.000
0.133
0.987
0.000
Default
0.000
1.000
0.000
0.533
0.929
0.000
0.000
1.000
0.000
0.133
0.988
0.000
0.000
1.000
0.000
Dutch
0.000
1.000
0.000
0.800
0.821
0.000
0.000
1.000
0.000
0.067
0.999
0.000
0.000
1.000
0.000
Table S-XV: Dominance relationship between MONBM-CDLG and 10 baseline methods in terms of ACC , DP , LI , and GI metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that our method dominates, does not dominate each other, and is dominated by the corresponding method.
Neural network interpretability studies how models represent information and make predictions, and how to explain, test, or control their internal computations.
Mechanistic interpretability (MI) is an emerging framework for interpreting neural networks. Given a task and model, MI aims to discover a succinct algorithmic process, an interpretation, that explains the model's decision process on that task. However, MI is difficult to scale and generalize. This stems in part from two key challenges: there is no precise notion of a valid interpretation; and, generating interpretations is often an ad hoc process. In this paper, we address these challenges by defining and studying the problem of interpretive equivalence: determining whether two different models share a common interpretation, without requiring an explicit description of what that interpretation is. At the core of our approach, we propose and formalize the principle that two interpretations of a model are equivalent if all of their possible implementations are also equivalent. We develop an algorithm to estimate interpretive equivalence and case study its use on Transformer-based models. To analyze our algorithm, we introduce necessary and sufficient conditions for interpretive equivalence based on models' representation similarity. We provide guarantees that simultaneously relate a model's algorithmic interpretations, circuits, and representations. Our framework lays a foundation for the development of more rigorous evaluation methods of MI and automated, generalizable interpretation discovery methods.
Alan Sun, Mariya Toneva
Carnegie Mellon University · Max Planck Institute for Software Systems
Deep neural networks (DNNs) exhibit attractive performance in various fields but often suffer from low interpretability. The neural additive model (NAM) and its variant called the neural basis model (NBM) use neural networks (NNs) as nonlinear shape functions in generalized additive models (GAMs). Both models are highly interpretable and exhibit good performance and flexibility for NN training. NAM and NBM can provide and visualize the contribution of each feature to the prediction owing to GAM-based architectures. However, when using two-input NNs to consider feature interactions or when applying them to high-dimensional datasets, training NAM and NBM becomes intractable due to the increase in the computational resources required. This paper proposes incorporating the feature selection mechanism into NAM and NBM to resolve computational bottlenecks. We introduce the feature selection layer in both models and update the selection weights during training. Our method is simple and can reduce computational costs and model sizes compared to vanilla NAM and NBM. In addition, it enables us to use two-input NNs even in high-dimensional datasets and capture feature interactions. We demonstrate that the proposed models are computationally efficient compared to vanilla NAM and NBM, and they exhibit better or comparable performance with state-of-the-art GAMs.
Yasutoshi Kishimoto, Kota Yamanishi, Takuya Matsuda +1
The deployment of Artificial Intelligence in high-risk domains, such as finance and healthcare, necessitates models that are both fair and transparent. While regulatory frameworks, including the EU's AI Act, mandate bias mitigation, they are deliberately vague about the definition of bias. In line with existing research, we argue that true fairness requires addressing bias at the intersections of protected groups. We propose a unified framework that leverages Mixed-Integer Optimization (MIO) to train intersectionally fair and intrinsically interpretable classifiers. We prove the equivalence of two measures of intersectional fairness (MSD and SPSF) in detecting the most unfair subgroup and empirically demonstrate that our MIO-based algorithm improves performance in finding bias. We train high-performing, interpretable classifiers that bound intersectional bias below an acceptable threshold, offering a robust solution for regulated industries and beyond.