Interpretable and Fair Generalized Additive Neural Networks via Multi-objective Learning

Ziming Wang; Changwu Huang; Ke Tang; Yew-Soon Ong; Xin Yao

Figure 1: Examples of explanations provided by GAM. For the global explanation, as shown in Fig. 1(a) , the x-axis represents the input feature values

x_{i}

, while the y-axis corresponds to the values of the shape function

f_{i}(x_{i})

. For the local explanation, as shown in Fig. 1(b) , the y-axis represents the features

x_{i}

, and the x-axis represents the shape function value

f_{i}(x_{i}^{(j)})

for the corresponding feature

x_{i}^{(j)}

of data point

\textnormal{{x}}^{(j)}

.

Figure 2: NBM architecture for the binary classification task, the figure taken from [ 42 ] with slight modifications.

Method	Category	Fairness	LI	GI	GI
Method	Category	Fairness	(Complexity)	(Smoothness)	(Monotonicity)
Spline-GAM [ 17 ]	Spline	✘	✘	✔	✘
EBM [ 40 ]	Tree	✘	✘	✘	✘
NAM [ 1 ]	NN	✘	✘	✘	✘
NBM [ 42 ]	NN	✘	✘	✘	✘
SNAM [ 52 ]	NN	✘	✔	✘	✘

Table 1: Comparison of the proposed MONBM framework with existing GAM-based approaches.

Figure 3: The computational pipeline of the MONBM instantiation.

Figure 4: Comparison of global explanations of continuous features to validate the proposed smoothness and monotonicity metrics. For each dataset, the blue, red, and green lines represent the global explanation results of the continuous features by the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. Each continuous feature has a smoothness requirement, and features highlighted in red also have a monotonicity requirement.

	Adult		Bank		COMPAS
	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$
Best CE	86.36	0.1686 (+0.1685)	90.26	0.0357 (+0.0357)	68.13	0.2025 (+0.2023)
“Middle DP”	85.61 (-0.75)	0.0851 (+0.0850)	90.21 (-0.05)	0.0176 (+0.0176)	67.77 (-0.35)	0.1041 (+0.1039)
Best DP	76.60 (-9.76)	0.0001	88.76 (-1.49)	0.0000	53.86 (-14.27)	0.0002
	Default		Dutch		LSAT
	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$

Table 2: Comparison of the mean values of the ACC and DP on the test set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CD instantiation over 15 trials. ACC values are expressed as percentages (

\%

). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.

Figure 5: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, the blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle DP ”, and best DP points obtained by the MONBM-CD instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The sensitive attribute is highlighted in red. The Pearson correlation coefficient between each attribute and the sensitive attribute is annotated in the respective subplots.

	Adult		Bank		COMPAS
	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$
Best CE	86.36	1.0271 (+0.9171)	90.25	1.1649 (+1.0811)	68.29	0.7289 (+0.4403)
“Middle LI”	84.81 (-1.55)	0.5720 (+0.4619)	89.75 (-0.50)	0.6292 (+0.5454)	61.51 (-6.78)	0.5102 (+0.2216)
Best LI	65.55 (-20.81)	0.1100	83.23 (-7.03)	0.0838	53.53 (-14.76)	0.2886
	Default		Dutch		LSAT
	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$

Table 3: Comparison of the mean values of the ACC and LI on the test set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CL instantiation over 15 trials. ACC values are expressed as percentages (

\%

). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.

Figure 6: Comparison of the average local explanations of the test set to assess the impact of improved LI on each feature. The three columns respectively display the average local explanation results of the model corresponding to: (1) the best CE point, (2) the “middle LI ” or “middle DP ” point, and (3) the best LI or best DP point, obtained by the MONBM-CL or MONBM-CD instantiation on the Bank dataset in one arbitrary trial. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.

	Adult		Bank		COMPAS
	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$
Best CE	85.70	0.1651 (+0.1338)	89.47 (-0.44)	0.2212 (+0.1779)	68.51 (-0.08)	0.0129 (+0.0019)
“Middle GI”	85.57 (-0.13)	0.1105 (+0.0792)	89.91	0.0791 (+0.0358)	68.59	0.0120 (+0.0009)
Best GI	81.01 (-4.69)	0.0313	89.65 (-0.26)	0.0433	68.19 (-0.40)	0.0111
	Default		Dutch		LSAT
	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$

Table 4: Comparison of the mean values of the ACC and GI on the test set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CG instantiation over 15 trials. ACC values are expressed as percentages (

\%

). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.

Figure 7: Comparison of global explanations to assess the impact of improved GI on each feature. The blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation on the Bank dataset in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The continuous features are highlighted in red.

Figure 8: Comparison of the average local explanations of the test set to assess the impact of improved GI on each feature. The three columns represent the average local explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation on the Bank dataset in one arbitrary trial, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.

	Adult				Bank
	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	GI $\downarrow$
Majority Baseline	76.1	—	—	—	88.3	—	—	—
Best CE	85.7	0.174 (+0.17)	1.026 (+0.92)	0.172 (+0.14)	89.5	0.040 (+0.04)	1.160 (+1.07)	0.229 (+0.20)
Best DP	65.6 (-20.1)	0.000	0.124 (+0.02)	0.155 (+0.12)	88.3 (-1.20)	0.000	0.525 (+0.43)	0.107 (+0.08)
Best LI	65.7 (-20.0)	0.000 (+0.00)	0.108	0.167 (+0.13)	83.2 (-6.30)	0.000 (+0.00)	0.095	0.301 (+0.27)
Best GI	69.7 (-16.0)	0.008 (+0.01)	0.463 (+0.36)	0.036	88.3 (-1.20)	0.001 (+0.00)	0.751 (+0.66)	0.032

Table 5: Comparison of the mean values of the ACC , DP , LI , and GI on the test set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLG instantiation over 15 trials. ACC values are expressed as percentages (

\%

). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses. The “Majority Baseline” row represents the accuracy of a constant classifier that consistently predicts the majority class for each dataset.

Figure 9: Comparison of the average local explanations of the test set to explore the interrelationships among the four objectives. For each dataset, the four columns represent the average local explanation results of the model corresponding to the best CE , best DP , best LI , and best GI points obtained by the MONBM-CDLG instantiation in one arbitrary trial, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.

Figure 10: Distribution of all MONBM-CDLG non-dominated solutions across pairs of trustworthy objectives to examine their interrelationships. Each subplot visualizes the solutions obtained from one arbitrary run of the MONBM-CDLG instantiation on a given dataset, showing pairwise relationships between DP , LI , and GI .

	HV $\uparrow$		MS $\uparrow$		PD $\uparrow$
	MONBM	FairEMOL	MONBM	FairEMOL	MONBM	FairEMOL
Adult	0.8849 (0.0183)	0.8678 (0.0203) -	0.2597 (0.0164)	0.2302 (0.0096) -	101910 (24450)	230.48 (27.923) -
Bank	0.9398 (0.0087)	0.8678 (0.0103) -	0.0866 (0.0058)	0.0568 (0.0029) -	26207 (7208.7)	46.04 (9.5900) -
COMPAS	0.6156 (0.0402)	0.6065 (0.0411) -	0.2458 (0.0443)	0.2938 (0.0443) +	75825 (21868)	191.89 (49.975) -
Default	0.8381 (0.0266)	0.8321 (0.0272) -	0.0457 (0.0067)	0.0465 (0.0128) $\approx$	19365 (4974.1)	30.611 (7.9945) -
Dutch	0.9092 (0.0136)	0.8951 (0.0160) -	0.8504 (0.3369)	0.4417 (0.0097) -	266770 (62707)	334.09 (77.768) -

Table 6: The average of 15 trials of HV , MS , and PD values for the final model set. “+/

\approx

/-” indicates that the average FairEMOL metric value is better/approximate/worse than the corresponding MONBM value on the Wilcoxon rank-sum test, with a significance level of 0.05. The best average value is marked in gray.

	Fair-NBM			LFR-NBM			Re-NBM			ROC-NBM			LPP-NBM			Oracle-NBM			FRAPPÉ-NBM			FairEMOL
	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.
Adult	0.53	0.97	0.01	0.93	0.66	0.00	0.33	0.95	0.03	1.00	0.00	0.00	0.13	0.84	0.14	0.07	0.95	0.04	0.07	0.86	0.14	0.40	0.60	0.00
Bank	0.73	0.73	0.02	0.67	0.83	0.00	0.87	0.50	0.03	1.00	0.02	0.00	0.33	0.73	0.13	0.93	0.60	0.00	0.93	0.65	0.02	0.39	0.61	0.00
COMPAS	0.53	0.74	0.00	0.93	0.50	0.00	0.73	0.69	0.05	0.87	0.93	0.00	0.40	0.71	0.19	0.87	0.68	0.02	0.73	0.75	0.01	0.17	0.83	0.00
Default	0.80	0.62	0.02	0.80	0.51	0.00	0.73	0.59	0.05	1.00	0.77	0.00	0.60	0.55	0.17	0.67	0.63	0.07	0.73	0.63	0.04	0.32	0.68	0.00
Dutch	0.00	0.96	0.04	0.80	0.82	0.01	0.40	0.98	0.01	1.00	0.75	0.00	0.20	0.94	0.05	0.00	0.96	0.04	0.00	0.94	0.06	0.13	0.87	0.00

Table 7: Dominance relationship between MONBM-CD and other methods in terms of ACC and DP metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that MONBM-CD dominates, does not dominate each other, and is dominated by the corresponding method. Dominance metrics for single-point and population-based baseline methods follow [ 58 ] and [ 47 ] , respectively.

Figure 11: Comparison of the performance of MONBM-CD and baseline methods on model accuracy and fairness metrics on one arbitrary trial.

	SNAM			GAMI-Net			NPLAM
	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.
Adult	1.000	0.788	0.000	0.573	0.914	0.316	0.933	0.775	0.000
Bank	0.967	0.541	0.000	0.433	0.793	0.108	1.000	0.412	0.000
COMPAS	0.953	0.659	0.000	0.507	0.779	0.110	0.733	0.657	0.004
Default	0.653	0.831	0.005	0.080	0.374	0.612	0.600	0.861	0.006
Dutch	0.993	0.586	0.000	0.773	0.854	0.007	1.000	0.573	0.000

Table 8: Dominance relationship between MONBM-CL and other methods in terms of ACC and LI metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that MONBM-CL dominates, does not dominate each other, and is dominated by the corresponding method. Dominance metrics for all baseline methods follow [ 58 ] .

Figure 12: Comparison of the performance of MONBM-CL and baseline methods on model accuracy and LI metrics on one arbitrary trial.

Dataset	$\|\bm{\mathcal{D}}\|$	$\|\textit{{X}}\|$	$\|\textit{{X}}_{disc.}\|$	$\|\textit{{X}}_{num.}\|$	$S$
Adult	$48,842$	14	8	6	Sex
Bank	$45,211$	16	9	7	Marital
COMAPS	$6172$	7	3	4	Race
Default	$30,000$	23	9	14	Sex
Dutch	$60,420$	11	9	2	Sex
LSAT	$20,798$	11	5	6	Race

Table S-I: Six datasets used in this paper. “

|\bm{\mathcal{D}}|

”, “

|\textit{{X}}|

”, “

|\textit{{X}}_{disc.}|

”, “

|\textit{{X}}_{num.}|

”, and “

S

” denote the number of samples, the number of features, the number of discrete features, the number of numerical features, and the sensitive attribute under consideration, respectively.

Dataset	NBM	MONBM
Adult	1,681.89	19,650.14
Bank	1,531.17	19,273.12
COMPAS	203.473	5,359.28
Default	1,515.16	20,792.25
Dutch	2,087.10	22,009.63
LSAT	863.613	10,367.60

Table S-II: Comparison of average wall-clock time (seconds) between a single NBM training run and the MONBM-CD instantiation across six datasets.

Dataset	Number of epochs	Batch size	Learning rate	Weight decay	Dropout	Basis dropout	Output penalty
Adult	1000	1024	0.02525	0.02234	0.0	0.05	0.00219
Bank	1000	1024	0.02078	0.09175	0.0	0.1	0.00803
COMPAS	1000	1024	0.01612	0.07796	0.2	0.05	0.00045
Default	1000	1024	0.00995	0.07137	0.2	0.1	0.02432
Dutch	1000	1024	0.00037	0.02814	0.2	0.0	0.01620
LSAT	1000	1024	0.01452	0.04858	0.1	0.05	0.02428

Table S-III: The parameter settings of the NBM on six datasets in this paper.

Figure S-1: Pareto front distributions of accuracy (ACC) and smoothness (

\mathcal{M}_{s}

) across different scaling factors

\eta

on the Adult dataset in one arbitrary trial.

Dataset	Monotonically Increasing Features	Monotonically Dncreasing Features
Adult	“capital_gain”, “education_num”, “hours_per_week”	“capital_loss”
Bank	“balance”, “duration”, “previous”	——
COMPAS	“priors_count”	——
Default	——	——
Dutch	“edu_level”	——
LSAT	“decile1b”, “decile3”, “lsat”, “ugpa”, “zfygpa”, “zgpa”	——

Table S-IV: The required monotonically increasing and decreasing features on each dataset in this paper.

	Adult		Bank		COMPAS
	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$
Best CE	86.57	0.1727 (+0.1721)	90.47	0.0360 (+0.0352)	69.03	0.1964 (+0.1955)
“Middle DP”	85.83 (-0.74)	0.0865 (+0.0859)	90.44 (-0.03)	0.0171 (+0.0162)	68.51 (-0.52)	0.0998 (+0.0990)
Best DP	76.60 (-9.97)	0.0005	88.73 (-1.49)	0.0008	53.94 (-15.10)	0.0008
	Default		Dutch		LSAT
	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$

Table S-V: Comparison of the mean values of the ACC and DP on the training set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CD instantiation over 15 trials. ACC values are expressed as percentages (

\%

). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.

	Adult		Bank		COMPAS
	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$
Best CE	86.60	1.0300 (+0.9205)	90.28	1.2031 (+1.1246)	67.41	0.7382 (+0.4508)
“Middle LI”	84.89 (-1.71)	0.5724 (+0.4629)	89.85 (-0.43)	0.6326 (+0.5541)	62.03 (-5.38)	0.5129 (+0.2255)
Best LI	65.70 (-20.90)	0.1095	83.26 (-7.02)	0.0785	54.03 (-13.38)	0.2874
	Default		Dutch		LSAT
	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$	ACC ( $\%$ ) $\uparrow$	LI $\downarrow$

Table S-VI: Comparison of the mean values of the ACC and LI on the training set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CL instantiation over 15 trials. ACC values are expressed as percentages (

\%

). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.

	Adult		Bank		COMPAS
	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$
Best CE	85.94	0.1713 (+0.1411)	89.52 (-0.55)	0.2287 (+0.1864)	68.79 (-0.02)	0.0129 (+0.0019)
“Middle GI”	85.89 (-0.04)	0.0915 (+0.0612)	90.07	0.0715 (+0.0291)	68.82	0.0120 (+0.0009)
Best GI	80.62 (-5.31)	0.0303	89.70 (-0.37)	0.0423	68.42 (-0.40)	0.0111
	Default		Dutch		LSAT
	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	GI $\downarrow$

Table S-VII: Comparison of the mean values of the ACC and GI on the training set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CG instantiation over 15 trials. ACC values are expressed as percentages (

\%

). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.

	Adult				Bank
	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	GI $\downarrow$
Majority Baseline	76.1	—	—	—	88.3	—	—	—
Best CE	85.9	0.178 (+0.18)	1.026 (+0.92)	0.159 (+0.12)	89.5	0.041 (+0.04)	1.160 (+1.07)	0.226 (+0.19)
Best DP	69.1 (-16.8)	0.000	0.130 (+0.02)	0.145 (+0.11)	88.3 (-1.20)	0.000	0.525 (+0.43)	0.111 (+0.08)
Best LI	65.7 (-20.2)	0.000 (+0.00)	0.108	0.163 (+0.13)	83.2 (-6.30)	0.000 (+0.00)	0.095	0.295 (+0.26)
Best GI	69.7 (-16.2)	0.009 (+0.01)	0.464 (+0.36)	0.036	88.3 (-1.20)	0.001 (+0.00)	0.751 (+0.66)	0.032

Table S-VIII: Comparison of the mean values of the ACC , DP , LI , and GI on the training set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLG instantiation over 15 trials. ACC values are expressed as percentages (

\%

). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses. The “Majority Baseline” row represents the accuracy of a constant classifier that consistently predicts the majority class for each dataset.

	Adult		Bank		COMPAS
	ACC ( $\%$ ) $\uparrow$	EOD $\downarrow$	ACC ( $\%$ ) $\uparrow$	EOD $\downarrow$	ACC ( $\%$ ) $\uparrow$	EOD $\downarrow$
Best CE	86.37	0.0485 (+0.0482)	90.32	0.0181 (+0.0178)	68.60	0.1280 (+0.1280)
“Middle EOD”	85.38 (-0.99)	0.0236 (+0.0233)	90.23 (-0.09)	0.0094 (+0.0091)	68.19 (-0.41)	0.0643 (+0.0643)
Best EOD	76.07 (-10.3)	0.0003	88.62 (-1.70)	0.0003	52.94 (-15.66)	0.0000
	Default		Dutch		LSAT
	ACC ( $\%$ ) $\uparrow$	EOD $\downarrow$	ACC ( $\%$ ) $\uparrow$	EOD $\downarrow$	ACC ( $\%$ ) $\uparrow$	EOD $\downarrow$

Table S-IX: Comparison of the mean values of the ACC and EOD on the test set of the models corresponding to the three points chosen from the final solution set obtained by the MONBM-CE instantiation over 15 trials. ACC values are expressed as percentages (

\%

). The best average is highlighted with a gray background. The difference between it and the best value is shown in parentheses.

Figure S-2: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, the blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle EOD ”, and best EOD points obtained by the MONBM-CE instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The sensitive attribute is highlighted in red. The Pearson correlation coefficient between each attribute and the sensitive attribute is annotated in the respective subplots.

Figure S-3: Comparison of global explanations of continuous features to validate the proposed smoothness and monotonicity metrics. For each dataset, the blue, red, and green lines represent the global explanation results of the continuous features by the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. Each continuous feature has a smoothness requirement, and features highlighted in red also have a monotonicity requirement.

Figure S-4: Variation of global explanations for continuous features among non-dominated knee-point solutions. In each subfigure, the collection of curves represents the shape functions of continuous features for the set of non-dominated knee-point solutions obtained from the final generation of the MONBM-CG instantiation in one arbitrary trial. The color gradient, ranging from deep purple (High ACC , prioritizing predictive accuracy) to bright yellow (High GI , emphasizing smoothness and monotonicity), illustrates the gradual transition in the continuous shape functions across the trade-off surface.

Figure S-5: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, The blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle DP ”, and best DP points obtained by the MONBM-CD instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The sensitive attribute is highlighted in red. The Pearson correlation coefficient between each attribute and the sensitive attribute is annotated in the respective subplots.

Figure S-6: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, the blue, red, and green lines and their corresponding shaded areas represent the mean global explanation results and standard deviations of the models corresponding to the best CE , “middle DP ”, and best DP points obtained by the MONBM-CD instantiation averaged over 15 independent trials, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The sensitive attribute is highlighted in red. The Pearson correlation coefficient between each attribute and the sensitive attribute is annotated in the respective subplots.

Figure S-7: Comparison of the average local explanations to assess the impact of improved LI on each feature. For each dataset, the three columns respectively display the average local explanation results of the model corresponding to: (1) the best CE point, (2) the “middle LI ” or “middle DP ” point, and (3) the best LI or best DP point, obtained by the MONBM-CL or MONBM-CD instantiation in one arbitrary trial. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.

Figure S-8: Comparison of the average local explanations to assess the impact of improved LI on each feature. For each dataset, the three columns respectively display the mean and standard deviation of the average local explanation results of the model corresponding to: (1) the best CE point, (2) the “middle LI ” or “middle DP ” point, and (3) the best LI or best DP point, obtained by the MONBM-CL or MONBM-CG instantiation averaged over 15 independent trials. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set, with error bars representing the standard deviation across the 15 trials.

Figure S-9: Comparison of global explanations to assess the impact of improved GI on each feature. For each dataset, the blue, red, and green lines represent the global explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation in one arbitrary trial, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The continuous features are highlighted in red.

Figure S-10: Comparison of global explanations to assess the impact of improved fairness on each feature. For each dataset, the blue, red, and green lines and their corresponding shaded areas represent the mean global explanation results and standard deviations of the models corresponding to the best CE , “middle GI ”, and best GI points obtained by the MONBM-CG instantiation averaged over 15 independent trials, respectively. Each subfigure corresponds to the global explanation of a feature, with the x-axis representing feature values and the y-axis showing the outputs of the shape functions. The continuous features are highlighted in red.

Figure S-11: Comparison of the average local explanations to assess the impact of improved GI on each feature. For each dataset, the three columns represent the average local explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation in one arbitrary trial, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.

Figure S-12: Comparison of the average local explanations to assess the impact of improved GI on each feature. For each dataset, the three columns represent the mean and standard deviation of the average local explanation results of the model corresponding to the best CE , “middle GI ,” and best GI points obtained by the MONBM-CG instantiation averaged over 15 independent trials, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set, with error bars representing the standard deviation across the 15 trials.

Figure S-13: Comparison of the average local explanations to explore the interrelationships among the four objectives. For each dataset, the four columns represent the average local explanation results of the model corresponding to the best CE , best DP , best LI , and best GI points obtained by the MONBM-CDLG instantiation in one arbitrary trial, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set.

Figure S-14: Comparison of the average local explanations to explore the interrelationships among the four objectives. For each dataset, the four columns represent the mean and standard deviation of the average local explanation results of the model corresponding to the best CE , best DP , best LI , and best GI points obtained by the MONBM-CDLG instantiation averaged over 15 independent trials, respectively. For each local explanation, the y-axis represents the features and the x-axis represents the average of the absolute values of the shape function outputs corresponding to the features for all data points in the test set, with error bars representing the standard deviation across the 15 trials.

Figure S-15: Distribution of all MONBM-CDLG non-dominated solutions across pairs of trustworthy objectives to examine their interrelationships. Each subplot visualizes the solutions obtained from one arbitrary run of the MONBM-CDLG instantiation on a given dataset, showing pairwise relationships between DP , LI , and GI .

Figure S-16: Performance of the non-dominant knee point solutions obtained by MONBM-CG in terms of model accuracy and GI metrics on one arbitrary trial.

Figure S-17: Performance of the non-dominated knee point solutions obtained by MONBM-CDLG in terms of model accuracy, DP , LI , and GI metrics on one arbitrary trial.

	Adult				Dutch
	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	$\mathcal{M}_{s}$ $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	$\mathcal{M}_{s}$ $\downarrow$
Best CE	85.8	0.178	1.029	0.384	82.4	0.328	0.700	0.388
Best DP	76.1	0.000	0.129	0.293	52.4	0.000	0.969	0.131
Best LI	76.1	0.000	0.104	0.330	66.9	0.930	0.111	0.814
Best GI	77.2	0.044	0.673	0.068	75.5	0.576	0.667	0.045

Table S-X: Comparison of the mean values of the ACC , DP , LI , and

\mathcal{M}_{s}

on the test set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLS instantiation on the Adult and Dutch datasets over 15 trials. ACC values are expressed as percentages (

\%

).

	Adult				Dutch
	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	$\mathcal{M}_{m}$ $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	$\mathcal{M}_{m}$ $\downarrow$
Best CE	85.9	0.182	1.024	0.067	82.7	0.336	0.719	0.009
Best DP	76.1	0.000	0.229	0.049	52.4	0.000	0.861	0.006
Best LI	76.1	0.000	0.115	0.076	65.8	0.972	0.099	0.048
Best GI	76.1	0.022	0.498	0.003	71.7	0.054	0.781	0.000

Table S-XI: Comparison of the mean values of the ACC , DP , LI , and

\mathcal{M}_{m}

on the test set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLM instantiation on the Adult and Dutch datasets over 15 trials. ACC values are expressed as percentages (

\%

).

	MONBM w/o AS				MONBM with AS
	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	GI $\downarrow$	ACC ( $\%$ ) $\uparrow$	DP $\downarrow$	LI $\downarrow$	GI $\downarrow$
Best CE	85.7	0.174	1.026	0.172	85.9	0.174	1.027	0.170
Best DP	65.6	0.000	0.124	0.155	76.1	0.000	0.097	0.151
Best LI	65.7	0.000	0.108	0.167	76.1	0.000	0.097	0.151
Best GI	69.7	0.008	0.463	0.036	76.2	0.002	0.414	0.028

Table S-XII: Comparison of the mean values of the ACC , DP , LI , and GI on the test set of the models corresponding to the four extreme points chosen from the final solution set obtained by the MONBM-CDLG instantiation with and without the adaptive strategy ( MONBM with AS vs. MONBM w/o AS) on the Adult dataset over 15 trials. ACC values are expressed as percentages (

\%

). For each metric, the better average between MONBM with AS and MONBM w/o AS is highlighted in bold.

	Fair-NBM			LFR-NBM			Re-NBM			ROC-NBM			LPP-NBM			Oracle-NBM			FRAPPÉ-NBM
	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.
Adult	0.13	0.99	0.00	0.00	1.00	0.00	0.27	0.99	0.00	0.73	0.85	0.00	0.13	0.99	0.00	0.00	1.00	0.00	0.13	0.99	0.00
Bank	0.07	0.99	0.00	0.07	0.98	0.00	0.13	0.96	0.00	1.00	0.48	0.00	0.27	0.49	0.00	0.27	0.77	0.00	0.73	0.63	0.00
COMPAS	0.00	1.00	0.00	0.33	0.87	0.00	0.20	0.98	0.01	0.00	1.00	0.00	0.00	1.00	0.00	0.00	1.00	0.00	0.07	0.99	0.00
Default	0.47	0.95	0.00	0.53	0.89	0.00	0.13	0.97	0.01	0.47	0.97	0.00	0.53	0.92	0.00	0.47	0.94	0.00	0.43	0.82	0.00
Dutch	0.00	0.99	0.01	0.73	0.87	0.00	0.00	1.00	0.00	0.00	1.00	0.00	0.00	1.00	0.00	0.00	1.00	0.00	0.00	1.00	0.00

Table S-XIII: Dominance relationship between MONBM-CD and other methods in terms of ACC , DP , LI , and GI metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that MONBM-CD dominates, does not dominate each other, and is dominated by the corresponding method. Dominance metrics for all baseline methods follow [ 58 ] .

	SNAM			GAMI-Net			NPLAM
	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.
Adult	0.000	1.000	0.000	0.067	0.999	0.000	0.000	1.000	0.000
Bank	0.000	1.000	0.000	0.067	0.999	0.003	0.000	1.000	0.000
COMPAS	0.000	1.000	0.000	0.067	0.999	0.000	0.015	0.985	0.000
Default	0.000	1.000	0.000	0.000	1.000	0.000	0.000	1.000	0.000
Dutch	0.000	1.000	0.000	0.000	1.000	0.000	0.002	0.998	0.000

Table S-XIV: Dominance relationship between MONBM-CL and other methods in terms of ACC , DP , LI , and GI metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that MONBM-CL dominates, does not dominate each other, and is dominated by the corresponding method. Dominance metrics for all baseline methods follow [ 58 ] .

	Fair-NBM			LFR-NBM			Re-NBM			ROC-NBM			LPP-NBM
Dataset	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.	Do.	Non-Do.	Be-Do.
Adult	0.000	1.000	0.000	1.000	0.607	0.000	0.000	0.100	0.000	1.000	0.135	0.000	0.000	0.000	0.000
Bank	0.000	1.000	0.000	0.600	0.680	0.000	0.000	1.000	0.000	0.867	0.457	0.000	0.000	1.000	0.000
COMPAS	0.067	0.997	0.000	0.867	0.626	0.000	0.000	1.000	0.000	0.200	0.983	0.000	0.133	0.987	0.000
Default	0.000	1.000	0.000	0.533	0.929	0.000	0.000	1.000	0.000	0.133	0.988	0.000	0.000	1.000	0.000
Dutch	0.000	1.000	0.000	0.800	0.821	0.000	0.000	1.000	0.000	0.067	0.999	0.000	0.000	1.000	0.000

Table S-XV: Dominance relationship between MONBM-CDLG and 10 baseline methods in terms of ACC , DP , LI , and GI metrics. Results are averaged over 15 trials. “Do.”, “Non-Do.”, and “Be-Do.” mean the probability that our method dominates, does not dominate each other, and is dominated by the corresponding method.

Interpretable and Fair Generalized Additive Neural Networks via Multi-objective Learning

Abstract

Figures & tables

Neural Network Interpretability

Tracking Equivalent Mechanistic Interpretations Across Neural Networks

Neural Additive and Basis Models with Feature Selection and Interactions

Intersectional Fairness via Mixed-Integer Optimization

Abstract

Neural Network Interpretability

Explore similar work

Tracking Equivalent Mechanistic Interpretations Across Neural Networks

Neural Additive and Basis Models with Feature Selection and Interactions

Intersectional Fairness via Mixed-Integer Optimization