Interpretability-by-Design with Accurate Locally Additive Models and Conditional Feature Effects

Vasilis Gkolemis; Loukas Kavouras; Dimitrios Kyriakopoulos; Konstantinos Tsopelas; Dimitrios Rontogiannis; Giuseppe Casalicchio; Theodore Dalamagas; Christos Diou

cs.LGFeb 18, 2026

Interpretability-by-Design with Accurate Locally Additive Models and Conditional Feature Effects

Authors: Vasilis Gkolemis, Loukas Kavouras, Dimitrios Kyriakopoulos, Konstantinos Tsopelas, Dimitrios Rontogiannis, Giuseppe Casalicchio, Theodore Dalamagas, Christos Diou

Organizations: ATHENA RC, Greece · HUA, Greece · Max Planck ISS, Germany · LMU, Germany

Abstract

Generalized additive models (GAMs) offer interpretability through independent univariate feature effects but underfit when interactions are present in data. GA $^2$ Ms add selected pairwise interactions which improves accuracy, but sacrifices interpretability and limits model auditing. We propose Conditionally Additive Local Models (CALMs), a new model class, that balances the interpretability of GAMs with the accuracy of GA $^2$ Ms. CALMs allow multiple univariate shape functions per feature, each active in different regions of the input space. These regions are defined independently for each feature as simple logical conditions (thresholds) on the features it interacts with. As a result, effects remain locally additive while varying across subregions to capture interactions. We further propose a principled distillation-based training pipeline that identifies homogeneous regions with limited interactions and fits interpretable shape functions via region-aware backfitting. Experiments on diverse classification and regression tasks show that CALMs consistently outperform GAMs and achieve accuracy comparable with GA $^2$ Ms. Overall, CALMs offer a compelling trade-off between predictive accuracy and interpretability.

Figures & tables

Figure 1: CALMs use conditional feature effects : every feature effect is expressed by a collection of 1D functions, each associated with a different region of the input space. In the example, the effect of

x_{1}

conditions on

x_{3}

, the effect of

x_{2}

x_{1}

, while

x_{d}

does not interact with any other feature and thus has a single plot. CALMs are interpretable-by-design —summarized in

d

figures of 1D plots—and accurate , as they can model feature interactions. Code available at : https://github.com/givasile/CALM .

Figure 2: CALM plot for

x_{1}

. Each curve shows the contribution of

x_{1}

y

(P1) in a different region of the input space (

x_{3}\lessgtr 0

). Vertical lines indicate interaction-induced discontinuities, e.g., at

x_{1}\approx-0.4

there exist a positive hidden jump of

[0,0.37]

due to interaction of

x_{1}

with

x_{2}

, which must be considered when assessing regional sensitivity (P2) or global properties (P3).

Panel of Figure 3 — Figure 3: Explanatory plots for the three synthetic regression tasks: top row (Task 1) shows two-region interaction; middle row (Task 2) shows four-region interaction; bottom row (Task 3) shows general interactions.

Dataset	GAM	GA 2 M	CALM
Case 1	$0.737\text{\scriptsize$ \pm $0.028}$	$0.974\text{\scriptsize$ \pm $0.011}$	$0.995\text{\scriptsize$ \pm $0.002}$
Case 2	$0.479\text{\scriptsize$ \pm $0.052}$	$0.712\text{\scriptsize$ \pm $0.036}$	$0.949\text{\scriptsize$ \pm $0.024}$
Case 3	$0.527\text{\scriptsize$ \pm $0.023}$	$0.961\text{\scriptsize$ \pm $0.009}$	$0.975\text{\scriptsize$ \pm $0.006}$

Table 1: Synthetic examples:

R^{2}

comparison (mean

\pm

std. dev. over 5-fold CV).

Dataset	BB	GAM		CALM	GA 2 M			CALM vs.
	XGB	NAM	EBM		EB 2 M	NODE	GAMI	GAM	GA 2 M
Adult	$0.870\text{\scriptsize$ \pm $0.002}$	$0.851\text{\scriptsize$ \pm $0.002}$	$0.870\text{\scriptsize$ \pm $0.002}$	$0.870\text{\scriptsize$ \pm $0.002}$	$0.871\text{\scriptsize$ \pm $0.003}$	$0.850\text{\scriptsize$ \pm $0.002}$	$0.857\text{\scriptsize$ \pm $0.003}$	✓	✓
COMPAS	$0.661\text{\scriptsize$ \pm $0.007}$	$0.682\text{\scriptsize$ \pm $0.016}$	$0.681\text{\scriptsize$ \pm $0.012}$	$0.684\text{\scriptsize$ \pm $0.013}$	$0.684\text{\scriptsize$ \pm $0.011}$	$0.667\text{\scriptsize$ \pm $0.013}$	$0.686\text{\scriptsize$ \pm $0.013}$	✓	✓
HELOC	$0.717\text{\scriptsize$ \pm $0.013}$	$0.723\text{\scriptsize$ \pm $0.011}$	$0.728\text{\scriptsize$ \pm $0.014}$	$0.728\text{\scriptsize$ \pm $0.012}$	$0.730\text{\scriptsize$ \pm $0.012}$	$0.715\text{\scriptsize$ \pm $0.008}$	$0.725\text{\scriptsize$ \pm $0.012}$	✓	✓
MIMIC2	$0.890\text{\scriptsize$ \pm $0.001}$	$0.886\text{\scriptsize$ \pm $0.001}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.888\text{\scriptsize$ \pm $0.002}$	$0.884\text{\scriptsize$ \pm $0.003}$	✓	✓
Appendicitis	$0.868\text{\scriptsize$ \pm $0.069}$	$0.848\text{\scriptsize$ \pm $0.056}$	$0.877\text{\scriptsize$ \pm $0.077}$	$0.878\text{\scriptsize$ \pm $0.063}$	$0.869\text{\scriptsize$ \pm $0.079}$	$0.849\text{\scriptsize$ \pm $0.016}$	$0.764\text{\scriptsize$ \pm $0.074}$	✓	✓

Table 2: Classification accuracy vs. baselines (mean

\pm

std. dev. over 5-fold CV). ✓ / ✗ denotes higher/lower accuracy than the average GAM (or GA 2 M) baseline.

Dataset	BB	GAM		CALM	GA 2 M			CALM vs.
	XGB	NAM	EBM		EB 2 M	NODE	GAMI	GAM	GA 2 M
Bike Sharing	$39.35\text{\scriptsize$ \pm $1.38}$	$101.97\text{\scriptsize$ \pm $1.31}$	$100.21\text{\scriptsize$ \pm $1.01}$	$55.67\text{\scriptsize$ \pm $1.22}$	$54.80\text{\scriptsize$ \pm $0.96}$	$54.47\text{\scriptsize$ \pm $1.58}$	$53.44\text{\scriptsize$ \pm $1.90}$	✓	✗
California Housing	$0.45\text{\scriptsize$ \pm $0.01}$	$0.61\text{\scriptsize$ \pm $0.01}$	$0.55\text{\scriptsize$ \pm $0.01}$	$0.51\text{\scriptsize$ \pm $0.01}$	$0.49\text{\scriptsize$ \pm $0.01}$	$0.50\text{\scriptsize$ \pm $0.01}$	$0.51\text{\scriptsize$ \pm $0.04}$	✓	✗
Parkinsons Motor	$1.44\text{\scriptsize$ \pm $0.09}$	$6.11\text{\scriptsize$ \pm $0.16}$	$4.20\text{\scriptsize$ \pm $0.09}$	$2.24\text{\scriptsize$ \pm $0.13}$	$2.35\text{\scriptsize$ \pm $0.06}$	$3.49\text{\scriptsize$ \pm $0.31}$	$2.76\text{\scriptsize$ \pm $0.31}$	✓	✓
Parkinsons Total	$1.86\text{\scriptsize$ \pm $0.08}$	$7.90\text{\scriptsize$ \pm $0.10}$	$4.85\text{\scriptsize$ \pm $0.11}$	$2.97\text{\scriptsize$ \pm $0.09}$	$2.77\text{\scriptsize$ \pm $0.06}$	$4.60\text{\scriptsize$ \pm $0.53}$	$3.81\text{\scriptsize$ \pm $0.66}$	✓	✓
Seoul Bike	$209.6\text{\scriptsize$ \pm $3.47}$	$320.2\text{\scriptsize$ \pm $4.38}$	$303.7\text{\scriptsize$ \pm $3.86}$	$238.9\text{\scriptsize$ \pm $1.64}$	$235.2\text{\scriptsize$ \pm $1.58}$	$231.0\text{\scriptsize$ \pm $4.23}$	$245.82\text{\scriptsize$ \pm $8.45}$	✓	✗

Table 3: Regression RMSE vs. baselines (mean

\pm

std. dev. over 5-fold CV). ✓ / ✗ denotes lower/higher RMSE than the average GAM (or GA 2 Ms) baseline.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Table 7

Dataset	CALM	EB 2 M	NodeGA 2 M	GAMINet
Adult	$4.2\text{\scriptsize$ \pm $1.4}$	$12.0\text{\scriptsize$ \pm $0.0}$	$72.6\text{\scriptsize$ \pm $3.2}$	$20.0\text{\scriptsize$ \pm $0.0}$
COMPAS	$1.4\text{\scriptsize$ \pm $1.7}$	$9.0\text{\scriptsize$ \pm $0.0}$	$36.0\text{\scriptsize$ \pm $0.0}$	$20.0\text{\scriptsize$ \pm $0.0}$
HELOC	$1.1\text{\scriptsize$ \pm $1.6}$	$21.0\text{\scriptsize$ \pm $0.0}$	$163.8\text{\scriptsize$ \pm $4.4}$	$20.0\text{\scriptsize$ \pm $0.0}$
MIMIC2	$11.5\text{\scriptsize$ \pm $2.1}$	$16.0\text{\scriptsize$ \pm $0.0}$	$115.0\text{\scriptsize$ \pm $3.3}$	$20.0\text{\scriptsize$ \pm $0.0}$
Appendicitis	$4.0\text{\scriptsize$ \pm $1.7}$	$7.0\text{\scriptsize$ \pm $0.0}$	$21.0\text{\scriptsize$ \pm $0.0}$	$6.2\text{\scriptsize$ \pm $8.1}$
Phoneme	$8.6\text{\scriptsize$ \pm $1.6}$	$5.0\text{\scriptsize$ \pm $0.0}$	$10.0\text{\scriptsize$ \pm $0.0}$	$10.0\text{\scriptsize$ \pm $0.0}$

Appendix

Table 6: Number of Interactions per Model in Classification Datasets

Dataset	CALM	EB 2 M	NodeGA 2 M	GAMINet
Bike Sharing	$19.3\text{\scriptsize$ \pm $2.7}$	$10.0\text{\scriptsize$ \pm $0.0}$	$53.6\text{\scriptsize$ \pm $1.4}$	$20.0\text{\scriptsize$ \pm $0.0}$
California Housing	$10.7\text{\scriptsize$ \pm $2.0}$	$8.0\text{\scriptsize$ \pm $0.0}$	$28.0\text{\scriptsize$ \pm $0.0}$	$20.0\text{\scriptsize$ \pm $0.0}$
Parkinsons Motor	$13.8\text{\scriptsize$ \pm $4.4}$	$18.0\text{\scriptsize$ \pm $0.0}$	$126.2\text{\scriptsize$ \pm $1.7}$	$20.0\text{\scriptsize$ \pm $0.0}$
Parkinsons Total	$14.4\text{\scriptsize$ \pm $5.8}$	$18.0\text{\scriptsize$ \pm $0.0}$	$129.2\text{\scriptsize$ \pm $3.9}$	$20.0\text{\scriptsize$ \pm $0.0}$
Seoul Bike	$20.4\text{\scriptsize$ \pm $1.0}$	$13.0\text{\scriptsize$ \pm $0.0}$	$83.2\text{\scriptsize$ \pm $0.7}$	$20.0\text{\scriptsize$ \pm $0.0}$
Wine	$13.9\text{\scriptsize$ \pm $3.3}$	$10.0\text{\scriptsize$ \pm $0.0}$	$54.6\text{\scriptsize$ \pm $0.5}$	$20.0\text{\scriptsize$ \pm $0.0}$

Appendix

Table 7: Number of Interactions per Model in Regression Datasets

Dataset	BlackBox	GAM		CALM	GA 2 M
	XGB	NAM	EBM		EB 2 M	NodeGA 2 M	GAMINet
Adult	$0.2\text{\scriptsize$ \pm $0.01}$	$38\text{\scriptsize$ \pm $2}$	$10\text{\scriptsize$ \pm $1}$	$36\text{\scriptsize$ \pm $1}$	$13\text{\scriptsize$ \pm $2}$	$97\text{\scriptsize$ \pm $9}$	$718\text{\scriptsize$ \pm $122}$
COMPAS	$0.1\text{\scriptsize$ \pm $0.004}$	$10\text{\scriptsize$ \pm $0.1}$	$2\text{\scriptsize$ \pm $2}$	$2\text{\scriptsize$ \pm $0.1}$	$1\text{\scriptsize$ \pm $0.04}$	$51\text{\scriptsize$ \pm $1}$	$129\text{\scriptsize$ \pm $5}$
HELOC	$0.2\text{\scriptsize$ \pm $0.01}$	$31\text{\scriptsize$ \pm $6}$	$2\text{\scriptsize$ \pm $2}$	$17\text{\scriptsize$ \pm $1}$	$2\text{\scriptsize$ \pm $0.2}$	$57\text{\scriptsize$ \pm $0.1}$	$249\text{\scriptsize$ \pm $60}$
MIMIC2	$0.2\text{\scriptsize$ \pm $0.01}$	$31\text{\scriptsize$ \pm $0.3}$	$4\text{\scriptsize$ \pm $3}$	$26\text{\scriptsize$ \pm $3}$	$5\text{\scriptsize$ \pm $1}$	$69\text{\scriptsize$ \pm $9}$	$359\text{\scriptsize$ \pm $32}$
Appendicitis	$0.1\text{\scriptsize$ \pm $0.001}$	$5\text{\scriptsize$ \pm $0.1}$	$2\text{\scriptsize$ \pm $4}$	$2\text{\scriptsize$ \pm $0.1}$	$1\text{\scriptsize$ \pm $0.5}$	$24\text{\scriptsize$ \pm $0.1}$	$6\text{\scriptsize$ \pm $5}$

Appendix

Table 8: Runtime (Seconds) for Classification Datasets

Dataset	BlackBox	GAM		CALM	GA 2 M
	XGB	NAM	EBM		EB 2 M	NodeGA 2 M	GAMINet
Bike Sharing	$0.1\text{\scriptsize$ \pm $0.01}$	$18\text{\scriptsize$ \pm $1}$	$2\text{\scriptsize$ \pm $1}$	$15\text{\scriptsize$ \pm $1}$	$19\text{\scriptsize$ \pm $3}$	$187\text{\scriptsize$ \pm $31}$	$299\text{\scriptsize$ \pm $33}$
California Housing	$0.2\text{\scriptsize$ \pm $0.01}$	$15\text{\scriptsize$ \pm $0.1}$	$4\text{\scriptsize$ \pm $2}$	$13\text{\scriptsize$ \pm $1}$	$16\text{\scriptsize$ \pm $0.5}$	$200\text{\scriptsize$ \pm $46}$	$677\text{\scriptsize$ \pm $147}$
Parkinsons Motor	$0.3\text{\scriptsize$ \pm $0.003}$	$21\text{\scriptsize$ \pm $6}$	$5\text{\scriptsize$ \pm $2}$	$17\text{\scriptsize$ \pm $2}$	$44\text{\scriptsize$ \pm $4}$	$193\text{\scriptsize$ \pm $34}$	$536\text{\scriptsize$ \pm $88}$
Parkinsons Total	$0.3\text{\scriptsize$ \pm $0.01}$	$18\text{\scriptsize$ \pm $0.2}$	$5\text{\scriptsize$ \pm $3}$	$16\text{\scriptsize$ \pm $2}$	$40\text{\scriptsize$ \pm $5}$	$227\text{\scriptsize$ \pm $76}$	$543\text{\scriptsize$ \pm $79}$
Seoul Bike	$0.2\text{\scriptsize$ \pm $0.01}$	$15\text{\scriptsize$ \pm $0.2}$	$3\text{\scriptsize$ \pm $3}$	$13\text{\scriptsize$ \pm $3}$	$12\text{\scriptsize$ \pm $4}$	$127\text{\scriptsize$ \pm $18}$	$268\text{\scriptsize$ \pm $53}$

Appendix

Table 9: Runtime (Seconds) for Regression Datasets

Dataset	BlackBox	GAM			CALM - NAM		CALM - EBM		CALM - Spline
	DNN	NAM	EBM	Spline	PDP	RHALE	PDP	RHALE	PDP	RHALE
Bike Sharing	$42.952\text{\scriptsize$ \pm $1.460}$	$101.968\text{\scriptsize$ \pm $1.309}$	$100.211\text{\scriptsize$ \pm $1.010}$	$100.876\text{\scriptsize$ \pm $0.868}$	$63.034\text{\scriptsize$ \pm $0.883}$	$64.941\text{\scriptsize$ \pm $2.566}$	$58.444\text{\scriptsize$ \pm $1.637}$	$60.966\text{\scriptsize$ \pm $2.983}$	$59.868\text{\scriptsize$ \pm $1.564}$	$62.108\text{\scriptsize$ \pm $2.989}$
California Housing	$0.519\text{\scriptsize$ \pm $0.017}$	$0.611\text{\scriptsize$ \pm $0.011}$	$0.554\text{\scriptsize$ \pm $0.008}$	$0.632\text{\scriptsize$ \pm $0.009}$	$0.580\text{\scriptsize$ \pm $0.012}$	$0.591\text{\scriptsize$ \pm $0.016}$	$0.522\text{\scriptsize$ \pm $0.010}$	$0.526\text{\scriptsize$ \pm $0.009}$	$0.593\text{\scriptsize$ \pm $0.018}$	$0.597\text{\scriptsize$ \pm $0.006}$
Parkinsons Motor	$3.654\text{\scriptsize$ \pm $0.162}$	$6.112\text{\scriptsize$ \pm $0.158}$	$4.204\text{\scriptsize$ \pm $0.087}$	$6.027\text{\scriptsize$ \pm $0.070}$	$4.220\text{\scriptsize$ \pm $0.192}$	$5.303\text{\scriptsize$ \pm $0.312}$	$2.681\text{\scriptsize$ \pm $0.105}$	$3.658\text{\scriptsize$ \pm $0.324}$	$4.149\text{\scriptsize$ \pm $0.211}$	$5.318\text{\scriptsize$ \pm $0.430}$
Parkinsons Total	$5.139\text{\scriptsize$ \pm $0.102}$	$7.897\text{\scriptsize$ \pm $0.102}$	$4.847\text{\scriptsize$ \pm $0.107}$	$7.519\text{\scriptsize$ \pm $0.072}$	$5.456\text{\scriptsize$ \pm $0.182}$	$6.776\text{\scriptsize$ \pm $0.381}$	$3.587\text{\scriptsize$ \pm $0.030}$	$4.319\text{\scriptsize$ \pm $0.222}$	$5.408\text{\scriptsize$ \pm $0.365}$	$6.827\text{\scriptsize$ \pm $0.209}$
Seoul Bike	$244.967\text{\scriptsize$ \pm $5.081}$	$320.225\text{\scriptsize$ \pm $4.377}$	$303.685\text{\scriptsize$ \pm $3.855}$	$302.020\text{\scriptsize$ \pm $4.701}$	$269.277\text{\scriptsize$ \pm $6.414}$	$273.241\text{\scriptsize$ \pm $8.379}$	$249.457\text{\scriptsize$ \pm $2.407}$	$245.880\text{\scriptsize$ \pm $5.153}$	$246.855\text{\scriptsize$ \pm $3.936}$	$243.926\text{\scriptsize$ \pm $4.433}$

Appendix

Table 10: RMSE Score for Regression Datasets, DNN

Dataset	BlackBox	GAM			CALM
	XGB	NAM	EBM	Spline	NAM	EBM	Spline
Bike Sharing	$39.346\text{\scriptsize$ \pm $1.375}$	$101.968\text{\scriptsize$ \pm $1.309}$	$100.211\text{\scriptsize$ \pm $1.010}$	$100.876\text{\scriptsize$ \pm $0.868}$	$59.732\text{\scriptsize$ \pm $1.414}$	$55.667\text{\scriptsize$ \pm $1.220}$	$56.656\text{\scriptsize$ \pm $1.320}$
California Housing	$0.454\text{\scriptsize$ \pm $0.010}$	$0.611\text{\scriptsize$ \pm $0.011}$	$0.554\text{\scriptsize$ \pm $0.008}$	$0.632\text{\scriptsize$ \pm $0.009}$	$0.566\text{\scriptsize$ \pm $0.010}$	$0.511\text{\scriptsize$ \pm $0.010}$	$0.594\text{\scriptsize$ \pm $0.014}$
Parkinsons Motor	$1.443\text{\scriptsize$ \pm $0.094}$	$6.112\text{\scriptsize$ \pm $0.158}$	$4.204\text{\scriptsize$ \pm $0.087}$	$6.027\text{\scriptsize$ \pm $0.070}$	$4.574\text{\scriptsize$ \pm $0.137}$	$2.243\text{\scriptsize$ \pm $0.126}$	$4.432\text{\scriptsize$ \pm $0.185}$
Parkinsons Total	$1.863\text{\scriptsize$ \pm $0.079}$	$7.897\text{\scriptsize$ \pm $0.102}$	$4.847\text{\scriptsize$ \pm $0.107}$	$7.519\text{\scriptsize$ \pm $0.072}$	$6.109\text{\scriptsize$ \pm $0.361}$	$2.968\text{\scriptsize$ \pm $0.089}$	$5.638\text{\scriptsize$ \pm $0.288}$
Seoul Bike	$209.587\text{\scriptsize$ \pm $3.474}$	$320.225\text{\scriptsize$ \pm $4.377}$	$303.685\text{\scriptsize$ \pm $3.855}$	$302.020\text{\scriptsize$ \pm $4.701}$	$270.365\text{\scriptsize$ \pm $7.049}$	$238.912\text{\scriptsize$ \pm $1.643}$	$237.705\text{\scriptsize$ \pm $2.658}$

Appendix

Table 11: RMSE Score for Regression Datasets, XGB

Dataset	BlackBox	GAM			CALM
	RF	NAM	EBM	Spline	NAM	EBM	Spline
Bike Sharing	$43.247\text{\scriptsize$ \pm $1.037}$	$101.968\text{\scriptsize$ \pm $1.309}$	$100.211\text{\scriptsize$ \pm $1.010}$	$100.876\text{\scriptsize$ \pm $0.868}$	$61.174\text{\scriptsize$ \pm $1.544}$	$57.158\text{\scriptsize$ \pm $1.288}$	$57.823\text{\scriptsize$ \pm $1.393}$
California Housing	$0.502\text{\scriptsize$ \pm $0.012}$	$0.611\text{\scriptsize$ \pm $0.011}$	$0.554\text{\scriptsize$ \pm $0.008}$	$0.632\text{\scriptsize$ \pm $0.009}$	$0.570\text{\scriptsize$ \pm $0.015}$	$0.515\text{\scriptsize$ \pm $0.012}$	$0.611\text{\scriptsize$ \pm $0.025}$
Parkinsons Motor	$1.477\text{\scriptsize$ \pm $0.155}$	$6.112\text{\scriptsize$ \pm $0.158}$	$4.204\text{\scriptsize$ \pm $0.087}$	$6.027\text{\scriptsize$ \pm $0.070}$	$4.515\text{\scriptsize$ \pm $0.094}$	$2.230\text{\scriptsize$ \pm $0.081}$	$4.349\text{\scriptsize$ \pm $0.061}$
Parkinsons Total	$1.869\text{\scriptsize$ \pm $0.243}$	$7.897\text{\scriptsize$ \pm $0.102}$	$4.847\text{\scriptsize$ \pm $0.107}$	$7.519\text{\scriptsize$ \pm $0.072}$	$5.761\text{\scriptsize$ \pm $0.259}$	$3.044\text{\scriptsize$ \pm $0.068}$	$5.393\text{\scriptsize$ \pm $0.162}$
Seoul Bike	$225.572\text{\scriptsize$ \pm $3.086}$	$320.225\text{\scriptsize$ \pm $4.377}$	$303.685\text{\scriptsize$ \pm $3.855}$	$302.020\text{\scriptsize$ \pm $4.701}$	$270.514\text{\scriptsize$ \pm $8.146}$	$243.910\text{\scriptsize$ \pm $2.580}$	$240.471\text{\scriptsize$ \pm $4.337}$

Appendix

Table 12: RMSE Score for Regression Datasets, RF

Dataset	BlackBox	GAM			CALM - NAM		CALM - EBM		CALM - Spline
	DNN	NAM	EBM	Spline	PDP	RHALE	PDP	RHALE	PDP	RHALE
Adult	$0.849\text{\scriptsize$ \pm $0.002}$	$0.851\text{\scriptsize$ \pm $0.002}$	$0.870\text{\scriptsize$ \pm $0.002}$	$0.856\text{\scriptsize$ \pm $0.001}$	$0.853\text{\scriptsize$ \pm $0.001}$	$0.854\text{\scriptsize$ \pm $0.003}$	$0.870\text{\scriptsize$ \pm $0.003}$	$0.871\text{\scriptsize$ \pm $0.003}$	$0.857\text{\scriptsize$ \pm $0.003}$	$0.859\text{\scriptsize$ \pm $0.003}$
COMPAS	$0.682\text{\scriptsize$ \pm $0.011}$	$0.682\text{\scriptsize$ \pm $0.016}$	$0.681\text{\scriptsize$ \pm $0.012}$	$0.685\text{\scriptsize$ \pm $0.012}$	$0.682\text{\scriptsize$ \pm $0.007}$	$0.681\text{\scriptsize$ \pm $0.014}$	$0.683\text{\scriptsize$ \pm $0.011}$	$0.679\text{\scriptsize$ \pm $0.012}$	$0.683\text{\scriptsize$ \pm $0.015}$	$0.683\text{\scriptsize$ \pm $0.011}$
HELOC	$0.721\text{\scriptsize$ \pm $0.009}$	$0.723\text{\scriptsize$ \pm $0.011}$	$0.728\text{\scriptsize$ \pm $0.014}$	$0.726\text{\scriptsize$ \pm $0.010}$	$0.723\text{\scriptsize$ \pm $0.011}$	$0.723\text{\scriptsize$ \pm $0.013}$	$0.727\text{\scriptsize$ \pm $0.015}$	$0.728\text{\scriptsize$ \pm $0.014}$	$0.721\text{\scriptsize$ \pm $0.012}$	$0.726\text{\scriptsize$ \pm $0.011}$
MIMIC2	$0.885\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.001}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.887\text{\scriptsize$ \pm $0.001}$	$0.886\text{\scriptsize$ \pm $0.001}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.002}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.003}$
Appendicitis	$0.877\text{\scriptsize$ \pm $0.072}$	$0.848\text{\scriptsize$ \pm $0.056}$	$0.877\text{\scriptsize$ \pm $0.077}$	$0.887\text{\scriptsize$ \pm $0.064}$	$0.839\text{\scriptsize$ \pm $0.025}$	$0.848\text{\scriptsize$ \pm $0.056}$	$0.897\text{\scriptsize$ \pm $0.055}$	$0.877\text{\scriptsize$ \pm $0.077}$	$0.887\text{\scriptsize$ \pm $0.064}$	$0.887\text{\scriptsize$ \pm $0.064}$

Appendix

Table 13: Accuracy Score for Classification Datasets, DNN

Dataset	BlackBox	GAM			CALM
	XGB	NAM	EBM	Spline	NAM	EBM	Spline
Adult	$0.870\text{\scriptsize$ \pm $0.002}$	$0.851\text{\scriptsize$ \pm $0.002}$	$0.870\text{\scriptsize$ \pm $0.002}$	$0.856\text{\scriptsize$ \pm $0.001}$	$0.853\text{\scriptsize$ \pm $0.002}$	$0.870\text{\scriptsize$ \pm $0.002}$	$0.858\text{\scriptsize$ \pm $0.001}$
COMPAS	$0.661\text{\scriptsize$ \pm $0.007}$	$0.682\text{\scriptsize$ \pm $0.016}$	$0.681\text{\scriptsize$ \pm $0.012}$	$0.685\text{\scriptsize$ \pm $0.012}$	$0.686\text{\scriptsize$ \pm $0.014}$	$0.684\text{\scriptsize$ \pm $0.013}$	$0.686\text{\scriptsize$ \pm $0.016}$
HELOC	$0.717\text{\scriptsize$ \pm $0.013}$	$0.723\text{\scriptsize$ \pm $0.011}$	$0.728\text{\scriptsize$ \pm $0.014}$	$0.726\text{\scriptsize$ \pm $0.010}$	$0.724\text{\scriptsize$ \pm $0.011}$	$0.728\text{\scriptsize$ \pm $0.012}$	$0.726\text{\scriptsize$ \pm $0.011}$
MIMIC2	$0.890\text{\scriptsize$ \pm $0.001}$	$0.886\text{\scriptsize$ \pm $0.001}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.001}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.003}$
Appendicitis	$0.868\text{\scriptsize$ \pm $0.069}$	$0.848\text{\scriptsize$ \pm $0.056}$	$0.877\text{\scriptsize$ \pm $0.077}$	$0.887\text{\scriptsize$ \pm $0.064}$	$0.868\text{\scriptsize$ \pm $0.069}$	$0.878\text{\scriptsize$ \pm $0.063}$	$0.878\text{\scriptsize$ \pm $0.063}$

Appendix

Table 14: Accuracy Score for Classification Datasets, XGB

Dataset	BlackBox	GAM			CALM
	RF	NAM	EBM	Spline	NAM	EBM	Spline
Adult	$0.843\text{\scriptsize$ \pm $0.001}$	$0.851\text{\scriptsize$ \pm $0.002}$	$0.870\text{\scriptsize$ \pm $0.002}$	$0.856\text{\scriptsize$ \pm $0.001}$	$0.853\text{\scriptsize$ \pm $0.002}$	$0.871\text{\scriptsize$ \pm $0.003}$	$0.859\text{\scriptsize$ \pm $0.001}$
COMPAS	$0.662\text{\scriptsize$ \pm $0.016}$	$0.682\text{\scriptsize$ \pm $0.016}$	$0.681\text{\scriptsize$ \pm $0.012}$	$0.685\text{\scriptsize$ \pm $0.012}$	$0.682\text{\scriptsize$ \pm $0.016}$	$0.683\text{\scriptsize$ \pm $0.012}$	$0.685\text{\scriptsize$ \pm $0.012}$
HELOC	$0.724\text{\scriptsize$ \pm $0.014}$	$0.723\text{\scriptsize$ \pm $0.011}$	$0.728\text{\scriptsize$ \pm $0.014}$	$0.726\text{\scriptsize$ \pm $0.010}$	$0.723\text{\scriptsize$ \pm $0.014}$	$0.728\text{\scriptsize$ \pm $0.014}$	$0.726\text{\scriptsize$ \pm $0.010}$
MIMIC2	$0.888\text{\scriptsize$ \pm $0.002}$	$0.886\text{\scriptsize$ \pm $0.001}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.003}$	$0.886\text{\scriptsize$ \pm $0.002}$	$0.887\text{\scriptsize$ \pm $0.002}$	$0.886\text{\scriptsize$ \pm $0.003}$
Appendicitis	$0.859\text{\scriptsize$ \pm $0.064}$	$0.848\text{\scriptsize$ \pm $0.056}$	$0.877\text{\scriptsize$ \pm $0.077}$	$0.887\text{\scriptsize$ \pm $0.064}$	$0.848\text{\scriptsize$ \pm $0.056}$	$0.877\text{\scriptsize$ \pm $0.077}$	$0.887\text{\scriptsize$ \pm $0.064}$

Appendix

Table 15: Accuracy Score for Classification Datasets, RF

Interpretable ML

Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.

See all 306 papers

Selecting Feature Interactions for Generalized Additive Models by Distilling Foundation Models
Apr 14, 2026Jingyun Jia, Chandan Singh, Rich Caruana +1Feature Interaction Modeling
University of Wisconsin–Madison · Microsoft Research · Intelligible AI
Interpretable AI with Local Distillation
Aug 24, 2026Erin Craig, Yiling Huang, Snigdha PanigrahiSurrogate Modeling Interpretable ML
Department of Biostatistics, University of Michigan · Department of Statistics, University of Michigan
ParamBoost: Gradient Boosted Piecewise Cubic Polynomials
Apr 20, 2026Nicolas Salvadé, Tim HillelInterpretable ML
Department of Civil, Environmental and Geomatic Engineering, University College London, Gower Street, London, WC1E 6BT,, United Kingdom

Apr 14, 2026J/K move · Enter open · S save

Selecting Feature Interactions for Generalized Additive Models by Distilling Foundation Models

Jingyun Jia, Chandan Singh, Rich Caruana, Ben Lengerich

University of Wisconsin–Madison · Microsoft Research · Intelligible AI

Feature Interaction Modeling

Open paper AskPDF arXiv

Identifying meaningful feature interactions is a central challenge in building accurate and interpretable models for tabular data. Generalized additive models (GAMs) have shown great success at modeling tabular data, but often rely on heuristic procedures to select interactions, potentially missing higher-order or context-dependent effects. To meet this challenge, we propose TabDistill, a method that leverages tabular foundation models and post-hoc distillation methods. Our key intuition is that tabular foundation models implicitly learn rich, adaptive feature dependencies through large-scale representation learning. Given a dataset, TabDistill first fits a tabular foundation model to the dataset, and then applies a post-hoc interaction attribution method to extract salient feature interactions from it. We evaluate these interactions by then using them as terms in a GAM. Across tasks, we find that interactions identified by TabDistill lead to consistent improvements in downstream GAMs' predictive performance. Our results suggest that tabular foundation models can serve as effective, data-driven guides for interaction discovery, bridging high-capacity models and interpretable additive frameworks.

Abstract

Interpretable ML

Explore similar work

Selecting Feature Interactions for Generalized Additive Models by Distilling Foundation Models

Interpretable AI with Local Distillation

ParamBoost: Gradient Boosted Piecewise Cubic Polynomials