Leaf Values as Coordinates: Exact Contrastive Explanation for Gradient-Boosted Ensembles

Emanuele Luzio

cs.LGAug 19, 2026

Leaf Values as Coordinates: Exact Contrastive Explanation for Gradient-Boosted Ensembles

Authors: Emanuele Luzio

Organizations: Independent Researcher

Abstract

A gradient-boosted ensemble predicts by summing one leaf value per tree. Read those values as coordinates rather than as intermediate results, and every instance becomes a point in R^M on which the model acts linearly: the score is the sum of the coordinates. This small change of view makes contrastive explanation exact. The difference between two instances is a vector that is identically zero wherever they share a leaf, so the gap between a rejected applicant and an accepted one is carried by a handful of coordinates, each traceable to a real split in a real tree. Nothing is fitted, sampled, or assumed additive in features -- the additivity is already there, in the right space. We build a recourse method on this representation and evaluate it on five tabular datasets under repeated cross-validation. Its recommendation reconstructs the model's own decision to 6.2 x 10^-15, so an auditor can re-check the arithmetic without the model. On the credit datasets it is Pareto-non-dominated on effort against realism. And when recommendations are restricted to changes the subject could actually make -- not their age, not a settled delinquency -- it retains 58% of its validity where the strongest baseline retains 41%, a distinction the standard evaluation cannot see because it never asks whether a recommendation can be carried out.

Figures & tables

Feature	Query	Comp.	Threshold	$\Delta v$ (trees)	Solo $\Delta\hat{f}$
NetFractionRevolvingBurden	81	0	31.5	+2.260 (75)	+1.110
AverageMInFile	38	74	53.1	+0.182 (14)	+0.291
MSinceOldestTradeOpen	85	160	120.5	+0.197 (16)	+0.081
NumInstallTradesWBalance	1	2	2.0	+0.050 (2)	+0.029
NetFractionInstallBurden	—	84	74.6	+0.045 (11)	+0.013
sum of entries				$\mathbf{+2.734}$

Table 1. A recommendation for a rejected HELOC applicant, scored

0.151

against a threshold of

0.50

, against a comparator at

0.733

. Each row is a feature reached by summing the coordinates of

\Delta

whose decisive split falls on it; trees counts those coordinates. The last two rows are the point of the representation: the entries reproduce the model’s own margin gap exactly, so the arithmetic can be re-checked without the model. Solo is a different quantity — the shift from moving that feature alone — and the fact that the two columns disagree is the feature-interaction effect that Section 3 introduces

\varepsilon

to absorb. Note also rows two and three: raising an account-age feature is not something an applicant can do, which Section 5.4 takes up.

Figure 1. Effort against realism, five datasets. COACH occupies the low-manifold, high-sparsity corner; NICE the opposite. Hollow markers mark validity below 90 %. The dashed line is where genuine accepted instances sit relative to their own neighbours — a recourse at or below it is, by this measure, indistinguishable from a real accepted person. Five scatter plots, one per dataset, with the fraction of features changed on the horizontal axis and manifold distance in standard deviations on the vertical axis. In each panel COACH sits toward the lower right — changing many features but landing close to real accepted instances — while NICE sits toward the upper left, changing few features but landing further away. A dashed horizontal line marks the distance separating genuine accepted instances from their own neighbours; COACH falls at or below it on three of the five datasets.

unconstrained
Method	FICO HELOC	Taiwan Credit	Adult Income
COACH	0.967	1.000	1.000
NICE	0.978	1.000	1.000
FACE	0.922	0.997	1.000
FeatureTweaking	0.782	1.000	1.000
feasible changes only

Table 2. Validity when the subject may change only what they can change. Immutable features (age, protected attributes, settled history) are frozen and one-directional features may move only the feasible way; the same projection is applied to every method. Filtered removes infeasible moves after retrieval; constraint-aware folds feasibility into comparator selection. Polish Bankruptcy is excluded: we have no defensible mutability labels for 64 derived accounting ratios.

Explore similar work

Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk
Aug 8, 2026Gregorius Reynaldi Pratama, Kuo-Kun TsengFeature Attribution Faithfulness of Language Model Explanations
School of Computer Science and Technology Harbin Institute of Technology, Shenzhen Shenzhen, China
Learning Auditable Classifier Models: Source-Disjoint Tree Ensembles
Aug 16, 2026Srikumar KrishnamoorthyInterpretable ML Ensemble Learning
Information Systems Area, Indian Institute of Management Ahmedabad, India
Optimal Counterfactual Search in Tree Ensembles: A Study Across Modeling and Solution Paradigms
May 7, 2026Awa Khouna, Youssouf Emine, Julien Ferry +1Counterfactual Explanations Mixed-Integer Linear Programming
Polytechnique Montréal · Ivey Business School

Aug 8, 2026, cs.LGJ/K move · Enter open · S save

Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk

Gregorius Reynaldi Pratama, Kuo-Kun Tseng

School of Computer Science and Technology Harbin Institute of Technology, Shenzhen Shenzhen, China

Feature Attribution Faithfulness of Language Model Explanations Ensemble Learning LLM Interpretability Credit Scoring Explainability Evaluation

Open paper AskPDF arXiv

Credit scoring increasingly relies on models whose decision logic cannot be read off their parameters, in tension with supervisory expectations that adverse decisions be explainable. A common proposal closes that gap with a language model: compute feature attributions, hand them to an LLM, and let it write the rationale. We build such a system end to end and test whether the second half of the promise holds. The predictive component is a multi-scale stacking ensemble fusing four differently regularised gradient-boosting learners with a residual network through a neural meta-learner trained on out-of-fold predictions. On a public 32,581-application credit dataset it reaches test ROC-AUC 0.9539 (95% CI [0.9462, 0.9616]) and PR-AUC 0.9137, beating the best single model by Delta-AUC = 0.0143 (p = 0.016 under a conservative independence assumption). Our central finding is asymmetric. The ranking gain is real but operationally small: at the F1-optimal threshold the ensemble avoids only six additional missed defaults out of 1,422 against a tuned random forest, cutting cost-weighted loss by under 2%. The narrative layer fails in a way prompt engineering alone does not fix. In an audited case the model named three factors as risk-increasing that the supplied attributions scored as risk-reducing, omitted the dominant driver, and introduced a feature never given to it. We trace this to properties we measure rather than assume: SHAP and LIME agree on which features matter (overlap@10 = 0.80) but not on their order (tau = 0.43, p = 0.18), and the attribution sign for the model's most sensitive input is near a coin flip across applicants (modal-sign share 0.53). Calibration (ECS = 0.117) and perturbation stability (DPD = 0.078) both fall short of our own thresholds. Constrained prompting is necessary but not sufficient: grounding must be verified after generation, not assumed.

Abstract

Explore similar work

Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk

Learning Auditable Classifier Models: Source-Disjoint Tree Ensembles

Optimal Counterfactual Search in Tree Ensembles: A Study Across Modeling and Solution Paradigms