We analyze two widely used local attribution methods, Local Shapley Values and LIME, which aim to quantify the contribution of a feature value xi to a specific prediction f(x1,…,xp). Despite their widespread use, we identify fundamental limitations in their ability to reliably detect locally important features, even under ideal conditions with exact computations and independent features. We argue that a sound local attribution method should not assign importance to features that neither influence the model output (e.g., features with zero coefficients in a linear model) nor exhibit statistical dependence with functionality-relevant features. We demonstrate that both Local SV and LIME violate this fundamental principle. To address this, we propose R-LOCO (Regional Leave Out COvariates), which bridges the gap between local and global explanations and provides more accurate attributions. R-LOCO segments the input space into regions with similar feature importance characteristics. It then applies global attribution methods within these regions, deriving an instance's feature contributions from its regional membership. This approach delivers more faithful local attributions while avoiding local explanation instability and preserving instance-specific detail often lost in global methods.
Figures & tables
1st-order
2nd-order
2nd-order-interact.
Method
TP
FP
NI attr.
TP
FP
NI attr.
TP
FP
NI attr.
R-LOCO
0.97
0.03
0.02 (0.01–0.04)
0.86
0.14
0.02 (0.01–0.04)
0.67
0.33
0.03 (0.02–0.07)
R-LOCO TC
1.00
0.00
0.03 (0.03–0.03)
1.00
0.00
0.03 (0.03–0.03)
0.94
0.06
0.04 (0.03–0.04)
R-LOCO tree
0.90
0.10
0.05 (0.02–0.11)
0.73
0.27
0.04 (0.02–0.06)
0.50
0.50
0.06 (0.05–0.07)
R-LOCO IC
0.89
0.11
0.05 (0.02–0.11)
0.74
0.26
0.04 (0.03–0.07)
0.65
0.35
0.05 (0.04–0.07)
LOCO
0.50
0.50
0.09 (0.09–0.09)
0.50
0.50
0.05 (0.05–0.05)
0.50
0.50
0.06 (0.05–0.07)
Table 1: Performance metrics for methods across datasets from 1st-order to 2nd-order-interactions. TP: True Positives, FP: False Positives, NI attr.: Mean (q0.1 – q0.95) of non-important features attributions.
1st-order
2nd-order
2nd-order-interact.
Method
TP
FP
NI attr.
TP
FP
NI attr.
TP
FP
NI attr.
R-LOCO KMeans-2
0.98
0.02
0.03 (0.03–0.03)
0.92
0.08
0.04 (0.03–0.06)
0.57
0.43
0.05 (0.03–0.09)
R-LOCO KMeans-4
0.97
0.03
0.03 (0.02–0.08)
0.83
0.17
0.03 (0.03–0.06)
0.62
0.38
0.04 (0.02–0.08)
R-LOCO KMeans-8
0.96
0.04
0.03 (0.02–0.06)
0.93
0.07
0.03 (0.02–0.04)
0.59
0.41
0.04 (0.02–0.08)
R-LOCO KMeans-20
0.97
0.03
0.02 (0.02–0.04)
0.96
0.04
0.03 (0.02–0.04)
0.64
0.36
0.04 (0.02–0.08)
R-LOCO Affinity-0.5
0.97
0.03
0.02 (0.01–0.04)
0.86
0.14
0.02 (0.01–0.04)
0.68
0.32
0.03 (0.02–0.07)
Table 2: Performance metrics for R-LOCO using different clustering algorithms across datasets from 1st-order to 2nd-order-interactions. TP: True Positives, FP: False Positives, NI attr.: Mean (q0.1 – q0.95) of non-important features attributions.
Figure 3
Figure 3: Feature attributions from R-LOCO and R-LOCO IC on the first-order model, highlighting the intra-group variance of feature importance.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5
Figure 6: This diagram presents and contrasts our proposed method ( R-LOCO ) with the global approach ( LOCO ). Both approaches first transform the features into the feature importance space. The global approach (in grey ) then averages the importance scores column-wise across all observations, while R-LOCO (in green ) averages the scores within each cluster, providing a more localized attribution. However, note that nothing stops us from applying any other global methods after the clusters are identified. The estimation of the region-based attributions is very generic; we could simply learn a linear model within each identified region and use it as a global explanation, or apply any more advanced global attribution techniques. For example, we could re-estimate the f0 , and f−j of Δj(X,Y)=V(f0(X),Y)−V(f−j(X),Y) used to estimate Ψj(PAk) within the corresponding cluster instead of using the one learned to identify the clusters. We found that both strategies perform similarly in our experiments.
Method
Train (s)
Infer (s)
Total (s)
Train (MB)
Infer (MB)
Peak (MB)
R-LOCO
33.521
0.032
33.553
1687.9
8.6
2743.0
LIME
0.038
34.684
34.722
0.5
2.3
1.0
SHAP
0.002
20.698
20.700
0.0
0.8
6.8
Appendix
Table 3: Runtime and memory usage comparison on the California dataset.
Method
Train (s)
Infer (s)
Total (s)
Train (MB)
Infer (MB)
Peak (MB)
R-LOCO
1.611
0.010
1.621
12.2
0.5
1.7
SHAP
0.002
1.433
1.436
0.0
145.0
16.7
LIME
0.008
0.838
0.846
0.2
1.3
0.4
Appendix
Table 4: Runtime and memory usage comparison on the Diabetes dataset.
Feature-attribution methods are central to explainable artificial intelligence. Their assumptions are expressed in several mathematical languages: cooperative-game values, path integrals, gradient operators, perturbation distributions, and backpropagation rules. This survey proposes a common framework for local additive feature attribution. It organizes Shapley, path-based, gradient/backpropagation, perturbation, and CAM-style methods around five specification choices: value function, reference, path, perturbation distribution, and conservation rule. It then compares these methods through an axiom-by-method matrix and links common failure modes, including baseline sensitivity, off-manifold perturbations, sanity-check failures, adversarial manipulation, and method disagreement, to the assumptions that produce them. Finally, the survey proposes a ten-item reporting checklist for studies that use local additive attributions. The central message is that attribution results are meaningful only relative to the mathematical assumptions under which they are defined, and that those assumptions should be reported.
Prototype-based explanations offer an intuitive, example-based approach to support the interpretability of machine learning black box classifiers but often lack feature-level granularity. We introduce a framework that integrates feature importance at two levels to address this gap. First, for local explanations, we propose alike parts: a method that uses feature importance scores to highlight the most relevant, shared feature subsets between a classified instance and its nearest prototype, guiding user attention. Second, we augment the global prototype selection objective function with a feature importance term to actively promote diversity in the feature attributions of the selected prototypes. Experiments on six benchmark datasets show that this augmented selection process maintains or, in some cases, increases the prediction fidelity of the surrogate model, suggesting that feature diversity does not compromise model fidelity.
Jacek Karolczak, Jerzy Stefanowski
Institute of Computing Science Poznan University of Technology ul. Piotrowo 2, 60-965 Pozna´n, Poland
Assessing the importance of individual features in Machine Learning is critical to understand the model's decision-making process. While numerous methods exist, the lack of a definitive ground truth for comparison highlights the need for alternative, well-founded measures. This paper introduces a novel post-hoc local feature importance method called Counterfactual Importance Distribution (CID). We generate two sets of positive and negative counterfactuals, model their distributions using Kernel Density Estimation, and rank features based on a distributional dissimilarity measure. This measure, grounded in a rigorous mathematical framework, satisfies key properties required to function as a valid metric. We showcase the effectiveness of our method by comparing with well-established local feature importance explainers. Our method not only offers complementary perspectives to existing approaches, but also improves performance on faithfulness metrics (both for comprehensiveness and sufficiency), resulting in more faithful explanations of the system. These results highlight its potential as a valuable tool for model analysis. Link to repository: https://github.com/EddieConti/CID
Eddie Conti, Álvaro Parafita, Axel Brando
TAIES Group, HPES Lab, Barcelona Supercomputing Center (BSC)