Shapley-value-based explanations are widely used for feature attribution in machine learning. However, in computer vision, a feature may reduce the target-class score when added to the complete genuine image, even when the predicted label remains unchanged. We formalize such features as contextual distractors and investigate their attributions under semivalues and the least core. Under a weak average marginal monotonicity assumption, we derive quantitative semivalue--least-core gap bounds and sufficient conditions based on the probability of sampling smaller contexts. We further show that shifting semivalue weights toward larger contexts decreases the signed gap. Finally, controlled image-classification experiments and evaluations on Salient ImageNet provide strong empirical support for our theoretical findings.
Figures & tables
Figure 1: Controlled marker-bias setup. A red marker is added to all dog images and a subset of cat images, while bird images remain unmarked. This creates a dog-associated cue that may act as a contextual distractor for cat predictions. In Section 4 , we observe that while Shapley by nature, averages contributions across contexts, the least core incorporates stability constraints that account for the marker’s effect on the complete genuine image and the predicted label.
Figure 2: Analysis of the controlled marker-bias experiment. (a) Example images with synthetic markers. (b) For 100 cat-class images, the x-axis shows the normalized cat-class logit v(T) of masked images, where coalitions T are sampled from non-marker regions. The y-axis shows the logit change caused by adding the marker region D , i.e., v(T∪D)−v(T) . (c) Comparison of Shapley and least-core scores for the marker region over the same 100 cat-class images. Each point corresponds to one image, and the dashed line indicates equal attribution by the two methods.
Figure 3: Representative examples of contextual distractors. Input + D: the original input, with the distractor region D outlined in yellow. Shapley and Least-Core: patch-attribution heatmaps, where red and blue indicate positive and negative attribution, respectively; AD gives the total attribution assigned to the patches in D . Context dependence: the marginal contribution ΔD(T)=v(T∪D)−v(T) plotted against the target-class logit v(T) obtained from the surrounding context T . Logit change in strong context: a sampled strong coalition T before and after adding D , reporting the resulting target-class logit v(T∪D) and its change ΔD from v(T) .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Grad-CAM
Grad-CAM++
Shapley-MC
Least-Core
ϵ (mean ± std)
0.6990±0.1445
0.6899±0.1397
0.6851±0.1637
0.5822±0.1389
Appendix
Table 1: Epsilon values on 100 ImageNet test images. Lower is better.
Figure 4: Controlled marker-bias experiment. Left: representative inputs. The marker is correlated with the dog class and can therefore mislead cat predictions. Middle and right: for both ResNet-50 and ViT-B/16, the marginal contribution ΔD(T)=v(T∪D)−v(T) decreases with the strength of the non-marker context. The marker helps in weak contexts but suppresses the cat logit in strong contexts.
Figure 5: Per-image marker attribution for ResNet-50 (left) and ViT-B/16 (right). We compare the Shapley and Banzhaf ( p=0.5 ) scores with the corresponding least-core scores. Each point represents one retained image, and the dashed line indicates equal attribution; points above the line receive more attribution from the semivalue than from the least-core.
Model
Least-core
Shapley
Gap
Banzhaf ( p=0.5 )
Gap
ResNet-50
0.041
0.220
0.179
0.076
0.035
ViT-B/16
-0.221
0.419
0.639
0.355
0.576
Appendix
Table 2: Mean marker-patch attribution. Each Gap column reports the corresponding semivalue minus the least-core attribution; a positive value indicates that the semivalue assigns more credit to the marker.
Figure 6: Effect of context-size weighting on the group attribution to the marker region D . The left two panels show ResNet-50 and the right two show ViT-B/16. Within each pair, the first panel varies p for the weighted-Banzhaf family, and the second shifts the Beta-Shapley parameters from small-context weighting (4,1) to large-context weighting (1,4) . The solid blue curves show the group semivalue obtained at each parameter setting. The dotted green and dashed red lines show the independently estimated group Shapley baseline and the least-core attribution, respectively.
Model
Region
Outcome
Shapley
Core
Gap
S > C
ResNet-50
Contextual distractor
AD
2.338
1.262
1.076
82.5%
Amask
1.528
0.873
0.655
78.8%
Random-SAM control
AD
0.614
0.838
−0.224
33.1%
Amask
0.434
0.579
−0.144
33.1%
ViT-B/16
Contextual distractor
AD
2.177
1.645
0.532
80.6%
Amask
1.435
1.112
0.323
76.9%
Appendix
Table 3: Shapley and sampled egalitarian least-core attribution for contextual distractors and prediction-active random-SAM controls. Gap is Shapley minus Core, and S > C is the fraction of regions for which the Shapley attribution exceeds the Core attribution.
Explainable artificial intelligence promises to yield insights into relevant features, thereby enabling humans to examine and scrutinize machine learning models or even facilitating scientific discovery. Considering the widespread technique of Shapley values, we find that purely data-driven operationalization of multivariate feature importance is unsuitable for such purposes. Even for simple problems with two features, spurious associations due to collider bias and suppression arise from considering one feature only in the observational context of the other, which can lead to misinterpretations. Causal knowledge about the data-generating process is required to identify and correct such misleading feature attributions. We propose cc-Shapley (causal context Shapley), an interventional modification of conventional observational Shapley values leveraging knowledge of the data's causal structure, thereby analyzing the relevance of a feature in the causal context of the remaining features. We show theoretically that this eradicates spurious association induced by collider bias. We compare the behavior of Shapley and cc-Shapley values on various, synthetic, and real-world datasets. We observe nullification or reversal of associations compared to univariate feature importance when moving from observational to cc-Shapley.
Machine learning pipelines commonly flatten relational data into single-table representations, discarding structural constraints. Widely used Shapley value-based feature attributions then rely on feature independence, evaluating the model on combinations that could never arise in the underlying data, producing misleading explanations. We propose RelShap, a framework that incorporates relational constraints and data provenance into Shapley value computation, restricting both background data and coalition evaluation to relationally valid configurations. The framework is estimator-agnostic and composes with Kernel SHAP, Monte Carlo, and Leverage SHAP without altering their sampling or weighting properties. Functional dependencies further induce equivalence classes over feature coalitions, which RelShap exploits to reduce runtime without changing Shapley values; we provide a combinatorial characterization of the expected speedup. Experiments across multiple datasets, models, and estimators show that RelShap produces explanations that are more faithful to the data-generating process, correctly identifying the dominant feature in controlled settings where existing methods, including Conditional SHAP and ManifoldShap, do not. Our code is available at: https://github.com/duneag2/relshap.
Seungeun Lee, Joao Fonseca, Julia Stoyanovich
New York University, New York, USA · INESC-ID, Lisbon, Portugal
Feature attribution analysis is critical for interpreting machine learning models and supporting reliable data-driven decisions. However, feature attribution measures often exhibit stochastic variation: different train--test splits, random seeds, or model-fitting procedures can produce substantially different attribution values and feature rankings. This paper proposes a framework for incorporating stochastic nature of feature attribution and a robust attribution metric, RoSHAP, for stable feature ranking based on the SHAP metric. The proposed framework models the distribution of feature attribution scores and estimates it through bootstrap resampling and kernel density estimation. We show that, under mild regularity conditions, the aggregated feature attribution score is asymptotically Gaussian, which greatly reduces the computational cost of distribution estimation. The RoSHAP summarizes the distribution of SHAP into a robust feature-ranking criterion that simultaneously rewards features that are active, strong, and stable. Through simulations and real-data experiments, the proposed framework and RoSHAP outperform standard single-run attribution measures in identifying signal features. In addition, models built using RoSHAP-selected features achieve predictive performance comparable to full-feature models while using substantially fewer predictors. The proposed RoSHAP approach improves the stability and interpretability of machine learning models, enabling reliable and consistent insights for analysis.
Lanxin Xiang, Liang Shi, Youhui Ye +3
Department of Statistics Virginia Tech · Department of Statistics Virginia Tech Transportation Institute · Department of Computer Science Virginia Tech