Training for Trustworthy Saliency Maps: Adversarial Training Meets Feature-Map Smoothing

Dipkamal Bhusal; Md Tanvirul Alam; Nidhi Rastogi

cs.CVMar 7, 2026

Training for Trustworthy Saliency Maps: Adversarial Training Meets Feature-Map Smoothing

Authors: Dipkamal Bhusal, Md Tanvirul Alam, Nidhi Rastogi

Organizations: Rochester Institute of Technology

Abstract

Gradient-based saliency methods such as Vanilla Gradient (VG) and Integrated Gradients (IG) are widely used to explain image classifiers, yet the resulting maps are often noisy and unstable, limiting their usefulness in high-stakes settings. Most prior work improves explanations by modifying the attribution algorithm, leaving open how the training procedure shapes explanation quality. We take a training-centered view and first provide a curvature-based analysis linking attribution stability to how smoothly the input-gradient field varies locally. Guided by this connection, we study adversarial training and identify a consistent trade-off: it yields sparser and more input-stable saliency maps, but can degrade output-side stability, causing explanations to change even when predictions remain unchanged and logits vary only slightly. To mitigate this, we propose augmenting adversarial training with a lightweight feature-map smoothing block that applies a differentiable Gaussian filter in an intermediate layer. Across FMNIST, CIFAR-10, and ImageNette, our method preserves the sparsity benefits of adversarial training while improving both input-side stability and output-side stability. A human study with 65 participants further shows that smoothed adversarial saliency maps are perceived as more sufficient and trustworthy. Overall, our results demonstrate that explanation quality is critically shaped by training, and that simple smoothing with robust training provides a practical path toward saliency maps that are both sparse and stable.

Figures & tables

Figure 1: Saliency maps (Vanilla Gradient) for the same correctly classified input under three training regimes. (a) Natural training: noisy and diffuse maps that obscure the decision rationale. (b) Adversarial training: sparser maps but with reduced stability, sometimes discarding structurally relevant regions. (c) Adversarial + Gaussian feature-map smoothing: restores stability while retaining sparsity, producing explanations that are more coherent.

Figure 2: Effect of training regimes on intermediate feature maps (channel= {7, 21, 127}, after first residual block) and saliency maps: (a) naturally trained, (b) adversarially trained, (c) adversarially trained with feature-map smoothing, and (d) corresponding VG saliency maps.

Figure 3: Feature-map smoothing block. A spatial filter is followed by a

1\times 1

convolution and a residual connection.

	Natural accuracy %			Robust accuracy %
	N	A	G	N	A	G
FMNIST	89.9	79.9	80.3	9.5	67.7	66.8
CIFAR-10	90.9	80.5	80.8	4.8	54.3	53.9
ImageNette	96.3	70.8	66.6	1.6	12.2	13.5

Table 1: Model performance on natural (N), adversarial (A) and Gaussian-smoothed adversarial (G) models.

		FMNIST			CIFAR-10			ImageNette
		N	A	G	N	A	G	N	A	G
VG	Gini $\uparrow$	0.54	0.74	0.73	0.49	0.68	0.67	0.51	0.53	0.54
VG	ROS $\downarrow$	11.03	13.12	11.77	2.33	2.55	2.54	2.74	2.88	2.77
IG	Gini $\uparrow$	0.77	0.83	0.82	0.61	0.71	0.71	0.63	0.66	0.67
IG	ROS $\downarrow$	7.26	9.19	8.56	4.25	4.48	4.36	5.56	5.60	5.57

Table 2: Sparsity (Gini) and Relative Output Stability (ROS) for Vanilla Gradient (VG) and Integrated Gradients (IG) on FMNIST, CIFAR-10, and ImageNette models: N (naturally trained), A (adversarially trained) & G (gaussian-filter smoothed adversarially trained).

\uparrow

\downarrow

means higher & lower values are better.

Panel of Figure 5 — Figure 5: Structural similarity (SSIM) evaluation of saliency maps for naturally trained (N), adversarially trained (A), and adversarially trained with Gaussian smoothing (G).

	FMNIST			CIFAR-10			ImageNette
	N	A	G	N	A	G	N	A	G
VG	0.22	0.21	0.23	0.20	0.23	0.24	0.13	0.13	0.19
IG	0.32	0.31	0.28	0.42	0.49	0.49	0.33	0.40	0.38

Table 3: ROAD AOPC (higher

\uparrow

is more faithful) for Vanilla Gradient (VG) and Integrated Gradients (IG) on different models of FMNIST, CIFAR-10, and ImageNette: N : naturally trained, A : adversarially trained & G : adversarially trained with Gaussian smoothing.

	Natural (N)	Adv (A)	Adv + Gaussian (G)
Sufficiency	2.08 $\pm$ 0.75	2.99 $\pm$ 0.93	3.33 $\pm$ 1.03
Trust	2.02 $\pm$ 0.82	3.08 $\pm$ 0.90	3.14 $\pm$ 1.01

Table 4: Human study ratings (mean

\pm

std). Higher is better.

Placement	$\Delta$ Gini $\uparrow$	$\Delta$ ROS $\downarrow$
After first block	0.18	0.21
After second block	0.18	0.23
After third block	0.19	0.24

Table 5: Ablation: Effect of Gaussian smoothing block placement on CIFAR-10: changes in sparsity (

\Delta

Gini

\uparrow

) and Relative Output Stability (

\Delta

ROS

\downarrow

) relative to natural training.

		VG		IG
		Gini $\uparrow$	ROS $\downarrow$	Gini $\uparrow$	ROS $\downarrow$
FMNIST	N	0.54	11.03	0.77	7.26
	A	0.74	13.12	0.83	9.19
	G	0.73	11.77	0.82	8.56
	M1	0.74	12.16	0.84	10.18
	M2	0.72	10.01	0.81	11.94

Table 6: Ablation : Sparsity (Gini) and Relative Output Stability (ROS) for Vanilla Gradient (VG) and Integrated Gradients (IG) on models of FMNIST, CIFAR-10, & ImageNette: N : naturally trained, A : adversarially trained, G : adversarially trained with Gaussian smoothing, M1 : adversarially trained with Mean smoothing, & M2 : adversarially trained with Median smoothing.

Panel of Figure 7 — Figure 7: Ablation: Structural similarity evaluation of saliency maps on various ImageNette models: naturally trained (N) , adversarially trained (A) , and adversarial trained with smoothing filters ( M1 : mean filter, M2 : median filter and (G) : Gaussian filter).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure 9: ROAD plots of Vanilla Gradient and Integrated Gradient on FMNIST, CIFAR-10 and ImageNette models: naturally-trained (N) , adversarially trained (A) and adversarially trained with Gaussian filter (G) .

		N	A	G
VG	Gini $\uparrow$	0.45	0.55	0.55
	ROS $\downarrow$	2.34	2.04	2.01
IG	Gini $\uparrow$	0.61	0.64	0.63
	ROS $\downarrow$	4.23	4.34	4.03

Appendix

Table 7: Ablation: Sparsity and stability results for VGG16 on CIFAR-10 models: naturally trained (N) , adversarially trained (A) , & adversarially trained with Gaussian smoothing (G)

	Wilcoxon signed-rank			(Reported) ANOVA
	N vs A	A vs G	N vs G	F-stat	p-value
Sufficiency	9.79E-41	4.26E-14	3.71E-27	200.38	7.82E-72
Trust	5.56E-39	3.24E-11	3.89E-24	193.86	6.58E-70

Appendix

Table 8: Wilcoxon and ANOVA test results on the survey where, N refers to a naturally trained model, A refers to an adversarially-trained model, and G refers to an adversarial-trained feature-map smoothed model.

Figure 10: A sample of question from the survey

Figure 11: Final block sample images in the survey

Figure 12: Additional visualization for VG (FMNIST) (N: naturally-trained, A: adversarially-trained, M1: adversarially-trained with mean-filter, M2: adversarially-trained with median-filter, G: adversarially-trained with Gaussian-filter)

Figure 13: Additional visualization for VG (ImageNette) (N: naturally-trained, A: adversarially-trained, M1: adversarially-trained with mean-filter, M2: adversarially-trained with median-filter, G: adversarially-trained with Gaussian-filter, E: adversarially-trained with embedded filter, NG: adversarially-trained with non-local gaussian)

Figure 14: Additional visualization for VG (CIFAR-10) (N: naturally-trained, A: adversarially-trained, M1: adversarially-trained with mean-filter, M2: adversarially-trained with median-filter, G: adversarially-trained with Gaussian-filter, E: adversarially-trained with embedded filter, NG: adversarially-trained with non-local gaussian)

Gradient-Based Attribution

Gradient-Based Attribution uses model derivatives to estimate which inputs, features, parameters, or training examples influence predictions and behaviors.

See all 107 papers

Oct 7, 2026

Explaining the Saliency Map Sparsity of Adversarially-Trained Neural Networks

Understanding why deep neural networks make a given prediction is of great importance for their safe deployment. In computer vision, saliency maps, which highlight the image region most influential for a prediction, remain a widely-used form of explanation. An empirical observation is the apparent sparsity of gradient saliency maps of adversarially-trained neural networks. In this paper, we propose a theoretical explanation of this phenomenon for two-layer ReLU networks. We build on the established equivalence of adversarial training to the minimization of the empirical risk with weight-decay penalization and an added adversarial total variation term -- valid for certain loss functions. As the number of data points and neurons grows and the regularization parameters are sent to zero at appropriate rates, we prove that minimizers converge to a Bayes classifier with minimal gradient and Barron norm. Sparsity appears since for adversarial training with

\ell_\infty

-attacks the gradient norm is anisotropic and favors axis-aligned / sparse gradients. We illustrate our theoretical findings experimentally by evaluating the gradient

\ell_1

-norm and thresholded sparsity of naturally versus adversarially trained models.

Yannick Lunk, Atell Yehor Krasnopolsky, Damien Garreau +1

Institute of Mathematics University of Würzburg, Germany · Technical University of Munich, Germany · Institute of Computer Science, CAIDAS University of Würzburg, Germany +1

Adversarial Training Gradient-Based Attribution

Ask PDF

Apr 2, 2026

Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?

Post-hoc feature attribution methods are widely deployed in safety-critical vision systems, yet their stability under realistic input perturbations remains poorly characterized. Existing metrics evaluate explanations primarily under additive noise, collapse stability to a single scalar, and fail to condition on prediction preservation, conflating explanation fragility with model sensitivity. We introduce the Feature Attribution Stability Suite (FASS), a benchmark that enforces prediction-invariance filtering, decomposes stability into three complementary metrics: structural similarity, rank correlation, and top-k Jaccard overlap-and evaluates across geometric, photometric, and compression perturbations. Evaluating four attribution methods (Integrated Gradients, GradientSHAP, Grad-CAM, LIME) across four architectures and three datasets-ImageNet-1K, MS COCO, and CIFAR-10, FASS shows that stability estimates depend critically on perturbation family and prediction-invariance filtering. Geometric perturbations expose substantially greater attribution instability than photometric changes, and without conditioning on prediction preservation, up to 99% of evaluated pairs involve changed predictions. Under this controlled evaluation, we observe consistent method-level trends, with Grad-CAM achieving the highest stability across datasets.

Kamalasankari Subramaniakuppusamy, Jugal Gajjar

The George Washington University Washington D.C.

Explainability Evaluation Perturbation-Based Feature Attribution Saliency Map Evaluation

Ask PDF

May 16, 2026

AIM: Adversarial Information Masking for Faithfulness Evaluation of Saliency Maps

Post-hoc saliency methods are widely used to interpret deep neural networks, but their faithfulness is difficult to evaluate reliably. Existing evaluations mask features according to saliency-induced feature ordering and measure performance degradation, but this degradation can be confounded by the masking operator: zero masking may create out-of-distribution artifacts, while interpolation-based masking may preserve residual predictive information. We propose Adversarial Information Masking (AIM), a saliency-guided adversarial feature replacement framework for evaluating both saliency-map faithfulness and masking-operator reliability. AIM replaces selected features with values from an adversarial counterpart of the input and compares degradation under complementary masking orders. We assess reliability using random-attribution bias and stability of explanation-method faithfulness rankings. Experiments on image, audio, and EEG tasks suggest that AIM reduces masking-induced bias compared with zero and interpolation-based masking, while revealing modality-dependent differences between signed and unsigned attributions.

Chia-Ying Hsieh, Hsin-Yuan Fang, Chun-Shu Wei

National Yang Ming Chiao Tung University

Gradient-Based Attribution Saliency Map Evaluation Explainability Evaluation

Ask PDF

Abstract

Gradient-Based Attribution

Explore similar work

Explaining the Saliency Map Sparsity of Adversarially-Trained Neural Networks

Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?

AIM: Adversarial Information Masking for Faithfulness Evaluation of Saliency Maps