Frequency-Aware Model Parameter Explorer: A new attribution method for improving explainability

Ali Yavari; Alireza Mohamadi; Elham Beydaghi; Philipp Seeböck; Rainer A. Leitgeb

cs.LGSep 25, 2025

Frequency-Aware Model Parameter Explorer: A new attribution method for improving explainability

Authors: Ali Yavari, Alireza Mohamadi, Elham Beydaghi, Philipp Seeböck, Rainer A. Leitgeb

Organizations: Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria · Independent · Dept. of Computer Engineering, Sharif University of Technology, Tehran, Iran · MANO Group, CIR Lab, Dept. of Biomedical Imaging and Image-Guided Therapy, Comprehensive Center for AI in Medicine, Medical University of Vienna, Vienna, Austria

Abstract

State-of-the-art attribution methods such as AttEXplore rely on adversarial sample generation that applies an all-pass filter across the frequency spectrum: they perturb all frequencies of the input image alike, from the low frequencies that carry coarse shape to the high frequencies that carry edges and textures. However, models rely on low and high frequencies to different degrees, and such an all-pass perturbation cannot adapt to the model being explained, which limits the faithfulness of the resulting attribution maps. We propose FAMPE (Frequency-Aware Model Parameter Explorer), an attribution method that overcomes this limitation by perturbing the low- and high-frequency bands of the input independently, so that the perturbation can be shifted toward the band that matters for the model. The boundary between the two bands is set for each image automatically from its spectral energy. On ImageNet with five convolutional neural network and Vision Transformer architectures, FAMPE significantly improves the Insertion Score over the strongest baseline, AttEXplore, on four of them, by up to 15.4%, at essentially the same computational cost. How the perturbed frequency bands are weighted matters: the best and the worst balance between them differ by 19-36% in Insertion Score, and the best balance depends on the architecture. Choosing the balance for each image with a probe that needs no ground truth raises the Insertion Score significantly further on all five architectures, reaching up to 24.4% above AttEXplore. Controlling which frequency band is perturbed is therefore a simple, low-cost way to improve attribution faithfulness.

Figures & tables

Figure 1: Comparison between frequency-based SOTA attribution methods and our proposed approach. Top: Existing methods such as AttEXplore apply a single all-pass multiplicative perturbation to the whole spectrum, so every frequency band is perturbed alike and no band can be weighted separately. Bottom: FAMPE splits the spectrum at an energy-defined cutoff and perturbs the low-frequency band (capturing the object’s overall shape, e.g., the dog’s silhouette) and the high-frequency band (capturing edges and fine details) with two independently sampled fields weighted by $\alpha$ and $1-\alpha$ . We use a centered Fourier spectrum because the cutoff is defined as a radius from the DC component; the same construction could be built on a DCT. In this example the resulting map is more closely aligned with the object; the population-level comparison is given in Section 4.2 .

Figure 2: Overview of FAMPE’s frequency-aware adversarial sample generation. Stage 1: input image (a) is transformed via FFT to obtain the centered Fourier spectrum (b). Stage 2: the spectrum is independently modulated by a low-frequency Gaussian mask scaled by $\alpha$ (c, top) and a high-frequency Gaussian mask scaled by $1-\alpha$ (d, bottom), each combined with multiplicative Gaussian noise $\mathcal{N}(1,\sigma)$ . Stage 3: modulated spectra are summed and mapped back via IFFT, yielding the reconstructed perturbed image (e). The parameter $\alpha$ controls the relative perturbation per spectral band.

Method	Inception-v3		ResNet-50		VGG-16		MaxViT-T		ViT-B/16		Mean
Method	Ins.	Del.	Ins.	Del.	Ins.	Del.	Ins.	Del.	Ins.	Del.	Ins.	Del.
IG	0.1990	0.1650	0.1853	0.1402	0.1360	0.1123	0.3342	0.2893	0.2547	0.2197	0.2219	0.1853
DeepLIFT	0.1975	0.1769	0.1820	0.1637	0.1415	0.1170	0.3360	0.2890	0.2560	0.2200	0.2226	0.1933
GIG	0.2076	0.1483	0.1912	0.1202	0.1409	0.0916	0.3506	0.2606	0.2544	0.1967	0.2289	0.1635
AGI	0.3839	0.1169	0.3100	0.0975	0.2487	0.0774	0.2928	0.3011	0.4350	0.1861	0.3341	0.1558
AttEXplore	0.5316	0.1027	0.3448	0.1068	0.2795	0.0899	0.3829	0.2488	0.4766	0.1572	0.4031	0.1411

Table 1: Performance of our method, FAMPE, compared to five competing approaches across five deep learning architectures (n=1000 images per cell). Evaluation uses Insertion Score (

\uparrow

better) and Deletion Score (

\downarrow

better); bold marks the best score in each column, excluding the oracle upper bound.

\alpha^{*}

is the best fixed

\alpha

per architecture: 0.4 (Inception-v3), 0.7 (ResNet-50), 0.8 (VGG-16), 0.1 (MaxViT-T), 0.8 (ViT-B/16). Selected

\alpha

is a deployable per-image choice made by a cheap 2-step/5-probe insertion probe with no access to ground truth, at

2.87\times

the cost of AttEXplore; oracle

\alpha

is a non-deployable upper bound — the per-image maximum Insertion and minimum Deletion achievable in the full 11-point sweep , each chosen independently, so the two may come from different

\alpha

for the same image; unobtainable without the full evaluation score itself — reported only to bound the remaining headroom. FAMPE (all-pass) is an ablation with the frequency split disabled (

\alpha=1

, low-pass mask forced to all-ones) and reproduces AttEXplore on every architecture and both metrics, indicating that the gains are not explained by other differences between the two implementations.

Figure 3: Qualitative comparison of attribution maps from FAMPE, AGI, and AttEXplore on three ImageNet samples of varying spectral complexity: (A) a strawberry on MaxViT-T (low-frequency content, small $c_{f}$ ), (B) an African hunting dog on MaxViT-T (mixed mid- to high-frequency content, intermediate $c_{f}$ ), and (C) a ruddy turnstone on Inception-v3 (high-frequency content, large $c_{f}$ ). For each panel, AGI and AttEXplore maps are followed by 11 FAMPE maps with $\alpha$ from 0 (purely high-frequency perturbation) to 1 (purely low-frequency perturbation). On these two architectures ( $\alpha^{*}=0.1$ and $0.4$ ), lower- $\alpha$ FAMPE maps place mass more tightly on the target object than the baselines.

Figure 4: Joint distribution of the per-image cutoff $c_{f}$ and the best- $\alpha$ value (maximizing Insertion Score) across 1,000 ImageNet samples on MaxViT-T; $c_{f}$ is recomputed with the estimator FAMPE uses at attribution time, on the preprocessed image at the network’s input size. The central heatmap shows image counts per $(c_{f},\alpha)$ bin (darker = higher density), with marginal histograms of cutoff radii (top) and best- $\alpha$ values (right). Both marginals are skewed — most images have a small $c_{f}$ , and most are best served by a low $\alpha$ — but the two are close to independent (upper-left quadrant: 410 images observed vs. 429 expected; Cramér’s $V=0.12$ ).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 1 — Figure 1: Attribution maps for low-frequency images (small $c_{f}$ ). Each panel shows the input image, the AGI and AttEXplore maps, and the FAMPE maps for $\alpha=0,0.1,\ldots,1$ .

Panel of Figure 2 — Figure 2: Attribution maps for middle-frequency images (intermediate $c_{f}$ ). Layout as in Fig. 1 .

Panel of Figure 3 — Figure 3: Attribution maps for high-frequency images (large $c_{f}$ ), part 1. Layout as in Fig. 1 .

Panel of Figure 4 — Figure 4: Attribution maps for high-frequency images (large $c_{f}$ ), part 2. Layout as in Fig. 1 .

Feature Attribution

Feature attribution estimates which input features or internal components contribute to a model’s predictions.

See all 231 papers

Learn to Rank: Visual Attribution by Learning Importance Ranking
Apr 7, 2026David Schinagl, Christian Fruhwirth-Reisinger, Alexander Prutsch +2Feature Attribution Explainable Image Classification
Institute of Visual Computing (IVC), Graz University of Technology, Graz, Austria · Nomadic AI, USA · Amazon, USA
Attribution Upsampling should Redistribute, Not Interpolate
Mar 17, 2026Vincenzo Buono, Peyman Sheikholharam Mashhadi, Mahmoud Rahat +2Feature Attribution Saliency Map Evaluation
Halmstad University, Halmstad, Sweden
Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?
Apr 2, 2026Kamalasankari Subramaniakuppusamy, Jugal GajjarExplainability Evaluation Perturbation-Based Feature Attribution
The George Washington University Washington D.C.

Apr 7, 2026J/K move · Enter open · S save

Learn to Rank: Visual Attribution by Learning Importance Ranking

David Schinagl, Christian Fruhwirth-Reisinger, Alexander Prutsch, Samuel Schulter+1

Institute of Visual Computing (IVC), Graz University of Technology, Graz, Austria · Nomadic AI, USA · Amazon, USA

Feature Attribution Explainable Image Classification Transformer Interpretability

Open paper AskPDF arXiv

Interpreting the decisions of complex computer vision models is crucial to establish trust and accountability, especially in safety-critical domains. An established approach to interpretability is generating visual attribution maps that highlight regions of the input most relevant to the model's prediction. However, existing methods face a three-way trade-off. Propagation-based approaches are efficient, but they can be biased and architecture-specific. Meanwhile, perturbation-based methods are causally grounded, yet they are expensive and for vision transformers often yield coarse, patch-level explanations. Learning-based explainers are fast but usually optimize surrogate objectives or distill from heuristic teachers. We propose a learning scheme that instead optimizes deletion and insertion metrics directly. Since these metrics depend on non-differentiable sorting and ranking, we frame them as permutation learning and replace the hard sorting with a differentiable relaxation using Gumbel-Sinkhorn. This enables end-to-end training through attribution-guided perturbations of the target model. During inference, our method produces dense, pixel-level attributions in a single forward pass with optional, few-step gradient refinement. Our experiments demonstrate consistent quantitative improvements and sharper, boundary-aligned explanations, particularly for transformer-based vision models. Code and pretrained models are available at https://github.com/dschinagl/AHA.

Abstract

Feature Attribution

Explore similar work

Learn to Rank: Visual Attribution by Learning Importance Ranking

Attribution Upsampling should Redistribute, Not Interpolate

Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?