Transformers are widely assumed to exhibit token-decoding explainability. In particular, in vision transformer (ViT) classifiers, decoding individual patch tokens has become a common diagnostic tool due to its tendency to yield human-interpretable outputs. In light of the increasing popularity of this approach, an important question is whether token-decoding explainability is an inherent property of transformers that should be expected to hold across architectures, scales, and training objectives, or merely an artifact of some implicit bias. In this paper we systematically study token-decoding explainability across diverse ViT settings, spanning architectures, scales, and training objectives, using both controlled experiments and real-world models. We find that while some models indeed exhibit aligned and interpretable token decodings, others exhibit either inverted explainability maps or a complete lack of correlation with the image semantics. As we show, these tendencies toward different explainability behaviors are not intrinsic to the architecture and can vary arbitrarily with changes in the optimization landscape. We provide theoretical support for these observations by demonstrating a setting where a single fixed toy transformer architecture can provably implement the optimal classification rule in multiple ways, which differ in their patch-decoding explainability patterns. Our results show that token-decoding explainability is not an inherent property of trained transformers, but rather an outcome contingent on the problem setting and thus should not be a-priori assumed to hold. Code can be found in https://ronensc.github.io/Explainability-Illusion/ .
Figures & tables
Figure 1: Explainability regimes. For an input image, token-decoding of patches produces explainability maps, where each patch-token receives some score S . The relation between the average score of object tokens, Sobj (tokens inside the black contour), and the average score of background tokens, Sbg (tokens outside the black contour), determine the explainability of a classified input image in a layer: \mathrel{\hbox to5.17pt{\vbox to5.17pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#14E01B} \lxSVG@begingroup@{fill=#14E01B} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} {}{{}}{} {{\lx@inpgf@ignorespaces}{}}{} {\lx@inpgf@ignorespaces}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 0 L 7.15 0 L 7.15 7.15 Z}{stroke:none} \lx@inpgf@ignorespaces \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}} aligned, \mathrel{\hbox to5.17pt{\vbox to5.17pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#8C24FF} \lxSVG@begingroup@{fill=#8C24FF} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} {{\lx@inpgf@ignorespaces}{}}{{}}{} {\lx@inpgf@ignorespaces}{} {\lx@inpgf@ignorespaces}{} {\lx@inpgf@ignorespaces}{} {{\lx@inpgf@ignorespaces}{}}{} {}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 2.98 L 4.17 7.15 L 7.15 7.15 L 7.15 4.17 L 2.98 0 L 0 0 Z}{stroke:none} \lx@inpgf@ignorespaces \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}} unexplainable, or \mathrel{\hbox to5.17pt{\vbox to5.17pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#0080FF} \lxSVG@begingroup@{fill=#0080FF} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} {}{{}}{} {{\lx@inpgf@ignorespaces}{}}{} {\lx@inpgf@ignorespaces}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 0 L 0 7.15 L 7.15 7.15 Z}{stroke:none} \lx@inpgf@ignorespaces \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}} inverted. Here, the elephant is aligned and the cat is inverted.
Figure 2: Aligned-explainability bias of real-world ViT classifiers. 2D histograms of Sbg and Sobj scores computed over all of PartImageNet++ (right) and explainability maps of six image inputs (left), on ViT-B/16, ViT-L/16, and DINOv2. All classifiers exhibit an aligned-explainability bias.
Figure 3: Evolution of aligned-explainability in ViT-B/16. 2D histograms of Sbg and Sobj scores computed over all of PartImageNet++ as they evolve through the model’s 12 layers. While the early layers are unexplainable, as the features evolve they are more aligned.
Table 1: Explainability of real-world ViT classifiers and contrastive VLMs. Scores are computed over all of PartImageNet++. Explainability regimes are marked according to Sec. 3.1 , where dashed rules mark the boundary between them. Contrastive VLMs exhibit a clear bias toward inverted-explainability, whereas classifiers are largely aligned-explainable. OpenCLIP’s parentheses indicate dataset size and number of samples seen, and “+reg” marks models trained with register tokens.
Figure 4: Different subsets can lie in different regimes. 2D histograms of Sbg and Sobj scores computed separately for animal and non-animal classes in PartImageNet++ on DINOv2-L/14. In the penultimate layer, animals have inverted-explainability and non-animals are unexplainable. In the last layer, animals are unexplainable and non-animals have aligned-explainability.
Figure 5: Inverted-explainability bias of real-world ViT-based contrastive VLMs. 2D histograms of Sbg and Sobj scores calculated over PartImageNet++ (right), as well as explainability maps of six image inputs (left), on three models: CLIP-B/16 trained by OpenCLIP on LAION, CLIP-L/14, and MetaCLIP. All contrastive VLMs exhibit an inverted explainability bias.
Figure 6: Explainability trends of controlled ViT experiments. Violin plots of A scores of 99 classifiers with 10 (left) or 20 (middle) classes, and 232 contrastive models (right), trained on our synthetic datasets, for different OBNR values. The violins are colored gradually according to their medians (indicated with black dots), with blue, green, and purple denoting inverted-explainability, aligned-explainability and unexplainability, respectively. The black lines connect the medians, the dashed rules mark A=50% , and the gray background indicates the unexplainability regime. In ViTs, low or high OBNR values induce aligned or inverted explainability biases, respectively, while OBNR≈1 promote unexplainability. In contrastive VLMs, low and high OBNR values promote inverted explainability, while OBNR≈1 yields aligned-explainability. Despite these biases, ViTs occasionally converge outside the dominant regime for the same data distribution.
Figure 7: The toy transformer model. Each token in the 1D z image input is embedded using ϕ . After prepending a class token, a single self-attention block with a residual connection produces representations. The classification head g is applied to the class token to output predictions.
\mathrel{\hbox to5.17pt{\vbox to5.17pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#8C24FF} \lxSVG@begingroup@{fill=#8C24FF} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} {{\lx@inpgf@ignorespaces}{}}{{}}{} {\lx@inpgf@ignorespaces}{} {\lx@inpgf@ignorespaces}{} {\lx@inpgf@ignorespaces}{} {{\lx@inpgf@ignorespaces}{}}{} {}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 2.98 L 4.17 7.15 L 7.15 7.15 L 7.15 4.17 L 2.98 0 L 0 0 Z}{stroke:none} \lx@inpgf@ignorespaces \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}}
\mathrel{\hbox to5.17pt{\vbox to5.17pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#8C24FF} \lxSVG@begingroup@{fill=#8C24FF} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} {{\lx@inpgf@ignorespaces}{}}{{}}{} {\lx@inpgf@ignorespaces}{} {\lx@inpgf@ignorespaces}{} {\lx@inpgf@ignorespaces}{} {{\lx@inpgf@ignorespaces}{}}{} {}{} {\lx@inpgf@ignorespaces}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 2.98 L 4.17 7.15 L 7.15 7.15 L 7.15 4.17 L 2.98 0 L 0 0 Z}{stroke:none} \lx@inpgf@ignorespaces \lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}}
Appendix
Table S1: Comparing the official classification heads of DINOv2 with a class-token head. We compare the original DINOv2 heads used in Tab. 1 (“Official”) with new trained versions of these heads (“CLS-only”). Specifically, the original heads expect as input a patch-token concatenated to the mean value of all patch-tokens, and the new heads are trained on a single input of the class-token. In both, we apply the token-decoding approach of replacing the class-token with patch-tokens of the final layer. Top-1 accuracy is computed over the ImageNet-1k validation set, and A over PartImageNet++ at the last layer. Explainability regimes are marked according to Sec. 3.1 . Both heads yield the same regime for every model.
Name
timm identifier
ViT-Ti/16
vit_tiny_patch16_224.augreg_in21k_ft_in1k
ViT-S/16
vit_small_patch16_224.augreg_in21k_ft_in1k
ViT-B/16
vit_base_patch16_224.augreg_in21k_ft_in1k
ViT-L/16
vit_large_patch16_224.augreg_in21k_ft_in1k
ViT-B/16 (orig)
vit_base_patch16_224.orig_in21k_ft_in1k
ViT-B/16 (AugReg2)
vit_base_patch16_224.augreg2_in21k_ft_in1k
Appendix
Table S2: timm checkpoints. Names used in Tabs. 1 and S4 and the corresponding timm ( Wightman, 2019 ) model identifiers ( timm v1.0.22). † The original ViT-L/16 weights ( Dosovitskiy et al., 2021 ) have no identifier in v1.0.22, so we use the file jx_vit_large_p16_224-4ee7a4dc.pth distributed by early timm versions.
Name
Model
Pretrained tag
CLIP-B/32
ViT-B-32
openai
CLIP-B/16
ViT-B-16
openai
CLIP-L/14
ViT-L-14
openai
MetaCLIP-B/32
ViT-B-32
metaclip_fullcc
MetaCLIP-B/16
ViT-B-16
metaclip_fullcc
MetaCLIP-L/14
ViT-L-14
metaclip_fullcc
Appendix
Table S3: OpenCLIP checkpoints. Names used in Tab. 1 and the corresponding OpenCLIP ( Ilharco et al., 2021 ) model names and pretrained tags.
Figure S1: Aligned-explainability bias of real-world ViT-based classifiers. We plot 2D histograms of Sbg and Sobj scores calculated over all of PartImageNet++ ( Li et al., 2024a ) for 25 ViT classifiers of different architectures. ViT classifiers exhibit a strong bias toward aligned-explainability. Some models still converge to the unexplainability or even the inverted-explainability regime, irrespective of accuracy or scale.
Table S4: Top-1 accuracy of real-world ViT classifiers across explainability regimes. Scores and accuracy values are computed over PartImageNet++ ( Li et al., 2024a ) . Explainability regimes are marked according to Sec. 3.1 . Dashed rules mark the boundary between explainability regimes. The aligned-explainability bias of ViT classifiers is unrelated to classification accuracy, as all models achieve high top-1 accuracy values.
Model
S(⋅) Approach
Mean Sobj
Mean Sbg
A (%)
Mean Norm (∥xobj∥/∥xbg∥)
ViT-B/16 (SAM)
Logits
−10.07
−8.33
6.4%
10.99/10.77
Cosine Similarity
−0.35
−0.30
8.4%
ViT-B/32 (SAM)
Logits
−10.09
−8.87
14.4%
13.22/13.16
Cosine Similarity
−0.33
−0.29
15.2%
DeiT-Ti/16
Logits
−2.99
−0.13
4.1%
25.88/24.44
Cosine Similarity
−0.17
−0.002
4.2%
Appendix
Table S5: Effect of activation norm on inverted ViT-based classifiers. The mean activation norms for both object and background patch tokens are very similar, which means our S computation approach of directly using the logits leads to similar results as using normalized cosine similarity decoding. In particular, both exhibit the same inverted-explainability, which rules out norm-scaling artifacts.
Samples seen
Arch.
Data
3B
13B
34B
B-32
80M
9.7†
8.3†
9.1
400M
9.4
5.7
6.2†
2B
11.3
3.4†
6.3
B-16
80M
13.8
8.5
7.0
400M
11.5
3.3†
3.5
Appendix
Table S6: Inverted-explainability across the CLIP scaling grid. We compute A scores for all models of Cherti et al. (2023) over all of PartImageNet++ ( Li et al., 2024a ) . The models vary in architecture size and training-set size (rows) and in the number of samples seen during training (columns), which is equivalent to varying training epochs. † marks the seven runs trained at 5×10−4 rather than the 10−3 used by the other 22 runs, and with a smaller batch-size. All 29 models exhibit inverted-explainability. Moreover, for a specific architecture, larger training sets and longer training promote a stronger inverted-explainability bias.
Figure S2: Inverted-explainability bias of real-world contrastive VLMs. We plot 2D histograms of Sbg and Sobj scores calculated over all of PartImageNet++ ( Li et al., 2024a ) for 41 contrastive VLMs of different architectures. Contrastive VLMs exhibit a clear bias toward inverted-explainability, irrespective of accuracy or scale.
Figure S3: Evolution of inverted explainability in CLIP. We plot 2D histograms of Sbg and Sobj scores calculated over all of PartImageNet++ ( Li et al., 2024a ) as they evolve through the model’s 12 layers. While the early layers are unexplainable, as the features evolve they exhibit more inversion.
Table S7: Explainability at the penultimate and last layers. A is computed over all of PartImageNet++ at the penultimate layer (Penult.), which we use throughout the paper, and at the last layer (Last), whose patch tokens are not affected by the loss. Explainability regimes are marked according to Sec. 3.1 . Each model is in the same regime at both layers. Dashed rules mark the boundary between regimes. DINOv2 is excluded, as its head reads the last-layer patch tokens, and thus our results in Tab. 1 already use its last layer decoding.
Figure S4: Different training seeds of the exact same configuration exhibiting different explainability regimes. 2D histograms of Sbg and Sobj scores of 6 different contrastive configurations, spanning different αx,αy,αn (and thus different OBNRs), trained over our synthetic dataset. While all configurations induce a bias toward some explainability regime, as seen in Fig. 6 , different training seeds of the exact same data distribution can still yield very different explainability regimes. For example, when αn=0.8 then OBNR≈1 and the contrastive models have a bias toward aligned-explainability, yet one of the three training seeds of Fig. 4(c) leads to inverted-explainability. Additionally, when αn=0.2 the OBNR is high and the natural bias is toward inverted-explainability, however one of the seeds in Fig. 4(b) cause the model to converge to aligned-explainability. In some configurations, such as in Fig. 4(a) , the three seeds even led to the three possible regimes: inverted-explainability, aligned-explainability, and unexplainability. Indeed, explainability is not an inherent property of transformers, but an outcome contigent on the problem setting and optimization.
Figure S5: Different training seeds of the exact same configuration exhibiting different explainability regimes. 2D histograms of Sbg and Sobj scores of 2 different ViT-classifier configurations, spanning different C,γ,αn (and thus different OBNRs), trained over our synthetic dataset. While a configuration induces a bias toward some explainability regime, as seen in Fig. 6 , different training seeds of the exact same data distribution can still yield very different explainability patterns. For example, in Fig. 5(a) , as the OBNR is low the bias is toward aligned-explainability ( Fig. 6 ). However, one seed exhibits at least partial inverted-explainability. Additionally, in Fig. 5(b) we can see some seeds exhibit different explainability for different classes of the model, similar to the phenomenon in DINOv2-L/14 ( Fig. 4 ). Here, the model might cluster classes, and converge to have aligned or inverted explainability depending on the classes cluster, similarly to the animals and non-animals clusters of DINOv2. This reinforces how explainability is not a static property of transformers.
Figure S6: Explainability trends of additional controlled ViT experiments. Violin plots of A scores of 66 classifiers with 10 (left) or 20 (middle) classes, and 144 contrastive models (right), trained on our synthetic datasets, for different OBNR values. The ViT classifiers are trained on a synthetic dataset variation where A is sampled differently per class. The contrastive models are trained using ViTs with 6 layers. Both make the task harder for the ViT to accomplish. The violins are colored gradually according to their medians (indicated with black dots), with blue, green, and purple denoting inverted-explainability, aligned-explainability and unexplainability, respectively. The black lines connect the medians, the dashed rules mark A=50% , and the gray background indicates the unexplainability regime. The trends of explainability biases seen in Fig. 6 are present here as well. Specifically, in ViTs, low or high OBNR values induce aligned or inverted explainability biases, respectively, while OBNR≈1 promotes unexplainability. Contrastive VLMs instead converge to inverted-explainability at both extremes, and to aligned-explainability at OBNR≈1 . Despite these biases, ViTs occasionally converge outside the dominant regime for the same data distribution.
Figure S7: Training loss plots of the contrastive VLMs from Fig. 6 , for αn∈[0.1,0.5] . All reported VLMs converge similarly when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S8: Training loss plots of the contrastive VLMs from Fig. 6 , for αn∈[0.65,4.0] . All reported VLMs converge similarly when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S9: Text-to-image retrieval recall at 5 (R@5) plots of the contrastive VLMs from Fig. 6 , for αn∈[0.1,0.5] , during training. All reported VLMs converge to perform similarly when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S10: Text-to-image retrieval recall at 5 (R@5) plots of the contrastive VLMs from Fig. 6 , for αn∈[0.65,4.0] , during training. All reported VLMs converge to perform similarly when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S11: Training loss plots of the ViT classifiers with 10 classes from Fig. 6 , for αn∈[0.05,2.0] . All reported ViT classifiers converge similarly when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S12: Training loss plots of the ViT classifiers with 10 classes from Fig. 6 , for αn∈[4,100] . All reported ViT classifiers converge similarly when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S13: Training loss plots of the ViT classifiers with 20 classes from Fig. 6 , for αn∈[0.05,2.0] . All reported ViT classifiers converge similarly when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S14: Training loss plots of the ViT classifiers with 20 classes from Fig. 6 , for αn∈[4,100] . All reported ViT classifiers converge similarly when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S15: Top-1 accuracy plots of the ViT classifiers with 10 classes from Fig. 6 , for αn∈[0.05,2.0] , during training. All reported ViT classifiers converge similarly to high accuracy when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S16: Training loss plots of the ViT classifiers with 10 classes from Fig. 6 , for αn∈[4,100] , during training. All reported ViT classifiers converge similarly to high accuracy when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S17: Training loss plots of the ViT classifiers with 20 classes from Fig. 6 , for αn∈[0.05,2.0] , during training. All reported ViT classifiers converge similarly to high accuracy when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Figure S18: Training loss plots of the ViT classifiers with 20 classes from Fig. 6 , for αn∈[4,100] , during training. All reported ViT classifiers converge similarly to high accuracy when trained on the same task. Therefore, the trends of explainability biases we see are not a product of some failure of the model, but rather an implicit bias of the problem setting.
Transformer-based architectures have become the shared backbone of natural language processing and computer vision. However, understanding how these models operate remains challenging, particularly in vision settings, where images are processed as sequences of patch tokens. Existing interpretability tools often focus on isolated components or expert-oriented analysis, leaving a gap in guided, end-to-end understanding of the full inference pipeline. To bridge this gap, we present ViT-Explainer, a web-based interactive system that provides an integrated visualization of Vision Transformer inference, from patch tokenization to final classification. The system combines animated walkthroughs, patch-level attention overlays, and a vision-adapted Logit Lens within both guided and free exploration modes. A user study with six participants suggests that ViT-Explainer is easy to learn and use, helping users interpret and understand Vision Transformer behavior.
Juan Manuel Hernandez, Mariana Fernandez-Espinosa, Denis Parra +1
Pontificia Universidad Católica de Chile · University of Notre Dame
How do vision transformers (ViTs) represent and process the world? This paper addresses this long-standing question through the first systematic analysis of 6.6K features across all layers, extracted via sparse autoencoders, and by introducing the residual replacement model, which replaces ViT computations with interpretable features in the residual stream. Our analysis reveals not only a feature evolution from low-level patterns to high-level semantics, but also how ViTs encode curves and spatial positions through specialized feature types. The residual replacement model scalably produces a faithful yet parsimonious circuit for human-scale interpretability by significantly simplifying the original computations. As a result, this framework enables intuitive understanding of ViT mechanisms. Finally, we demonstrate the utility of our framework in debiasing spurious correlations.
Vision Transformers (ViTs) have become a dominant architecture in computer vision, yet their prediction process remains difficult to interpret because information is propagated through complex interactions across layers and attention heads. Existing attention based explanation methods provide an intuitive way to trace information flow. However, they rely mainly on raw attention weights, which do not explicitly reflect the final decision and often lead to explanations with limited class discriminability. In contrast, gradient based localization methods are more effective at highlighting class specific evidence, but they do not fully exploit the hierarchical attention propagation mechanism of transformers. To address this limitation, we propose Decision-Aware Attention Propagation (DAP), an attribution method that injects decision-relevant priors into transformer attention propagation. By estimating token importance through gradient based localization and integrating it into layer wise attention rollout, the method captures both the structural flow of attention and the evidence most relevant to the final prediction. Consequently, DAP produces attribution maps that are more class sensitive, compact, and faithful than those generated by conventional attention based methods. Extensive experiments across Vision Transformer variants of different model scales show that DAP consistently outperforms existing baselines in both quantitative metrics and qualitative visualizations, indicating that decision aware propagation is an effective direction for improving ViT interpretability.
Sehyeong Jo, Gangjae Jang, Haesol Park
Aarhus University · University of Colorado Boulder · KIST