Transformer models achieve state-of-the-art performance across domains and tasks, yet their deeply layered representations make their predictions difficult to interpret. Existing explainability methods rely on final-layer attributions, capture either local token-level attributions or global attention patterns without unification, and lack context-awareness of inter-token dependencies and structural components. They also fail to capture how relevance evolves across layers and how structural components shape decision-making. To address these limitations, we proposed the Context-Aware Layer-wise Integrated Gradients (CA-LIG) Framework, a unified hierarchical attribution framework that computes layer-wise Integrated Gradients within each Transformer block and fuses these token-level attributions with class-specific attention gradients. This integration yields signed, context-sensitive attribution maps that capture supportive and opposing evidence while tracing the hierarchical flow of relevance through the Transformer layers. We evaluate the CA-LIG Framework across diverse tasks, domains, and transformer model families, including sentiment analysis and long and multi-class document classification with BERT, hate speech detection in a low-resource language setting with XLM-R and AfroLM, and image classification with Masked Autoencoder vision Transformer model. Across all tasks and architectures, CA-LIG provides more faithful attributions, shows stronger sensitivity to contextual dependencies, and produces clearer, more semantically coherent visualizations than established explainability methods. These results indicate that CA-LIG provides a more comprehensive, context-aware, and reliable explanation of Transformer decision-making, advancing both the practical interpretability and conceptual understanding of deep neural models.
Figures & tables
Figure 1: Proposed architecture of the Context-Aware Layer-wise Integrated Gradients (CA-LIG) framework.
Figure 2Figure 3
Figure 7: Token-F1 scores on the Movie Reviews reasoning task. CA-LIG and CA-IG(last layer) achieve consistently higher performance than baseline methods.
Figure 8: Layer-wise sensitivity profiles for CA-LIG, classifier contribution, and mean attention to [CLS] . Transitional peaks at Layers 4 and 8 reflect key representational shifts. The final peak at Layer 12 corresponds to sentiment consolidation.
Figure 9: Qualitative comparison of explanations generated by baseline XAI methods and CA-LIG using a MAE model. Warmer colors denote regions with higher positive relevance, while cooler colors indicate lower relevance. Images are taken from the ASIRRA dataset [ 18 ] .
Figure 10: Example of an explanation generated using CA-LIG for a prediction made by MAE model. (a) Original input image, (b) CA-LIG explanation heatmap, (c) positively attributed regions, and (d) negatively attributed regions. Warmer colors indicate stronger relevance.
Perturbation
Class
GradCAM [ 61 ]
LRP- ϵ [ 9 ]
LRP (Attn+Grad) [ 10 ]
IG [ 70 ]
Ours
Insertion ↑
Predicted
0.424
0.448
0.567
0.473
0.617
Deletion ↓
Predicted
0.374
0.437
0.237
0.346
0.215
Table 1: Perturbation-based AUC evaluation of explanation faithfulness. Higher insertion AUC (↑) indicates faster confidence recovery when important patches are added to a blurred baseline, while lower deletion AUC (↓) reflects faster confidence drop when critical patches are removed from the original image.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Aspect
IG [ 70 ]
Integrated Hessians [ 31 ]
Transformer-LRP [ 10 ]
CA-LIG Framework (Ours)
Core Principle
Path-integrated gradients from baseline to input
Second-order IG using Hessians for interaction effects
LRP/Deep Taylor rules
Layer-wise IG fused with class-specific attention gradients
Primary Objective
Token-level importance at input layer.
Quantify pairwise feature interactions.
Propagate and conserve relevance across attention and residuals.
Produce hierarchical, context-aware token relevance across layers.
Derivative Order
First-order
Second-order
Rule-based
First-order IG with gradient-weighted attention integration
Explicit LIG at each layer to model relevance trajectories.
Appendix
Table 2: Methodological comparison of IG, Integrated Hessians, Transformer-LRP, and the proposed CA-LIG framework, highlighting how CA-LIG combines layer-wise Integrated Gradients with attention-gradient fusion to produce hierarchical, context-aware explanations.
Figure 12: CA-LIG token-level attributions for a positive IMDB review using BERT-Large. Brighter green indicates stronger positive evidence, red indicates negative relevance, and white denotes neutral tokens.
Figure 13: CA-LIG token-level attributions for a document labeled politics class from the 20 Newsgroups dataset using BERT-base. Brighter green tokens provide stronger positive evidence, lighter green indicates weaker support, red shows negative influence, and white denotes neutral relevance.
Figure 14: Token-level attribution visualization generated by the CA-LIG using the AfroLM model on an Amharic hate speech sample. Brighter Red tokens provide stronger negative evidence (hate); lighter red indicates weaker support; green shows positive influence; and white denotes neutral relevance. English translation: That religion is not respected, all its followers are destructive, and their behavior shows that they do not like peace anywhere.
Figure 15: Token-level attribution visualization generated by the CA-LIG using the XLM-R model on an Amharic hate speech sample. Brighter green tokens provide stronger positive evidence (not hate), lighter green indicates weaker support, red shows negative influence, and white denotes neutral relevance. English translation: It is important to believe in the rule of law. Anyone who breaks the law must be held accountable.
Figure 16: Example of a CA-LIG explanation for a prediction made by the MAE model on the CIFAR-10 dataset. (a) Original input image, (b) CA-LIG explanation heatmap, (c) positively attributed regions, and (d) negatively attributed regions. Warmer colors indicate stronger relevance to the model’s prediction. The CA-LIG explanation indicates that the MAE model’s predictions are driven by contextually related and semantically meaningful object parts. As illustrated in (b) and (c), strong positive relevance is consistently assigned to the head, torso, and lower body/feet. In contrast, (d) indicates negative relevance to background and peripheral regions that do not contribute to the prediction.
Figure 17: Qualitative comparison of explanations generated by various XAI methods using a Masked Autoencoder (MAE) fine-tuned on the CIFAR-10 dataset for image classification. Warmer colors denote regions with higher positive relevance to the predicted class, while cooler colors indicate lower relevance. Sample images are taken from the ASIRRA cat vs. dog dataset [ 18 ] .
Figure 18: Layer-wise token relevance visualization generated by our proposed CA-IG method using a BERT-base model fine-tuned on IMDB sentiment analysis. Brighter green indicates stronger positive evidence, red indicates negative relevance, and white denotes neutral tokens. The progression shows early layers distributing relevance across tokens, middle layers emphasizing sentiment-bearing words ( absolutely , amazing ), and deeper layers consolidating decision-relevant cues.
Transformers achieve strong performance, but their internal computations remain opaque. We view each Transformer layer as a dynamic graph whose nodes are token representations and per-head attention outputs, with Multi-Head Attention (ATT) and MLP as module boundaries. On this graph we use LIG (Layer-wise Integrated Gradients), which applies set-to-set Integrated Gradients (IG) at nonlinear module boundaries. Set-to-set IG applies IG to a map from a set of input token representations to a set of output representations, evaluating token-to-token contributions, which is not standard in prior IG applications. This extends IG from the usual scalar-objective setting to set-to-set maps via an L2 scalarization, and composes within-layer contributions in the spirit of Layer-wise Relevance Propagation (LRP), with IG completeness playing the role of LRP-style conservation at each boundary. We use LIG to analyze (i) the agreement between module-wise composition and layer-whole attribution under an L2 criterion, and (ii) within-layer information flow by tracing separated ATT and MLP contributions. On BERT-base and PTB, configurations that best preserved within-layer consistency used the target token's embedding as the ATT baseline and either the ATT output at a=0 or Zero as the MLP baseline. We therefore present LIG as a diagnostic XAI tool at module-boundary granularity, without model-specific retraining or per-operation interpreter design. Code is available at https://github.com/eightsuzuki/layer-wise-integrated-gradients.
Eight Suzuki, Hideitsu Hino, Noboru Murata
Waseda University · Fujitsu Limited · The Institute of Statistical Mathematics
Vision Transformers (ViTs) have become a dominant architecture in computer vision, yet their prediction process remains difficult to interpret because information is propagated through complex interactions across layers and attention heads. Existing attention based explanation methods provide an intuitive way to trace information flow. However, they rely mainly on raw attention weights, which do not explicitly reflect the final decision and often lead to explanations with limited class discriminability. In contrast, gradient based localization methods are more effective at highlighting class specific evidence, but they do not fully exploit the hierarchical attention propagation mechanism of transformers. To address this limitation, we propose Decision-Aware Attention Propagation (DAP), an attribution method that injects decision-relevant priors into transformer attention propagation. By estimating token importance through gradient based localization and integrating it into layer wise attention rollout, the method captures both the structural flow of attention and the evidence most relevant to the final prediction. Consequently, DAP produces attribution maps that are more class sensitive, compact, and faithful than those generated by conventional attention based methods. Extensive experiments across Vision Transformer variants of different model scales show that DAP consistently outperforms existing baselines in both quantitative metrics and qualitative visualizations, indicating that decision aware propagation is an effective direction for improving ViT interpretability.
Sehyeong Jo, Gangjae Jang, Haesol Park
Aarhus University · University of Colorado Boulder · KIST
We ask a simple question about decoder-only transformers: between which two layers is the probability of a predicted token actually produced? Existing layer-wise readout tools answer only approximately. The logit lens and its trained variant report a per-layer level of probability but give no additive decomposition; their estimates are biased and non-monotone across depth. Direct Logit Attribution and related residual-stream methods are additive, but only in logit space, the softmax nonlinearity breaks additivity in probability space, precisely the quantity one usually cares about. Layer Conductance integrates gradients per layer, but attributes each to its own baseline and so does not sum to the total change in prediction. We introduce IG-Lens, a telescoping application of Integrated Gradients along a single path through the hidden states from a baseline to the final layer. Crediting each segment to the layer it terminates at yields a layer-wise attribution whose sum is exactly the change in target probability, with the softmax inside the integration path rather than linearized away. Our default estimator credits each integration step its observed change in target probability (a prediction-aware reweighting in the spirit of IDGI) rather than its raw gradient. Because the readout is a one-dimensional probability, this collapses each segment to a telescoping sum of endpoint values, so completeness holds exactly (to floating point) at any step count, removing Riemann discretization error while suppressing steps that show gradient sensitivity without a change in output. We give the telescoping identity and its proof, verify completeness to floating point, and describe a single-pass batched implementation computing the full token-by-layer map without any backward call. Code: https://github.com/anhnda/IGLens.