Evaluating Explainable AI Attribution Methods in Neural Machine Translation via Attention-Guided Knowledge Distillation

Aria Nourbakhsh; Salima Lamsiyah; Adelaide Danilov; Christoph Schommer

cs.CLMar 11, 2026

Evaluating Explainable AI Attribution Methods in Neural Machine Translation via Attention-Guided Knowledge Distillation

Authors: Aria Nourbakhsh, Salima Lamsiyah, Adelaide Danilov, Christoph Schommer

Organizations: Department of Computer Science, University of Luxembourg, 4364 Esch-sur-Alzette, Luxembourg

Abstract

The study of the attribution of input features to the output of neural network models is an active area of research. While numerous Explainable AI (XAI) techniques have been proposed to interpret these models, the systematic and automated evaluation of these methods in sequence-to-sequence (seq2seq) models is less explored. This paper introduces a new approach for evaluating explainability methods in transformer-based seq2seq models. We use teacher-derived attribution maps as a structured side signal to guide a student model, and quantify the utility of different attribution methods through the student's ability to simulate targets. Using the Inseq library, we extract attribution scores over source-target sequence pairs and inject these scores into the attention mechanism of a student transformer model under four composition operators (addition, multiplication, averaging, and replacement). Across three language pairs (de-en, fr-en, ar-en) and attributions from Marian-MT and mBART models, Attention, Value Zeroing, and Layer Gradient $\times$ Activation consistently yield the largest gains in BLEU (and corresponding improvements in chrF) relative to baselines. In contrast, other gradient-based methods (Saliency, Integrated Gradients, DeepLIFT, Input $\times$ Gradient, GradientShap) lead to smaller and less consistent improvements. These results suggest that different attribution methods capture distinct signals and that attention-derived attributions better capture alignment between source and target representations in seq2seq models. Finally, we introduce an Attributor transformer that, given a source-target pair, learns to reconstruct the teacher's attribution map. Our findings demonstrate that the more accurately the Attributor can reproduce attribution maps, the more useful an injection of those maps is for the downstream task. The source code can be found on GitHub.

Figures & tables

Panel of Figure 1 — Figure 1: An example of attribution maps derived from different XAI methods. For the source sentence ’Dann gibt es noch Anbieter, die kaum Fahrraderfahrung, jedoch gute Fernostkontakte haben und so an günstige E-Bikes kommen.’ and the target ’Then there are suppliers with little or no experience in the bicycle industry but good contacts in the Far East, thus giving them access to low-cost e-bikes.’ . In the heatmaps, the rows correspond to source tokens, and the columns to target tokens. The heatmaps are generated from the normalized columns using the MinMax normalizer.

Panel of Figure 2 — Figure 2: (a) Illustrates the overall design of our approach. The input sequence and the gold output $(\mathbf{x},\mathbf{y})$ are given to a teacher model, and their attributions $E$ are obtained. Then, a new untrained model is trained using the same $(\mathbf{x},\mathbf{y},E)$ triples. In the testing phase, the model gets the $(\mathbf{x},E)\rightarrow\hat{\mathbf{y}}$ . (b) Shows two places where we inject the attributions obtained from XAI methods.

BLEU scores for de-en (Baseline: 25.82)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	32.1 +6.3	32.3 +6.4	35.5 +9.6	33.6 +7.8	27.5 +1.7	32.8 +7.0	40.5 +14.7	41.8 +16.0
$\mu$	29.5 +3.7	29.7 +3.9	33.5 +7.7	32.0 +6.2	26.8 +1.0	30.5 +4.7	37.3 +11.5	40.2 +14.4
$\odot$	33.7 +7.9	33.8 +8.0	37.7 +11.8	35.4 +9.6	29.1 +3.3	34.2 +8.3	45.8 +20.0	45.8 +19.9
R	31.4 +5.6	31.7 +5.9	35.4 +9.5	33.0 +7.1	26.7 +0.9	32.3 +6.5	40.2 +14.3	41.9 +16.1
chrF scores for de-en (Baseline: 49.27)

Table 1: BLEU and chrF scores for de-en Marian-MT attributions. Scores followed by

\Delta

over the baseline.

BLEU scores for fr-en (Baseline: 27.01)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	35.5 +8.5	35.7 +8.7	39.1 +12.1	37.6 +10.6	31.5 +4.5	35.9 +8.8	45.1 +18.1	47.8 +20.8
$\mu$	32.8 +5.8	33.0 +6.0	36.0 +9.0	34.8 +7.8	28.8 +1.8	33.4 +6.4	40.8 +13.8	45.6 +18.6
$\odot$	37.6 +10.6	37.9 +10.9	42.1 +15.1	40.1 +13.1	33.5 +6.5	37.7 +10.7	55.8 +28.8	54.1 +27.1
R	34.6 +7.5	34.6 +7.6	38.2 +11.2	37.2 +10.1	30.4 +3.4	35.1 +8.1	44.4 +17.4	47.1 +20.1
chrF scores for fr-en (Baseline: 53.01)

Table 2: BLEU and chrF scores for fr-en Marian-MT attributions. Scores followed by

\Delta

over the baseline.

BLEU scores for ar-en (Baseline: 40.68)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	52.0 +11.2	51.7 +10.9	54.9 +14.1	52.1 +11.3	46.4 +5.6	52.7 +11.9	60.8 +19.9	62.9 +22.1
$\mu$	47.3 +6.5	46.9 +6.1	51.3 +10.5	47.3 +6.5	43.7 +2.9	47.4 +6.6	55.7 +14.9	60.8 +20.0
$\odot$	54.1 +13.2	54.2 +13.4	57.5 +16.7	54.2 +13.4	48.1 +7.3	54.2 +13.4	66.4 +25.6	68.7 +27.9
R	51.1 +10.3	51.0 +10.2	54.4 +13.6	51.1 +10.3	45.1 +4.3	51.9 +11.1	60.2 +19.4	62.7 +21.9
chrF scores for ar-en (Baseline: 66.07)

Table 3: BLEU and chrF scores for ar-en Marian-MT attributions. Scores followed by

\Delta

over the baseline.

BLEU scores for de-en (Baseline: 26.08)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	24.9 -1.2	25.9 -0.2	29.8 +3.8	26.0 0.0	23.2 -2.8	26.2 +0.1	25.0 -1.1	43.4 +17.3
$\mu$	23.1 -3.0	23.3 -2.8	24.6 -1.5	23.5 -2.6	22.6 -3.5	22.9 -3.2	24.2 -1.9	40.4 +14.3
$\odot$	28.2 +2.1	28.4 +2.3	32.2 +6.2	28.8 +2.7	24.9 -1.2	28.4 +2.4	39.0 +12.9	51.4 +25.3
R	25.0 -1.1	25.4 -0.7	29.0 +2.9	26.2 +0.1	23.9 -2.2	25.4 -0.7	24.7 -1.4	41.4 +15.3
chrF scores for de-en (Baseline: 48.4)

Table 4: BLEU and chrF scores for de-en mBART attributions. Scores followed by

\Delta

over the baseline.

BLEU scores for fr-en (Baseline: 27.12)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	33.2 +6.1	33.9 +6.8	38.9 +11.8	34.8 +7.7	28.4 +1.3	34.0 +6.9	31.8 +4.7	51.2 +24.1
$\mu$	29.5 +2.4	30.9 +3.7	35.2 +8.1	29.9 +2.7	26.8 -0.3	29.9 +2.8	30.8 +3.7	45.2 +18.1
$\odot$	37.5 +10.4	38.0 +10.9	42.9 +15.8	37.5 +10.4	32.7 +5.6	38.2 +11.0	59.7 +32.6	63.0 +35.9
R	32.2 +5.1	32.4 +5.3	37.4 +10.2	32.3 +5.2	28.0 +0.9	32.7 +5.5	31.3 +4.2	48.9 +21.8
chrF scores for fr-en (Baseline: 53.01)

Table 5: BLEU and chrF scores for fr-en mBART attributions. Scores followed by

\Delta

over the baseline.

BLEU scores for ar-en (Baseline: 40.68)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	50.6 +9.9	51.2 +10.5	55.3 +14.6	50.5 +9.8	44.0 +3.3	50.3 +9.6	49.0 +8.3	63.6 +22.9
$\mu$	45.1 +4.4	44.7 +4.0	49.7 +9.1	45.0 +4.3	42.1 +1.5	44.2 +3.6	45.7 +5.0	58.4 +17.7
$\odot$	53.7 +13.0	54.5 +13.8	59.5 +18.9	53.5 +12.8	48.3 +7.6	54.2 +13.6	73.4 +32.7	74.4 +33.7
R	48.3 +7.6	48.7 +8.0	52.8 +12.1	48.5 +7.7	42.0 +1.3	48.6 +7.9	47.6 +6.9	60.4 +19.7
chrF scores for ar-en (Baseline: 65.41)

Table 6: BLEU and chrF scores for ar-en mBART attributions. Scores followed by

\Delta

over the baseline.

BLEU scores for de-en (Baseline: 25.82)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	25.3 -0.5	25.4 -0.4	24.9 -0.9	24.8 -1.0	22.4 -3.4	25.6 -0.3	16.9 -8.9	23.9 -1.9
$\mu$	24.8 -1.0	25.0 -0.8	26.0 +0.2	23.8 -2.0	22.3 -3.5	24.8 -1.1	20.0 -5.8	23.0 -2.8
$\odot$	26.1 +0.3	26.1 +0.3	28.6 +2.8	28.5 +2.6	21.8 -4.1	26.3 +0.5	25.6 -0.2	24.7 -1.1
R	24.2 -1.6	24.2 -1.6	22.6 -3.2	23.0 -2.9	18.2 -7.6	24.4 -1.4	15.7 -10.1	21.0 -4.8
chrF scores for de-en (Baseline: 49.27)

Table 7: BLEU and chrF scores for de-en Cross-attention Marian-MT attributions (Baseline: 25.82). Scores followed by

\Delta

over the baseline.

BLEU scores for fr-en (Baseline: 27.01)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	26.9 -0.1	26.9 -0.1	26.1 -0.9	28.2 +1.2	25.2 -1.8	25.3 -1.7	21.0 -6.0	27.6 +0.6
$\mu$	26.4 -0.6	26.6 -0.4	26.9 -0.1	27.6 +0.6	24.7 -2.3	24.5 -2.6	22.2 -4.8	25.6 -1.4
$\odot$	27.7 +0.6	27.8 +0.8	29.1 +2.1	31.7 +4.7	25.2 -1.8	27.7 +0.7	19.9 -7.2	26.3 -0.7
R	25.4 -1.6	25.5 -1.5	23.3 -3.7	26.2 -0.8	21.5 -5.5	25.1 -1.9	16.6 -10.4	24.5 -2.5
chrF scores for fr-en (Baseline: 53.01)

Table 8: BLEU and chrF scores for fr-en Cross-attention Marian-MT attributions (Baseline: 27.01). Scores followed by

\Delta

over the baseline.

BLEU scores for ar-en (Baseline: 40.68)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	36.2 -4.7	41.9 +1.1	35.9 -4.9	36.2 -4.7	36.0 -4.9	36.1 -4.8	36.1 -4.7	40.6 -0.2
$\mu$	35.3 -5.5	41.7 +0.9	35.2 -5.6	35.3 -5.5	35.2 -5.6	35.3 -5.5	35.3 -5.5	40.5 -0.3
$\odot$	28.5 -12.3	42.2 +1.4	26.5 -14.3	29.6 -11.2	28.3 -12.5	29.3 -11.5	26.3 -14.5	42.9 +2.1
R	14.3 -26.5	40.0 -0.8	13.9 -26.9	14.5 -26.3	14.3 -26.5	14.2 -26.6	13.7 -27.1	39.1 -1.7
chrF scores for ar-en (Baseline: 66.07)

Table 9: BLEU and chrF scores for ar-en Cross-attention Marian-MT attributions (Baseline: 40.68). Scores followed by

\Delta

over the baseline.

BLEU scores for de-en (Baseline: 54.55)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	64.5 +9.9	64.6 +10.1	68.5 +13.9	65.5 +11.0	58.1 +3.5	65.1 +10.5	72.0 +17.4	73.7 +19.1
$\mu$	58.4 +3.8	59.7 +5.2	65.2 +10.7	61.8 +7.3	55.1 +0.5	61.1 +6.6	68.8 +14.3	71.6 +17.0
$\odot$	66.5 +11.9	66.6 +12.0	70.0 +15.5	67.3 +12.8	59.7 +5.1	66.3 +11.8	75.4 +20.9	74.2 +19.6
R	63.4 +8.8	63.1 +8.6	67.9 +13.3	64.4 +9.8	56.0 +1.4	64.2 +9.7	72.0 +17.4	73.4 +18.9
chrF scores for de-en (Baseline: 72.77)

Table 10: BLEU and chrF scores for de-en generated by the Marian-MT model. Scores followed by

\Delta

over the baseline.

BLEU scores for fr-en (Baseline: 53.38)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	53.7 +0.3	53.6 +0.2	72.6 +19.2	70.7 +17.3	52.8 -0.6	53.6 +0.2	76.3 +22.9	80.6 +27.2
$\mu$	52.8 -0.6	52.9 -0.4	63.6 +10.2	64.0 +10.7	52.4 -1.0	52.8 -0.6	69.5 +16.1	74.3 +20.9
$\odot$	62.2 +8.8	73.4 +20.1	76.5 +23.1	75.8 +22.5	58.7 +5.3	62.9 +9.5	82.9 +29.5	83.0 +29.6
R	54.1 +0.7	66.5 +13.1	72.8 +19.4	70.9 +17.5	52.2 -1.2	54.2 +0.8	77.4 +24.0	80.5 +27.1
chrF scores for fr-en (Baseline: 74.05)

Table 11: BLEU and chrF scores for fr-en generated by the Marian-MT model. Scores followed by

\Delta

over the baseline.

BLEU scores for ar-en (Baseline: 59.45)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	69.1 +9.6	68.6 +9.1	73.1 +13.6	69.8 +10.3	63.7 +4.2	70.0 +10.6	76.8 +17.4	78.9 +19.4
$\mu$	65.1 +5.6	64.6 +5.1	65.3 +5.9	63.8 +4.3	61.7 +2.3	65.1 +5.7	67.4 +7.9	75.1 +15.7
$\odot$	72.6 +13.2	72.8 +13.3	75.6 +16.1	72.4 +13.0	66.7 +7.2	72.9 +13.5	81.6 +22.2	82.8 +23.3
R	68.9 +9.4	68.5 +9.0	72.7 +13.3	68.9 +9.4	63.3 +3.8	69.5 +10.1	77.0 +17.6	78.7 +19.2
chrF scores for ar-en (Baseline: 77.96)

Table 12: BLEU and chrF scores for ar-en generated by the Marian-MT model. Scores followed by

\Delta

over the baseline.

	Random matrix			Diagonal matrix
Op.	de-en	fr-en	ar-en	de-en	fr-en	ar-en
$\odot$	22.98 -2.8	24.85 -2.1	37.96 -2.9	21.64 -4.2	24.11 -2.9	37.34 -3.3

Table 13: BLEU scores for random attribution matrices (left) and diagonal attribution matrices (right). Scores followed by

\Delta

over the baseline.

Figure 3: Column-wise Entropy based on Marian-MT attributions.

Figure 4: Column-wise Entropy based on mBART attributions.

Figure 5: Column-wise entropy based on Marian-MT attributions (Marian-MT-generated targets)

Figure 6: KL-divergence, Overlap@3, and

\tau@3

for the prediction of attribution on Marian-MT attribution and gold (human) data.

Figure 7: KL-divergence, Overlap@3, and

\tau@3

for the prediction of attribution on mBART attribution and gold (human) data.

Figure 8: KL-divergence, Overlap@3, and

\tau@3

for the prediction of attribution on Marian-MT attribution generated target.

Panel of Figure 9 — Figure 9: Regression plots of the Marian-MT attributions with gold (human) target sentence.

Panel of Figure 10 — Figure 10: Regression plots of the mBART attributions with gold (human) target sentence.

Panel of Figure 11 — Figure 11: Regression plot of the Marian-MT attributions with generated target sentences.

BLEU scores for de-en (Baseline: 25.82)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	32.7 +6.8	32.4 +6.6	35.7 +9.9	33.7 +7.8	27.8 +2.0	32.9 +7.1	40.0 +14.1	41.8 +16.0
$\mu$	29.9 +4.0	29.6 +3.8	33.5 +7.7	32.1 +6.3	26.2 +0.4	30.5 +4.7	36.1 +10.2	40.5 +14.7
$\odot$	34.4 +8.5	34.7 +8.9	38.3 +12.5	36.0 +10.2	29.9 +4.0	34.6 +8.8	47.2 +21.4	44.8 +18.9
R	32.4 +6.6	32.3 +6.5	35.9 +10.1	33.8 +8.0	27.9 +2.1	32.9 +7.1	40.2 +14.4	41.7 +15.8
chrF scores for de-en (Baseline: 49.27)

Table 14: BLEU and chrF scores for de-en Marian-MT 4 head attribution injection. Scores followed by

\Delta

over the baseline.

BLEU scores for fr-en (Baseline: 27.01)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	34.9 +7.9	34.6 +7.6	38.5 +11.5	37.5 +10.5	31.2 +4.2	35.1 +8.1	44.3 +17.3	47.5 +20.5
$\mu$	32.1 +5.1	33.1 +6.1	34.9 +7.9	34.1 +7.1	28.5 +1.5	32.8 +5.8	39.9 +12.9	44.9 +17.9
$\odot$	38.2 +11.2	37.9 +10.9	42.3 +15.3	40.8 +13.8	34.1 +7.1	38.4 +11.4	56.1 +29.1	53.7 +26.7
R	35.3 +8.3	35.4 +8.4	38.9 +11.9	37.8 +10.8	31.1 +4.1	35.7 +8.7	43.5 +16.5	46.8 +19.8
chrF scores for fr-en (Baseline: 53.01)

Table 15: BLEU and chrF scores for fr-en Marian-MT 4 head attribution injection. Scores followed by

\Delta

over the baseline.

BLEU scores for ar-en (Baseline: 40.68)
Op.	I $\times$ G	Saliency	LG $\times$ A	IG	GSHAP	DeepLIFT	Attention	ValueZeroing
$+$	51.9 +11.1	51.6 +10.8	54.9 +14.1	51.8 +11.0	46.1 +5.3	52.7 +11.9	60.1 +19.3	63.0 +22.2
$\mu$	47.5 +6.7	47.3 +6.5	50.3 +9.5	47.0 +6.2	43.8 +3.0	47.8 +7.0	54.6 +13.8	59.8 +19.0
$\odot$	54.9 +14.1	55.0 +14.2	58.1 +17.3	54.9 +14.0	49.0 +8.2	55.1 +14.3	66.9 +26.1	68.1 +27.3
R	51.9 +11.1	51.5 +10.7	55.1 +14.3	51.8 +11.0	46.2 +5.4	52.3 +11.5	60.7 +19.9	63.3 +22.5
chrF scores for ar-en (Baseline: 66.07)

Table 16: BLEU and chrF scores for ar-en Marian-MT 4 head attribution injection. Scores followed by

\Delta

over the baseline.

Model	Text	BLEU	chrF
Marian-MT
Source:	Ich dachte, dass es damit beendet wäre.	–	–
Reference:	I thought that would be the end of it.	–	–
Baseline:	I thought it was finished.	9.91	29.19
I $\times$ G	I thought it would have been finished with it.	12.55	43.76
Saliency	I thought it would have been finished with it.	12.55	43.76

Table 17: Sample translations and metrics for Marian-MT and mBART under different attribution/XAI variants for a sample de-en.

Attention-Guided Distillation

See all 11 papers

Aug 3, 2026

Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs

Most evidence on the effectiveness of explainable artificial intelligence (XAI) attribution methods has been established on convolutional neural networks (CNNs), with limited investigation into whether these conclusions generalize to the diverse Vision Transformer (ViT) architectures that now dominate computer vision. This paper presents a controlled benchmark that evaluates attribution quality across five dimensions: faithfulness, localization, robustness, complexity, and computational cost. A standardized framework assesses 13 attribution methods from four algorithmic families on eight representative backbones spanning CNNs, isotropic ViTs, hierarchical transformers, hybrid architectures, and linear-attention transformers. The results show that attribution performance is strongly architecture-dependent and that rankings established on CNNs do not reliably transfer to transformer-based models. CAM-based methods achieve the highest scores under the conventional bounding-box localization metric on CNNs and most ViTs but perform poorly on linear-attention architectures. Pixel-level dense-mask evaluation further reveals that these gains largely reflect metric saturation rather than accurate localization. CAM-based methods also exhibit limited robustness on global-attention transformers, whereas attention rollout provides consistently stable explanations with poor localization. Furthermore, faithfulness correlation offers limited discrimination between attribution methods, highlighting the limitations of single-metric evaluation. These findings challenge prevailing conclusions on attribution performance and demonstrate the need for architecture-aware, multi-dimensional evaluation. The open-source code for the evaluation framework and benchmark results is available at https://github.com/Nishan-Charlie/VIT_XAI_Bench.

Sathiyamohan Nishankar, Nethmi Pathirana, Pubudu Sanjeewani +2

aFaculty of Computing, Sabaragamuwa University of Sri Lanka, Sri Lanka · bFaculty of Engineering, University of Moratuwa, Katubedda, Sri Lanka · cSchool of Computing Technologies, RMIT University, Melbourne, Australia +3

Transformer Interpretability Feature Attribution Vision Transformer Explainability Evaluation

Ask PDF

Feb 18, 2026

Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models

Transformer models achieve state-of-the-art performance across domains and tasks, yet their deeply layered representations make their predictions difficult to interpret. Existing explainability methods rely on final-layer attributions, capture either local token-level attributions or global attention patterns without unification, and lack context-awareness of inter-token dependencies and structural components. They also fail to capture how relevance evolves across layers and how structural components shape decision-making. To address these limitations, we proposed the Context-Aware Layer-wise Integrated Gradients (CA-LIG) Framework, a unified hierarchical attribution framework that computes layer-wise Integrated Gradients within each Transformer block and fuses these token-level attributions with class-specific attention gradients. This integration yields signed, context-sensitive attribution maps that capture supportive and opposing evidence while tracing the hierarchical flow of relevance through the Transformer layers. We evaluate the CA-LIG Framework across diverse tasks, domains, and transformer model families, including sentiment analysis and long and multi-class document classification with BERT, hate speech detection in a low-resource language setting with XLM-R and AfroLM, and image classification with Masked Autoencoder vision Transformer model. Across all tasks and architectures, CA-LIG provides more faithful attributions, shows stronger sensitivity to contextual dependencies, and produces clearer, more semantically coherent visualizations than established explainability methods. These results indicate that CA-LIG provides a more comprehensive, context-aware, and reliable explanation of Transformer decision-making, advancing both the practical interpretability and conceptual understanding of deep neural models.

Melkamu Abay Mersha, Jugal Kalita

College of Engineering and Applied Science, University of Colorado Colorado Springs, Colorado Springs, 80918, CO, USA

Transformer Interpretability Gradient-Based Attribution Integrated Gradients Explainability Evaluation

Ask PDF

Jun 12, 2026

Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models

Transformer-based automatic speech recognition (ASR) models such as Whisper are highly accurate, but their predictions remain difficult to interpret. Existing explainable AI (XAI) methods often lack faithfulness and precise temporal grounding. We propose Listening with Entropy-guided Attention for Faithful explainability (LEAF-X), a model-intrinsic XAI framework for transformer-based ASR. LEAF-X combines entropy-guided attention weighting, multi-layer attention rollout, and optional causal ablations to identify low-entropy, high-impact heads and layers, producing sparse token-to-frame attributions. Unlike perturbation-based explainers or raw attention maps, LEAF-X exploits the internal structure of encoder-decoder and speech-augmented decoder-only models to generate explanations that better reflect model computation. Results show 32% improved faithfulness, 35-39% stronger locality/sparsity, and the most stable attributions, supporting more transparent and auditable ASR.

Ravi Ranjan, Utkarsh Grover, Xiaomin Lin +1

Florida International University · Miami, USA · University of South Florida +1

Transformer Interpretability Explainable Artificial Intelligence Automatic Speech Recognition Transformer Attention

Ask PDF

Abstract

Attention-Guided Distillation

Explore similar work

Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs

Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models

Listening with Attention: Entropy-Guided Explainability for Transformer-Based Audio Models