cs.LGMar 25, 2026

No Single Metric Tells the Whole Story: A Multi-Dimensional Evaluation Framework for Uncertainty Attributions

Authors: Emily Schiller, Teodor Chiaburu, Marco Zullich, Luca Longo

Organizations: XITASO GmbH IT & Software Solutions, Augsburg, Germany · The Artificial Intelligence and Cognitive Load Research Lab, University College Cork · Berliner Hochschule für Technik, Berlin, Germany · Faculty of Technology, Policy and Management, Delft University of Technology, Delft, the Netherlands · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Research on explainable AI (XAI) has frequently focused on explaining model predictions. More recently, methods have been proposed to explain prediction uncertainty by attributing it to input features (uncertainty attributions). However, the evaluation of these methods remains inconsistent as studies rely on heterogeneous proxy tasks and metrics, hindering comparability. We address this by aligning uncertainty attributions with the well-established Co-12 framework for XAI evaluation. We propose concrete implementations for the correctness, consistency, continuity, and compactness properties. Additionally, we introduce conveyance, a property tailored to uncertainty attributions that evaluates whether controlled increases in epistemic uncertainty reliably propagate to feature-level attributions. We demonstrate our evaluation framework with eight metrics across combinations of uncertainty quantification and feature attribution methods on tabular and image data. Our experiments show that gradient-based methods consistently outperform perturbation-based approaches in consistency and conveyance, while Monte-Carlo dropconnect outperforms Monte-Carlo dropout in most metrics. Although most metrics rank the methods consistently across samples, inter-method agreement remains low. This suggests no single metric sufficiently evaluates uncertainty attribution quality. The proposed evaluation framework contributes to the body of knowledge by establishing a foundation for systematic comparison and development of uncertainty attribution methods.

Figures & tables

Figure 1: Uncertainty attribution method by Bley et al. Bley et al. (2025) . Predictive variance is estimated via ensemble-based UQ. Each of KK predictions is explained, and uncertainty attributions are derived as diagonal elements of the covariance matrix of these explanations. Illustration adapted from Bley et al. Bley et al. (2025) .
Figure 2: Uncertainty and feature attributions using mcd and (a) LRP and (b) Integrated Gradients for test samples from the MNIST dataset. Dark blue pixels indicate low importance and dark red pixels indicate high importance to predictive uncertainty (uncertainty attribution) or the prediction (feature attribution).
Figure 3: Overview of the proposed evaluation framework. We combine a feature attribution method with an uq method (blue panel) to generate uncertainty attributions. We evaluate these uncertainty attributions by adapting and extending properties (orange panel) and metrics (red panel) from feature attribution to the uncertainty setting; novel categories and metrics appear in italics. Metric results are aggregated into an overall score for each uncertainty attribution method.
Co-12 PropertyMetrics used in Literature on uncertainty attributionsMetrics selected for our Evaluation Framework
correctness 4.1feature flipping Bley et al. (2025) , blurring test Perez et al. (2022) ; Wang et al. (2023a) ; Iversen et al. (2025)feature flipping
consistency 4.2-repeatability
continuity 4.3local Lipschitz continuity Iversen et al. (2025)relative input stability (RIS)
compactness 4.4-complexity
conveyance 4.5distribution shift detection Brown and Talbert (2022) ; Wang et al. (2023b) ; Watson et al. (2023) ; Pohland and Tomlin (2024) , controlled aleatoric uncertainty Iversen et al. (2025)relative rank improvement (RRI) , uncertainty conveyance similarity (UCS)
Table 1: Evaluation properties and metrics for uncertainty attributions. We adapt four Co-12 properties Nauta et al. (2023) for evaluating uncertainty attribution methods. For each property, we report metrics from the prior uncertainty-attribution literature and from our framework. Conveyance (highlighted in blue) is our proposed property specific to uncertainty attributions. Bold metrics are novel adaptations from feature attribution research to uncertainty attributions. Bold italics indicate our novel uncertainty similarity conveyance metric.
Figure 4: Experimental setup. Using 5-fold cross-validation, we train a CNN for MNIST and an MLP for Wine Quality. We compute uncertainty attributions for 100 test samples per fold using combinations of UQ and feature attribution methods (blue panel), rate them using our evaluation framework, and assess metrics via sanity checks.
Figure 5: Metric scores for the Wine Quality dataset with 95% confidence interval. Arrows indicate whether lower ( ↓\downarrow ) or higher ( ↑\uparrow ) values are better. Gradient-based methods (IxG, IG) and LRP outperform perturbation-based methods on most metrics for mcd and mcdc . mcdc shows better repeatability, feature flipping, and higher UCS than MCD.
Figure 6: Metric scores for the MNIST dataset with 95% confidence interval. Arrows indicate whether lower ( ↓\downarrow ) or higher ( ↑\uparrow ) values are better. mcdc outperforms mcd in UCS but achieves higher feature flipping scores. All methods exhibit near-perfect repeatability and RIS.
 MetricUQ MethodLRPIGIxGSHAPLIME
Wine Qualitycomplexity ↓\downarrowmcd1.7221.7041.6952.3782.356
 mcdc1.6891.6241.5672.3752.357
repeatability (cosine) ↑\uparrowmcd0.9760.9790.9790.9640.947
 mcdc0.9760.9880.9870.9610.948
repeatability (Spearman) ↑\uparrowmcd0.9610.9670.9660.0050.128
 mcdc0.9650.9750.9740.0140.115
Table 2: Metric scores averaged over all 5 folds for Wine Quality and MNIST datasets. Arrows indicate whether lower ( ↓\downarrow ) or higher ( ↑\uparrow ) values are better. Best metric values across all uncertainty attribution methods are highlighted in blue.
MetricDatasetInter-method reliability ↑\uparrowRanking consistency ↑\uparrowAverage coefficient of variation ↓\downarrow
complexityWine Quality0.2790.7660.018
MNIST0.6560.5530.135
repeatability (cosine)Wine Quality0.0760.6460.019
MNIST0.2130.3280.014
repeatability (Spearman)Wine Quality0.1680.7621.227
MNIST0.6470.5270.000
Table 3: Sanity checks of all evaluation metrics for Wine Quality and MNIST. Arrows indicate whether lower ( ↓\downarrow ) or higher ( ↑\uparrow ) values are better for each sanity check. Best scores for Wine Quality and MNIST are highlighted in blue and red, respectively.
 MetricsUQ MethodLRPIGIxGSHAPLIME
Wine QualityUCS (Spearman) vs RRImcd0.0250.0450.087-0.015-0.002
 mcdc0.032-0.0030.0340.049-0.001
 UCS (cosine) vs RRImcd-0.1000.038-0.1060.014-0.002
 mcdc-0.0310.0310.0110.027-0.004
 MetricsUQ MethodLRPIGIxGGradientSHAP
MNISTUCS (Spearman) vs RRImcd0.025-0.011-0.0240.001
Table 4: Spearman’s ρ\rho (internal consistency reliability) between pairs of conveyance metrics for each uncertainty attribution method for Wine Quality and MNIST.
More in this topic

Explainability Evaluation

Explainability evaluation tests whether AI explanations are faithful, stable, understandable, useful, and reliable across models and settings.

See all 319 papers

Explore similar work

CardsList
Mar 27, 2026

Concerning Uncertainty -- A Systematic Survey of Uncertainty-Aware XAI

This paper surveys uncertainty-aware explainable artificial intelligence (UAXAI), examining how uncertainty is incorporated into explanatory pipelines and how such methods are evaluated. Across the literature, three recurring approaches to uncertainty quantification emerge (Bayesian, Monte Carlo, and Conformal methods), alongside distinct strategies for integrating uncertainty into explanations: assessing trustworthiness, constraining models or explanations, and explicitly communicating uncertainty. Evaluation practices remain fragmented and largely model centered, with limited attention to users and inconsistent reporting of reliability properties (e.g., calibration, coverage, explanation stability). Recent work leans towards calibration, distribution free techniques and recognizes explainer variability as a central concern. We argue that progress in UAXAI requires unified evaluation principles that link uncertainty propagation, robustness, and human decision-making, and highlight counterfactual and calibration approaches as promising avenues for aligning interpretability with reliability.

Helena Löfström, Tuwe Löfström, Anders Hjort +1

Jönköping International Business School, Gjuterigatan 5, 553 18 Jönköping, Sweden · Jönköping School of Engineering, Gjuterigatan 5, 553 18 Jönköping, Sweden · Eiendomsverdi AS, Nedre Slottsgate 8, 0157 Oslo, Norway +2

Explainable Artificial IntelligenceUncertainty QuantificationExplainability EvaluationUncertainty Propagation
AskPDF
Apr 23, 2025

Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps

The evaluation of explainable AI (XAI) approaches often relies on three families of methods: subjective measures (e.g., questionnaires on trust or satisfaction), objective measures (e.g., task performance metrics), and mathematical metrics (e.g., for faithfulness). Yet, it remains unclear how these families align or diverge in practice. In a{preregistered} between-subjects study (N=166), we use three established saliency map techniques (LIME, Grad-CAM, Guided Backpropagation) as a testbed to examine this issue. We find that each family of methods leads to different conclusions: participants reported no differences in trust or satisfaction, Grad-CAM improved user performance, while mathematical metrics favored Guided Backpropagation. At the same time, mathematical metrics were only partially related to user performance, and these relationships were sometimes counterintuitive. Our findings highlight the methodological importance of comparing subjective, objective, and mathematical approaches when evaluating XAI, illustrating both tensions and aspects that are aligned. We discuss implications for XAI evaluation frameworks.

Felix Kares, Timo Speith, Hanwei Zhang +1

Saarland University, Germany · University of Bayreuth, Germany · University of Freiburg, Germany

Saliency Map EvaluationExplainability Evaluation
AskPDF
Oct 22, 2026

XALIGN: A Human-centric Visual Analytics Approach to Understanding AI Explanations

Explainable AI (XAI) methods are usually evaluated for technical properties such as faithfulness and robustness. In high-stakes application domains, experts also need to check model predictions against established knowledge. Different explanation methods often disagree, and this disagreement is hard to see when each method is inspected separately. We propose XALIGN, a visual analytics workflow and interface that shows rule-based explanations and several feature attribution methods (TreeSHAP, KernelSHAP, and LIME) side by side for the same Random Forest model. Agreements and contradictions between the methods become visible signals that users can explore and compare with domain knowledge. We apply XALIGN in two case studies: the German Credit dataset and a meteorological dataset, in which events derived from ICON model output and Cloudnet observations are classified. In both, the views reveal discrepancies between the rule-based explanation and the attribution methods that a single method would not show. In an expert evaluation with three specialists, participants found XALIGN useful for identifying influential features and for avoiding over-trust in predictions. Linking rules to attributions was harder for the participant without an XAI background.

Bahavathy Kathirgamanathan, Gennady Andrienko, Natalia Andrienko +1

University of Cologne, Cologne, Germany · Fraunhofer IAIS, Sankt Augustin, Germany · City St George’s Institute of London, London, UK

Explainable Artificial IntelligenceExplainability EvaluationFeature Attribution
AskPDF