A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods

Miquel Miró-Nicolau; Francesco Spinnato; Riccardo Guidotti

cs.AISep 24, 2026

A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods

Authors: Miquel Miró-Nicolau, Francesco Spinnato, Riccardo Guidotti

Organizations: Universitat de les Illes Balears, Cra. de Valldemossa, km 7.5., Palma, 07122, Spain · Uiniversity of Pisa, Largo B. Pontecrovo, 3, 56127, Pisa, Italy · ISTI-CNR, Via G. Moruzzi, 1, Pisa, 56127, Italy

Abstract

Evaluating explainable Artificial Intelligence (XAI) methods is a challenging task due to the lack of reliable evaluation procedures and, in particular, the absence of ground truth explanations. In the literature, existing evaluation approaches typically assess explanations by measuring their fidelity with respect to the predictions of a black-box model. However, such evaluation strategies only quantify the degree to which an explanation reproduces the model's output, without ensuring that the explanation correctly reflects the underlying decision process. As a consequence, different explanations may achieve similar fidelity scores while providing inconsistent or misleading interpretations of the model behavior. In this paper, we propose a framework for the evaluation of XAI methods based on synthetic ground truth. The proposed approach relies on controlled interventions to generate synthetic datasets in which the importance of input components can be determined by design. This enables the construction of ground truth explanations that are directly aligned with the behavior of the model under analysis. The framework is instantiated across three data domains, namely binary images, tabular data, and time series, allowing a comprehensive assessment of explanation methods in heterogeneous settings. Experimental results obtained by evaluating nine widely used XAI methods show significant limitations in current techniques and highlight the importance of synthetic, intervention-based benchmarks for a reliable assessment of explanation quality.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of the intervention procedure across data modalities: color images, binary images, and time series. The latter two are constructed from the original color image using the proposed framework.

Panel of Figure 2 — Figure 2: Representative samples from the FunnyBirds dataset across distinct classes. Each category is defined by a unique combination of geometric and color parts: wing color and foot shape (left image), wing and tail color (center image), and the foot shape (right image).

Panel of Figure 3 — Figure 3: Illustration of class ambiguity induced by binarization. Two images belonging to different classes (first and second) become identical when converted to binary representations (third and fourth), as color information is removed.

Figure 4: Step-by-step procedure for extracting a time series from a color image. From left to right: the original image, the binarized image, the measurement of radial distances from the object center to its contour, and the resulting time series plotted from these measurements

Data Modality	Model	Precision	Recall	F1-Score
Binary images	ResNet50	$0.91$	$0.88$	$0.89$
Time series	Multirocket w. Ridge	$0.64$	$0.63$	$0.63$
Tabular data	MLP	$0.98$	$0.98$	$0.98$

Table 1: Performance measure results for each AI model used in each experiment.

Method	Hyperparameter	Value
RISE	N Masks	$600$
	S Masks	$8$
	Probability remaining	$0.1$
LIME	Occlusion Value	$0$
	N Samples	$1500$
	Kernel Width	$0.25$

Table 2: XAI method hyperparameter values for binary image experiments.

Method	Hyperparameter	Value
RISE	N Masks	$1000$
	Grid resolution	$16$
	Cell activation probability	$0.5$
	Mask generation	Linear upsampling + random shift
LIME	N Samples	$5000$
	Sampling	Gaussian

Table 3: XAI method hyperparameters values for time series experiments.

Method	Hyperparameter	Value
LIME	Occlusion Value	$0$
	N Samples	$1500$
	Kernel Width	$0.25$
	Feature Selection	Highest Weights
	Kernel	Similarity kernel
	Distance Metric	Cosine

Table 4: XAI method hyperparameters values for tabular data experiments.

	$\minf\uparrow$	$\auc\uparrow$	$\kl\downarrow$
RISE [ 42 ]	$0.343\pm 0.382$	$0.573\pm 0.273$	$1.915\pm 1.505$
LIME [ 43 ]	$0.293\pm 0.164$	$\mathbf{0.932\pm 0.083}$	$\mathbf{1.217\pm 0.664}$
Gradient [ 48 ]	$0.378\pm 0.427$	$0.500\pm 0.026$	$1.903\pm 1.517$
DeepLIFT [ 47 ]	$\mathbf{0.380\pm 0.429}$	$0.555\pm 0.123$	$1.895\pm 1.510$
KernelSHAP [ 31 ]	$0.342\pm 1.210$	$0.848\pm 0.165$	$1.551\pm 1.210$
Integrated Gradients [ 51 ]	$\mathbf{0.380\pm 0.430}$	$0.510\pm 0.098$	$1.905\pm 1.519$

Table 5: Metrics results to compare XAI methods output and proposed Ground Truth in the Binary images task. All reported value are the mean and standard deviation for the test set. Best values for each metric in bold.

Panel of Figure 5 — Figure 5: Comparison of different XAI saliency map techniques for the binary image dataset The more intense the color the more important the pixel.

Figure 6: Time series attribution for an instance of class 2 . From top to bottom, the ground truth (GT), and the predicted attributions from LIME, RISE, TSHAP (ROI and WIN), and KernelSHAP. The more intense the color the more important the time series observation.

	$\minf\uparrow$	$\auc\uparrow$	$\kl\downarrow$
RISE	$\mathbf{0.522\pm 0.136}$	$\mathbf{0.709\pm 0.161}$	$\mathbf{0.790\pm 0.405}$
LIME	$0.442\pm 0.199$	$0.646\pm 0.134$	$0.907\pm 0.510$
Kernel SHAP	$0.007\pm 0.003$	$0.500\pm 0.003$	$5.315\pm 0.509$
T-SHAP ROI	$0.386\pm 0.170$	$0.490\pm 0.089$	$1.784\pm 0.767$
T-SHAP Window	$0.424\pm 0.191$	$0.490\pm 0.132$	$1.055\pm 0.562$

Table 6: Metrics results to compare XAI methods output and proposed Ground Truth in the Time series task. All reported value are the mean and standard deviation for the validation set. Best values for each metric in bold.

	$\minf\uparrow$	$\kl\downarrow$
LORE [ 15 ]	$0.784\pm 0.177$	$1.204\pm 1.855$
LIME [ 43 ]	$0.560\pm 0.269$	$3.806\pm 3.535$
Gradient [ 48 ]	$\mathbf{0.980\pm 0.099}$	$\mathbf{0.092\pm 0.618}$
DeepLIFT [ 47 ]	$0.636\pm 0.221$	$2.011\pm 2.559$
Kernel SHAP [ 31 ]	$0.601\pm 0.229$	$2.315\pm 2.701$
Integrated Gradients [ 51 ]	$0.642\pm 0.221$	$1.983\pm 2.559$

Table 7: Metrics results to compare XAI methods output and proposed Ground Truth in the Tabular data task. All reported value are the mean and standard deviation for the validation set. Best values for each metric in bold.

Explainability Evaluation

Explainability evaluation tests whether AI explanations are faithful, stable, understandable, useful, and reliable across models and settings.

See all 322 papers

Learning Quantifiable Visual Explanations Without Ground-Truth
May 18, 2026Amritpal Singh, Andrey Barsky, Mohamed Ali Souibgui +2Perturbation-Based Feature Attribution Causal Interpretability
Computer Vision Center, Barcelona, Spain · Autonomous University of Barcelona, Spain
Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps
Apr 23, 2025Felix Kares, Timo Speith, Hanwei Zhang +1Saliency Map Evaluation Explainability Evaluation
Saarland University, Germany · University of Bayreuth, Germany · University of Freiburg, Germany
Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models
Jul 15, 2026Georgios Makridis, Georgios Fatouros, Athanasios Kiourtis +4Feature Attribution Explainability Evaluation
Department of Digital Systems University of Piraeus Piraeus, Greece · Department of Business Administration University of Piraeus Piraeus, Greece · Innov-Acts Ltd Innov-Acts Nicosia, Cyprus

May 18, 2026J/K move · Enter open · S save

Learning Quantifiable Visual Explanations Without Ground-Truth

Amritpal Singh, Andrey Barsky, Mohamed Ali Souibgui, Ernest Valveny+1

Computer Vision Center, Barcelona, Spain · Autonomous University of Barcelona, Spain

Perturbation-Based Feature Attribution Causal Interpretability Explainability Evaluation Explanation Evaluation

Open paper AskPDF arXiv

Explainable AI (XAI) techniques are increasingly important for the validation and responsible use of modern deep learning models, but are difficult to evaluate due to the lack of good ground-truth to compare against. We propose a framework that serves as a quantifiable metric for the quality of XAI methods, based on continuous input perturbation. Our metric formally considers the sufficiency and necessity of the attributed information to the model's decision-making, and we illustrate a range of cases where it aligns better with human intuitions of explanation quality than do existing metrics. To exploit the properties of this metric, we also propose a novel XAI method, considering the case where we fine-tune a model using a differentiable approximation of the metric as a supervision signal. The result is an adapter module that can be trained on top of any black-box model to output causal explanations of the model's decision process, without degrading model performance. We show that the explanations generated by this method outperform those of competing XAI techniques according to a number of quantifiable metrics.

Abstract

Explainability Evaluation

Explore similar work

Learning Quantifiable Visual Explanations Without Ground-Truth

Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps

Towards a Unified Multidimensional Explainability Metric: Evaluating Trustworthiness in AI Models