Extended Empirical Validation of the Explainability Solution Space

Antoni Mestre; Manoli Albert; Miriam Gil; Vicente Pelechano

cs.AIMar 1, 2026

Extended Empirical Validation of the Explainability Solution Space

Authors: Antoni Mestre, Manoli Albert, Miriam Gil, Vicente Pelechano

Organizations: VRAIN Institute, Universitat Politècnica de València, Camí de Vera s/n, Valencia 46022, Spain · Departament d’Informàtica, Universitat de València, Burjassot 46100, Spain

Abstract

This technical report provides an extended validation of the Explainability Solution Space (ESS) through cross-domain evaluation. While initial validation focused on employee attrition prediction, this study introduces a heterogeneous intelligent urban resource allocation system to demonstrate the generality and domain-independence of the ESS framework. The second case study integrates tabular, temporal, and geospatial data under multi-stakeholder governance conditions. Explicit quantitative positioning of representative XAI families is provided for both contexts. Results confirm that ESS rankings are not domain-specific but adapt systematically to governance roles, risk profiles, and stakeholder configurations. The findings reinforce ESS as a generalizable operational decision-support instrument for explainable AI strategy design across socio-technical systems.

Figures & tables

Dimension	Description
Model type	XGBoost ensemble (250 trees, max depth 6, learning rate 0.05)
Input features	87 features: transaction attributes, geospatial signals, device fingerprints, velocity features (1 h / 24 h / 7 d windows)
Volume	$\approx$ 4.2M transactions/day; $\approx$ 0.08% fraud prevalence
Latency budget	$\leq$ 200 ms end-to-end (feature engineering + scoring + explanation)
Model performance	AUC-ROC: 0.974; Precision: 0.71; Recall: 0.83; F1: 0.77
Deployment mode	Substitution — automated blocking with optional analyst override

Table 1: System profile of the fraud detection use case.

Technique	Audit.	Trace.	Compr.	Action.	Fidelity	Debug.	Eff.
SHAP	3	4	3	3	5	5	4
LIME	2	3	4	4	4	3	3
Counterfactuals	2	3	5	5	4	3	3
Rule Extraction	5	5	3	2	4	4	2
Prototypes	2	2	5	4	3	3	3

Table 2: Intrinsic property vectors

\mathbf{p}_{t}

for each XAI family (scale 1–5). Colour coding: green

\geq 3.5

; yellow

\in[2.5,3.4]

; red

<2.5

Technique	$C^{\prime}_{t}$	Level	$U^{\prime}_{t}$	Level	$D^{\prime}_{t}$	Level
SHAP	3.91	High	3.30	Med.	4.70	High
LIME	2.76	Med.	4.40	High	3.50	High
Counterfactuals	2.76	Med.	5.00	High	3.50	High
Rule Extraction	5.00	High	2.86	Med.	3.80	High
Prototypes	2.30	Low	5.00	High	3.00	Med.

Table 3: Final ESS coordinates after contextual adjustment with substitution multipliers (

\gamma_{C}=1.15

\gamma_{U}=1.10

\gamma_{D}=1.00

) and qualitative level classification.

Technique	$\mathcal{U}_{t}$	$\mathcal{R}_{t}$	$\mathcal{U}/\mathcal{R}$	Eff. Score	Latency Fit
SHAP	3.82	0.25	15.3	4	$<$ 50 ms (✓)
LIME	3.56	0.33	10.8	3	$\sim$ 80 ms (✓)
Counterfactuals	3.80	0.33	11.5	3	$\sim$ 100 ms ( $\approx$ )
Rule Extraction	3.90	0.50	7.8	2	Offline only (×)
Prototypes	3.52	0.33	10.7	3	$\sim$ 60 ms (✓)

Table 4: Multi-objective selection results. The

\mathcal{U}/\mathcal{R}

ratio provides an efficiency-adjusted measure of stakeholder value. Latency assessments are based on typical TreeExplainer, LIME tabular, DiCE, and

k

-NN runtimes on tabular data of this dimensionality.

Explainable Artificial Intelligence

Explainable artificial intelligence develops ways to make AI decisions understandable, testable, and useful to people.

See all 326 papers

A Human-Centered Validation of the Explainability-Performance Coefficient
Jul 31, 2026Christian Oliva, Luis F. Lago-FernándezExplainable Artificial Intelligence Explainability Evaluation
Grupo de Neurocomputaci´on Biol´ogica, Departamento de Ingenier´ıa Inform´atica, Escuela Polit´ecnica Superior, Universidad Aut´onoma de Madrid, Spain
XALIGN: A Human-centric Visual Analytics Approach to Understanding AI Explanations
Oct 22, 2026Bahavathy Kathirgamanathan, Gennady Andrienko, Natalia Andrienko +1Explainable Artificial Intelligence Explainability Evaluation
University of Cologne, Cologne, Germany · Fraunhofer IAIS, Sankt Augustin, Germany · City St George’s Institute of London, London, UK
Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps
Apr 23, 2025Felix Kares, Timo Speith, Hanwei Zhang +1Saliency Map Evaluation Explainability Evaluation
Saarland University, Germany · University of Bayreuth, Germany · University of Freiburg, Germany

Jul 31, 2026J/K move · Enter open · S save

A Human-Centered Validation of the Explainability-Performance Coefficient

Christian Oliva, Luis F. Lago-Fernández

Grupo de Neurocomputaci´on Biol´ogica, Departamento de Ingenier´ıa Inform´atica, Escuela Polit´ecnica Superior, Universidad Aut´onoma de Madrid, Spain

Explainable Artificial Intelligence Explainability Evaluation

Open paper AskPDF arXiv

The rapid adoption of deep learning models in high-risk domains has intensified the need for trustworthy Explainable Artificial Intelligence (XAI). However, objectively evaluating explanation fidelity and aligning XAI metrics with human-centered understanding remain critical open challenges. In this work, we propose a model-agnostic metric, the EPC score, which is an extension of the Explainability-Performance Coefficient (EPC), that quantifies explanation quality by explicitly balancing the trade-off between feature selection sparsity and preserved model performance. Through an empirical validation across tabular, text, and image modalities, we show that the EPC score effectively uncovers operational dependencies among network activations, data dimensionality, and explainer performance. Furthermore, we validate the EPC score against independent human-based explanations, proving that higher EPC scores strongly align with human lexical sentiment judgments and spatial visual annotations.

Abstract

Explainable Artificial Intelligence

Explore similar work

A Human-Centered Validation of the Explainability-Performance Coefficient

XALIGN: A Human-centric Visual Analytics Approach to Understanding AI Explanations

Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps