Explainability Evaluation

Follow this topic Ask about this topic

Explainability evaluation studies whether an explanation reflects a model’s behavior and whether people can use it to understand, verify, or act on a decision. Common approaches measure faithfulness with feature removal, perturbations, counterfactuals, or causal interventions; assess stability across inputs, models, and data shifts; and compare explanations with ground-truth annotations or domain knowledge. Human studies test comprehension, decision performance, trust, and error detection, while benchmarks increasingly combine these outcomes with computational cost and robustness. No single metric works across tasks: scores can depend on architecture and perturbation choices, and mathematical quality or fluent explanations do not reliably predict human understanding or better decisions.

Latest papers

322 papers24 papers in the last four weeks, down 35% on the four weeks before. 0.1% of all new papers.

Using Large Language Models and Knowledge Graphs to Improve the Interpretability of Machine Learning Models in Manufacturing
Apr 17, 2026Thomas Bayer, Alexander Lohr, Sarah Weiß +2Manufacturing LLM Grounding
University of Applied Sciences Ravensburg-Weingarten, Germany
Ranking XAI Methods for Head and Neck Cancer Outcome Prediction
Apr 17, 2026Baoqiang Ma, Djennifer K. Madzia-Madzou, Rosa C. J. Kraaijveld +1Gradient-Based Attribution Cancer Survival Prediction
Image Sciences Institute, University Medical Center Utrecht
Applied Explainability for Large Language Models: A Comparative Study
Apr 15, 2026Venkata Abhinandan KancharlaLLM Interpretability Explainability Evaluation
NRI Institute of Technology
Post-Hoc Uncertainty-Aware Explanations for Deployed Power Quality Disturbance Classifiers via Laplace Approximation
Apr 15, 2026Yinsong Chen, Samson S. Yu, Kashem M. MuttaqiExplainable Artificial Intelligence Feature Attribution
School of Engineering, Deakin University, Melbourne, 3216, VIC, Australia · ARC Training Centre in Energy Technologies for Future Grids, School of Engineering, University of Wollongong, Wollongong, 2522, NSW, Australia
Shapley Value-Guided Adaptive Ensemble Learning for Explainable Financial Fraud Detection with U.S. Regulatory Compliance Validation
Apr 14, 2026Mohammad Nasir Uddin, Md Munna AzizFinancial Fraud Detection Ensemble Learning
Visual Data Analyst and Applied AI Researcher, Westcliff University, Irvine, CA, USA · Data Analyst and Applied AI Researcher, Westcliff University, Irvine, CA, USA
A Two-Stage LLM Framework for Accessible and Verified XAI Explanations
Apr 14, 2026Georgios Mermigkis, Dimitris Metaxakis, Marios Tyrovolas +4Explainable Artificial Intelligence Faithfulness of Language Model Explanations
Department of Computer Engineering and Informatics, University of Patras, Patras, Greece · Archimedes Unit, Athena Research Center, Athens, Greece · Department of Informatics and Telecommunications, University of Ioannina, Arta, Greece +1
Robust Explanations for User Trust in Enterprise NLP Systems
Apr 13, 2026Guilin Zhang, Kai Zhao, Jeffrey Friedman +3Explainable Artificial Intelligence LLM Interpretability
Workday AI Research
From Attribution to Action: A Human-Centered Application of Activation Steering
Apr 13, 2026Tobias Labarta, Maximilian Dreyer, Katharina Weitz +2Explainable Artificial Intelligence Activation Steering
Fraunhofer Heinrich-Hertz-Institut · Technische Universität Berlin · BIFOLD – Berlin Institute for the Foundations of Learning and Data
Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?
Apr 13, 2026Ziqian Zhong, Aashiq Muhamed, Mona T. Diab +2Mechanistic Interpretability Explainability Evaluation
Carnegie Mellon University
GLEaN: A Text-to-image Bias Detection Approach for Public Comprehension
Apr 10, 2026Bochu Ding, Brinnae Bent, Augustus WendellDemographic Bias in T2I Models Explainable Artificial Intelligence
Duke University
Explainable Human Activity Recognition: A Unified Review of Concepts and Mechanisms
Apr 10, 2026Mainak Kundu, Catherine Chen, Rifatul Islam +2Explainable Artificial Intelligence Human Activity Recognition
Department of Electrical Engineering, University of South Florida, Tampa, FL 33620 USA · Department of Computer Engineering, California Polytechnic State University, San Luis Obispo, CA 93407 USA
Quantifying Explanation Consistency: The C-Score Metric for CAM-Based Explainability in Medical Image Classification
Apr 9, 2026Kabilan Elangovan, Daniel TingExplainability Evaluation Saliency Map Evaluation
Singapore Health Services, Singapore · Singapore Eye Research Institute, Singapore
A comparative analysis of machine learning models in SHAP analysis
Apr 8, 2026Justin Lin, Julia FukuyamaSHAP Feature Attribution Shapley Value Attribution
Department of Mathematics, Indiana University · Department of Statistics, Indiana University
Non-identifiability of Explanations from Model Behavior in Deep Networks of Image Authenticity Judgments
Apr 8, 2026Icaro Re Depaolini, Uri HassonExplainability Evaluation Feature Attribution
Center for Mind/Brain Sciences, The University of Trento, Trento, Italy
Empirical Characterization of Rationale Stability Under Controlled Perturbations for Explainable Pattern Recognition
Apr 6, 2026Abu Noman Md Sakib, Zhensen Wang, Merjulah Roby +1Explanation Stability Explainability Evaluation
Department of Computer Science, The University of Texas at San Antonio, San Antonio, TX, 78249, USA · Department of Mechanical, Aerospace, and Industrial Engineering, The University of Texas at San Antonio, San Antonio, TX, 78249, USA
Overcoming the "Impracticality" of RAG: Proposing a Real-World Benchmark and Multi-Dimensional Diagnostic Framework
Apr 3, 2026Kenichirou Narita, Siqi Peng, Taku Fukui +3Retrieval-Augmented Generation RAG Evaluation
Artificial Intelligence Laboratories, Fujitsu Limited
Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?
Apr 2, 2026Kamalasankari Subramaniakuppusamy, Jugal GajjarExplainability Evaluation Perturbation-Based Feature Attribution
The George Washington University Washington D.C.
TSQueryBench: LLM-as-a-Judge for Time Series Explanations
Apr 2, 2026Preetham Sivalingam, Murari Mandal, Dhruv Kumar +1Explanation Evaluation Explainability Evaluation
BITS Pilani, Pilani, India · KIIT & Birla AI Labs, Bhubaneshwar, India · BITS Pilani & Birla AI Labs, Pilani, India +1
Reproducible, Explainable, and Effective Evaluations of Agentic AI for Software Engineering
Apr 1, 2026Jingyue Li, André StorhaugAI Agent Evaluation Agent Evaluation
Norwegian University of Science and Technology, Trondheim, Norway
Using Learning Theories to Evolve Human-Centered XAI: Future Perspectives and Challenges
Apr 1, 2026Karina Cortinas-Lorenzo, Gavin DohertyExplainable Artificial Intelligence Human-Centered AI
Trinity College Dublin Dublin 2, Dublin, Ireland
Uncertainty Gating for Cost-Aware Explainable Artificial Intelligence
Mar 31, 2026Georgii Mikriukov, Grégoire Montavon, Marina M. -C. HöhneExplainable Artificial Intelligence Explainability Evaluation
Leibniz Institute for Agricultural Engineering and Bioeconomy (ATB), Potsdam, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany · Charité – Universitätsmedizin Berlin, Berlin, Germany +1
A User-Centric Analysis of Explainability in AI-Based Medical Image Diagnosis
Mar 31, 2026Julia Wagner, Tim SchlippeExplainable Medical Image Analysis Explainability Evaluation
IU International University of Applied Sciences, Germany
Structural Compactness as a Complementary Criterion for Explanation Quality
Mar 31, 2026Mohammad Mahdi Mesgari, Jackie Ma, Wojciech Samek +2Explainability Evaluation Explanation Evaluation
Fraunhofer Heinrich Hertz Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany +1
Coherent Without Grounding, Grounded Without Success: Observability and Epistemic Failure
Mar 30, 2026Camilo Chacón SartoriLLM Grounding Explainability Evaluation
Institut Català de Nanociència i Nanotecnologia (ICN2). · Artificial Intelligence Research Institute (IIIA-CSIC), Barcelona, Spain.
Concerning Uncertainty -- A Systematic Survey of Uncertainty-Aware XAI
Mar 27, 2026Helena Löfström, Tuwe Löfström, Anders Hjort +1Explainable Artificial Intelligence Uncertainty Quantification
Jönköping International Business School, Gjuterigatan 5, 553 18 Jönköping, Sweden · Jönköping School of Engineering, Gjuterigatan 5, 553 18 Jönköping, Sweden · Eiendomsverdi AS, Nedre Slottsgate 8, 0157 Oslo, Norway +2
Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understanding
Mar 26, 2026Gregor Baer, Chao Zhang, Isel Grau +1Explainable Artificial Intelligence Explainability Evaluation
Information Systems Group, Eindhoven University of Technology · Human-Technology Interaction Group, Eindhoven University of Technology
A Practical Guide Towards Interpreting Time-Series Deep Clinical Predictive Models: A Reproducibility Study
Mar 25, 2026Yongda Fan, John Wu, Andrea Fitzpatrick +3Explainable Time Series Forecasting Neural Network Interpretability
University of Illinois Urbana-Champaign, Champaign, IL 61801, USA · PyHealth · University of Illinois College of Medicine, Chicago, IL 60612, USA
No Single Metric Tells the Whole Story: A Multi-Dimensional Evaluation Framework for Uncertainty Attributions
Mar 25, 2026Emily Schiller, Teodor Chiaburu, Marco Zullich +1Explainability Evaluation Uncertainty Quantification
XITASO GmbH IT & Software Solutions, Augsburg, Germany · The Artificial Intelligence and Cognitive Load Research Lab, University College Cork · Berliner Hochschule für Technik, Berlin, Germany +2
Toward Faithful Segmentation Attribution via Benchmarking and Dual-Evidence Fusion
Mar 23, 2026Abu Noman Md Sakib, OFM Riaz Rahman Aranya, Kevin Desai +1Feature Attribution Explainability Evaluation
The University of Texas at San Antonio
Assessing Model-Agnostic XAI Methods against EU AI Act Explainability Requirements
Mar 19, 2026Francesco Sovrano, Giulia Vilone, Michael LognoulAI Regulatory Compliance Explainable Artificial Intelligence
University of Italian-Speaking Switzerland (USI) · Analog Devices International · CRIDS, NADI, University of Namur

Apr 17, 2026J/K move · Enter open · S save

Using Large Language Models and Knowledge Graphs to Improve the Interpretability of Machine Learning Models in Manufacturing

Thomas Bayer, Alexander Lohr, Sarah Weiß, Bernd Michelberger+1

University of Applied Sciences Ravensburg-Weingarten, Germany

Manufacturing LLM Grounding Explainable Artificial Intelligence Explainability Evaluation

Open paper AskPDF arXiv

Explaining Machine Learning (ML) results in a transparent and user-friendly manner remains a challenging task of Explainable Artificial Intelligence (XAI). In this paper, we present a method to enhance the interpretability of ML models by using a Knowledge Graph (KG). We store domain-specific data along with ML results and their corresponding explanations, establishing a structured connection between domain knowledge and ML insights. To make these insights accessible to users, we designed a selective retrieval method in which relevant triplets are extracted from the KG and processed by a Large Language Model (LLM) to generate user-friendly explanations of ML results. We evaluated our method in a manufacturing environment using the XAI Question Bank. Beyond standard questions, we introduce more complex, tailored questions that highlight the strengths of our approach. We evaluated 33 questions, analyzing responses using quantitative metrics such as accuracy and consistency, as well as qualitative ones such as clarity and usefulness. Our contribution is both theoretical and practical: from a theoretical perspective, we present a novel approach for effectively enabling LLMs to dynamically access a KG in order to improve the explainability of ML results. From a practical perspective, we provide empirical evidence showing that such explanations can be successfully applied in real-world manufacturing environments, supporting better decision-making in manufacturing processes.