Explainability Evaluation

Ask about this topic

Overview

Explainability evaluation studies whether an explanation reflects a model’s behavior and whether people can use it to understand, verify, or act on a decision. Common approaches measure faithfulness with feature removal, perturbations, counterfactuals, or causal interventions; assess stability across inputs, models, and data shifts; and compare explanations with ground-truth annotations or domain knowledge. Human studies test comprehension, decision performance, trust, and error detection, while benchmarks increasingly combine these outcomes with computational cost and robustness. No single metric works across tasks: scores can depend on architecture and perturbation choices, and mathematical quality or fluent explanations do not reliably predict human understanding or better decisions.

Latest papers

CardsList

322 papers24 papers in the last four weeks, down 35% on the four weeks before. 0.1% of all new papers.

  1. DocXEval: A Systematic Evaluation of Explainability in Multimodal Document Understanding

    Oct 27, 2026Saifullah Saifullah, David Dembinsky, Stefan Agne +2Multimodal Model InterpretabilityExplainability Evaluation

    Smart Data and Knowledge Services (SDS), German Research Center for Artificial Intelligence (DFKI), Trippstadter Str. 122, Kaiserslautern, 67663, Germany. · Department of Computer Science, University of Kaiserslautern-Landau (RPTU), Erwin-Schrödinger-Str. 52, Kaiserslautern, 67663, Germany. · DeepReader GmbH, Kaiserslautern, 67663, Germany.

  2. Counterfactual Profile Perturbations for Recommendation Explanations: A Reproducibility Study

    Oct 26, 2026Meysam Varasteh, Elizabeth McKinnie, Robin BurkeExplainable RecommendationCounterfactual Explanations

    Department of Computer Science, University of Colorado, Boulder, Boulder, Colorado, USA · Department of Information Science, University of Colorado, Boulder, Boulder, Colorado, USA

  3. Beyond feature attribution: Recovering latent interaction structures in machine learning models via localized CBFI

    Oct 26, 2026Sejong OhFeature AttributionFeature Interaction Modeling

    Department of Software Science, Dankook University, Gyeonggi-do, Republic of Korea

  4. XALIGN: A Human-centric Visual Analytics Approach to Understanding AI Explanations

    Oct 22, 2026Bahavathy Kathirgamanathan, Gennady Andrienko, Natalia Andrienko +1Explainable Artificial IntelligenceExplainability Evaluation

    University of Cologne, Cologne, Germany · Fraunhofer IAIS, Sankt Augustin, Germany · City St George’s Institute of London, London, UK

  5. Rethinking SAE Evaluation: An Atomic Interpretable Unit Framework Reveals Hidden Polysemanticity and Redundancy

    Oct 21, 2026Riteng Zhang, Xiaoqian WangSparse AutoencodersExplainability Evaluation

    Purdue University

  6. Engram Memory Network: Brain-Inspired Prototype Explanations

    Oct 20, 2026Hyunjun Kim, Myoung Hoon HaPrototype-Based ExplanationExplainable Image Classification

    Department of Electrical and Computer Engineering, Seoul National University, Seoul, South Korea · Center for Neuroscience-inspired AI, KAIST, Daejeon, South Korea

  7. AnomalyJev: Towards Evaluating Image Anomaly Explanations via Claim-Level Verification

    Oct 20, 2026Qizhou Wang, Jiawen Zhu, Guansong Pang +1Explainability EvaluationInterpretable Anomaly Detection

    The University of Melbourne · Singapore Management University

  8. Explaining Chess and Go in Natural Language

    Oct 16, 2026Ori Press, Alexander GolubevConcept-Based ExplanationsExplainability Evaluation

    Nebius AI

  9. FRACTION: Fraud Explanation Generation via Agentic Graph Context Navigation

    Oct 13, 2026Cecelia Soh, Jun Hu, Yuan Li +6Faithfulness of Language Model ExplanationsLLMs for Graph Learning

    National University of Singapore · Hefei University of Technology · ByteDance Inc. +1

  10. ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning

    Oct 12, 2026Aotian Chen, Frazier N. Baker, Reza Averly +5Scientific ReasoningExplainability Evaluation

    The Ohio State University · Adelaide University

  11. Integrating Feature Understandability into Feature Selection for Human-Centred Tabular Explanations

    Oct 12, 2026Nicola Rossberg, Bennett Kleinberg, Barry O'Sullivan +2Explainable Artificial IntelligenceFeature Selection

    School of Computer Science and Information Technology, University College Cork, Ireland · Centre for Research Training in Artificial Intelligence, University College Cork, Ireland · Department of Methodology and Statistics, Tilburg University +1

  12. VisExMEM: Visual Evidence-Based Explainable Multimodal Evaluation Metric

    Oct 12, 2026Subhadeep Roy, Christoph Leiter, Steffen EgerVLM EvaluationImage Captioning Evaluation

    Natural Language Learning & Generation Lab, University of Technology Nuremberg

  13. Unreliable Narrator Framing as a Cost-effective Mechanism for Satire Explanation of Australian and American Satirical News

    Oct 9, 2026Kanay Gupta, David Y. Liu, Aditya JoshiExplainability Evaluation

    University of New South Wales, Australia

  14. How to train your model organism

    Oct 7, 2026Xilin Wang, David Bau, Byron C. WallaceMechanistic InterpretabilityExplainability Evaluation

    Northeastern University Boston, MA 02115, USA

  15. Does Explainability Survive Data Drift?

    Oct 4, 2026Samuel OzechiExplainability Evaluation

  16. MOXIE: Discovering Alternative Explanations for Biomedical Image Classifiers

    Oct 3, 2026Abiha Tahsin Chowdhury, Rahul DubeyExplainable Image ClassificationExplainable Medical Image Analysis

    Department of Computer Science, Missouri State University, Springfield, MO, USA

  17. Beyond Trained Models: Compiling GNNs for a Sound Explainer Benchmark

    Oct 2, 2026Steve Azzolin, Francesco Paolo Nerini, Stefano Teso +4GNN ExplainabilityExplainability Evaluation

    University of Trento Trento, Italy

  18. MEA: A Reward-Driven Multi-Agent System for Faithful Model Explanations

    Oct 1, 2026Yuyang Cheng, Raghav Kaushik Ravi, Srivarshinee Sridhar +3Explainable Artificial IntelligenceFaithfulness of Language Model Explanations

    University of Virginia · Vellore Institute of Technology · Indian Institute of Technology, Patna

  19. Confidence-Controlled XAI Auditing for Pedestrian Detection under Domain Shift

    Oct 1, 2026Ruben Dario Florez-ZelaExplainability EvaluationBlack-Box Model Auditing

    Universidad Nacional de San Agustín de Arequipa, Arequipa, Peru

  20. A Comparative Explainability Framework for DeBERTa-v3 in Zero-Shot Medical Abstract Classification

    Oct 1, 2026Javier Diaz Esteban-Herreros, David Muñoz-Valero, Raquel Martínez-España +2Transformer InterpretabilityPerturbation-Based Feature Attribution

    Department of Technologies and Information Systems, Universidad de Castilla-La Mancha, Avenida Carlos III, s/n, Toledo, 45071, Spain · University of Murcia, Avenida Teniente Flomesta, 5, Murcia, 3003, Spain · Murcian Bio-Health Institute (IMIB-Arrixaca), Pabellón Docente del Hospital Clínico Universitario Virgen de la Arrixaca, Murcia, 3120, Spain

  21. Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability

    Oct 1, 2026Chuqin Geng, Li Zhang, Haolin Ye +3Circuit DiscoveryMechanistic Interpretability

    University of Toronto · McGill University

  22. MCIR: A Feature Dependence-Aware Explainability Method with Reliability Guarantees

    Oct 1, 2026Poushali Sengupta, Sabita Maharjan, Frank Eliassen +2Feature AttributionExplainability Evaluation

    Department of Informatics, University of Oslo Oslo, Norway · Department of Electronic Systems, Aalborg University Aalborg, Denmark

  23. WOMBAT: Whitebox Oracle for Molecular Benchmarking and Attribution Testing

    Sep 30, 2026Dominik Matuszek, Bartosz Zieliński, Tomasz Danel +1Gradient-Based AttributionExplainability Evaluation

    Jagiellonian University, Faculty of Mathematics and Computer Science · Jagiellonian University, Doctoral School of Exact and Natural Sciences · Jagiellonian University, Jagiellonian Center for Artificial Intelligence +2

  24. Equally Good, Yet Different: Benchmarking Rashomon sets in AutoML packages

    Sep 29, 2026Katarzyna Woźnica, Katarzyna Rogalska, Zuzanna Sieńko +1Automated Machine LearningExplainability Evaluation

    Warsaw University of Technology · Systems Research Institute, Polish Academy of Sciences · Eskisehir Technical University

  25. Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?

    Sep 28, 2026Li Zhang, Chuqin Geng, Mark Zhang +4Faithfulness of Language Model ExplanationsMechanistic Interpretability

    University of Toronto · McGill University · Tsinghua University

  26. Verifying the Linear Representation Hypothesis: How Interpretable Are Vision SAEs?

    Sep 28, 2026Teodor Chiaburu, Franz Motzkus, Frank Haußer +1Linear Representation HypothesisSparse Autoencoders

    Fraunhofer SIT · ATHENE National Research Center for Applied Cybersecurity · AUMOVIO +3

  27. A Unifying Framework of Concept-based Explainable AI with Completeness Guarantees

    Sep 28, 2026Vojtěch Kůr, Adam Kukučka, Tomáš Brázdil +1Concept-Based ExplanationsNeural Network Interpretability

    Masaryk University, Faculty of Informatics, Brno, Czechia

  28. Controllable GNN Explanations via Multi-Metric Preference Selection

    Sep 26, 2026Rachit Verma, Yashraj J. Deshmukh, Anirban DasguptaGNN ExplainabilityExplanation Stability

    Indian Institute of Technology Gandhinagar

  29. Robust to Which Model Change? A Unified Evaluation of Robust Counterfactual Explanations

    Sep 25, 2026Marcin Kostrzewa, Maciej ZiębaCounterfactual EvaluationCounterfactual Explanations

    Department of Artificial Intelligence Wrocław University of Science and Technology

  30. A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods

    Sep 24, 2026Miquel Miró-Nicolau, Francesco Spinnato, Riccardo GuidottiExplainability EvaluationExplanation Evaluation

    Universitat de les Illes Balears, Cra. de Valldemossa, km 7.5., Palma, 07122, Spain · Uiniversity of Pisa, Largo B. Pontecrovo, 3, 56127, Pisa, Italy · ISTI-CNR, Via G. Moruzzi, 1, Pisa, 56127, Italy