Explainability Evaluation

Follow this topic Ask about this topic

Momentum

27 papers in the last four weeks, up 50% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 218

All topics

ObserverBench: Testing Mechanistic Estimates for Intervention and Control
Sep 2, 2026Vijay ErramilliMechanistic Interpretability AI Safety Evaluation
Taking the Whys Seriously: Limitations of Counterfactual Explanations in Justification and Recourse
Aug 31, 2026Mattia Cerrato, Otto Sahlgren, Xenia HeilmannCounterfactual Explanations AI Accountability
Johannes Gutenberg University Mainz, Germany · University of Tampere, Finland
Assessing Alignment and Stability of Feature Importance Explanations via Weight of Evidence
Aug 31, 2026Eddie Conti, Claudio Daka, Álvaro Parafita +3Feature Attribution SHAP Feature Attribution
Barcelona Supercomputing Center, Barcelona, Spain · University of Florence, Florence, Italy · SMARTEST Research Center, eCampus University, Novedrate, Italy +2
Automated Testing of LLM-Based Post Hoc Explainers Using Model Checking as an Oracle
Aug 31, 2026Dennis Gross, Helge SpiekerLLM Evaluation Faithfulness of Language Model Explanations
Institut für Kommunikations- und Prüfungsforschung gGmbH · Simula Research Laboratory, Oslo, Norway
XQDT: eXplainable and Quantitative Data-Text Alignment Metric with Feedback Signals
Aug 30, 2026Kun Efimov-Zhang, Yifei Song, Claire GardentAutomated Evaluation Explainability Evaluation
CNRS/LORIA and Université de Lorraine
Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Delivered
Aug 29, 2026Aryo Pradipta Gema, Neel Rajani, Rohit Saxena +2CoT Faithfulness LLM Auditing
University of Edinburgh · University of Edinburgh, Miniml.AI
Are Concept Bottleneck Models Effective as Decision-Support Systems?
Aug 26, 2026Alessandro Bogani, Nicola Debole, Emanuele Marconato +3Concept Bottleneck Models Human-AI Decision Making
DISI, University of Trento, Italy · CIMeC, University of Trento, Italy
Decomposition of Evidence, Contradiction, and Fragility in Perturbation Responses
Aug 13, 2026Lei YouFeature Attribution Perturbation-Based Feature Attribution
Technical University of Denmark
When Explanations Betray Backdoors: Black-Box Auditing for Language Model Classifiers
Aug 12, 2026Yang Liu, Ran ZouLLM Backdoor Attacks Backdoor Detection
Department of Statistics and Operations Research, University of North Carolina at Chapel Hill · Department of Statistics, University of California, Irvine
CAS: A Causal Attribution Score for Local and Global Explainable Artificial Intelligence
Aug 12, 2026Michael Georgiades, Charalambia VarnavaFeature Attribution Causal Attribution
Department of Computer Science Neapolis University Pafos Cyprus · CaSToRC The Cyprus Institute Nicosia, Cyprus
Entropy-Centric Explainable AI for Remote Sensing Image Segmentation
Aug 11, 2026Ali Saleh, Abdul Karim Gizzini, Mohamad Ghassany +1Remote Sensing Image Segmentation Explainable Artificial Intelligence
Faculty of Engineering, Lebanese University, Beirut, Lebanon · EFREI, EFREI Research Lab, F-94800 Villejuif, France. · University of Paris-Est Cr´eteil (UPEC), LISSI/TincNET, F-94400, Vitry-sur-Seine, France. +1
Who Are You Explaining To? A Multi-Agent System for Audience-Aware XAI Narratives
Aug 11, 2026Francesco Musicco, Danilo Danese, Giuseppe Fasano +3Multi-Agent Systems Explainability Evaluation
Politecnico di Bari, Italy
Conversational versus Dashboard Explainable AI for UAV Intrusion Detection: An Empirical Study of Operator Trust and Reliance
Aug 11, 2026Cong Chi Nguyen, Trang Mai Xuan, Vu-Duc Ngo +3Explainable Artificial Intelligence Human-in-the-Loop Evaluation
A2I Lab, Phenikaa School of Computing, Phenikaa University, Hanoi, Vietnam · MobiFone HighTech Center, MobiFone Corporation, Hanoi, Vietnam · Business AI Lab, College of Technology, National Economics University, Vietnam
Beyond Detection Accuracy: Measuring Explanation Cost, Stability, and Utility for Resource-Aware IoT Intrusion Detection
Aug 11, 2026Abdurrahman TolaySHAP Feature Attribution IoT Intrusion Detection
Independent Researcher, Istanbul, Türkiye
How Simple Can It Get? From Interpretable Equations to Readable Rules for Financial Decision Making
Aug 10, 2026Adia Lumadjeng, Ilker Birbil, Erman AcarInterpretable ML Explainability Evaluation
University of Amsterdam Amsterdam, the Netherlands
Right Answer, Wrong Heat: Explanation-Aware Evaluation and Thermal-Grounded Feedback for MLLMs on Infrared Images
Aug 10, 2026Yongsong Huang, Xiaofeng Liu, Tomo Miyazaki +2Vision-Language Models Vision-Language Grounding
Tohoku University · Yale University
Backward Compatibility in Tree-Based Explanations and Enhanced CART Algorithm
Aug 9, 2026Hirofumi SuzukiExplanation Stability Decision Tree Learning
Fujitsu Limited Kawasaki, Kanagawa, Japan
Accurate Ensembles, Fragile Narratives: Multi-Scale Stacking and a Fidelity Audit of LLM-Generated Explanations for Credit Risk
Aug 8, 2026Gregorius Reynaldi Pratama, Kuo-Kun TsengFeature Attribution Faithfulness of Language Model Explanations
School of Computer Science and Technology Harbin Institute of Technology, Shenzhen Shenzhen, China
Explanation Stability of Test-Time Adaptation in Computational Pathology: A Large-Scale Benchmark
Aug 7, 2026R. G. Bahumanya, Harshith V. M., Shreyank N. Gowda +1Computational Pathology Test-Time Adaptation
Department of Computer Science and Engineering, R.V. College of Engineering, Bengaluru, India · School of Computer Science, University of Nottingham, Nottingham, United Kingdom
Challenges in Evaluating Explanation Methods for Static and Evolving Data
Aug 6, 2026Jerzy StefanowskiExplainable Artificial Intelligence Concept Drift
Poznan University of Technology, Institute of Computing Sciences, Poland
Beyond Feature Importance: A Comparative Analysis of Pattern Detection Methods in Cluster Interpretation
Aug 6, 2026Benjamin Connor, Anna Jurek-Loughrey, Lu Bai +1Clustering Unsupervised Clustering
School of EEECS Queen’s University Belfast 18 Malone Road, Belfast, BT9 5AF
Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs
Aug 3, 2026Sathiyamohan Nishankar, Nethmi Pathirana, Pubudu Sanjeewani +2Transformer Interpretability Feature Attribution
aFaculty of Computing, Sabaragamuwa University of Sri Lanka, Sri Lanka · bFaculty of Engineering, University of Moratuwa, Katubedda, Sri Lanka · cSchool of Computing Technologies, RMIT University, Melbourne, Australia +3
Measuring Explainer Stability via Attribution Separability
Aug 3, 2026Eddie Conti, Álvaro Parafita, Axel BrandoFeature Attribution Explanation Stability
Barcelona Supercomputing Center, Barcelona, Spain
Understanding Online Failure Prediction in Linux Through Complementary Multi-View Explainability
Aug 1, 2026Diogo Dória, João R. CamposFault Detection Causal Interpretability
University of Coimbra, CISUC/LASI Department of Informatics Engineering Coimbra, Portugal
A Human-Centered Validation of the Explainability-Performance Coefficient
Jul 31, 2026Christian Oliva, Luis F. Lago-FernándezExplainable Artificial Intelligence Explainability Evaluation
Grupo de Neurocomputaci´on Biol´ogica, Departamento de Ingenier´ıa Inform´atica, Escuela Polit´ecnica Superior, Universidad Aut´onoma de Madrid, Spain
Contrastive Concept Importance: Explaining Pairwise Class Decisions Through Automatically Extracted Concept Representations
Jul 30, 2026Roel Visser, Isaac Roberts, Barbara HammerFeature Attribution Pairwise Comparison
Bielefeld University, Inspiration 1, 33619 Bielefeld, Germany
AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations
Jul 28, 2026Javier Irigoyen, Roberto Daza, Francisco Jurado +5Generative AI in Education Explainability Evaluation
BiometricsAI, Universidad Autónoma de Madrid (UAM), Spain · GHIA, Universidad Autónoma de Madrid (UAM), Spain · Universidad de Las Palmas de Gran Canaria (ULPGC), Spain
Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review
Jul 27, 2026Zhenhan Gao, Marvin Muñoz Barón, Umm-e Habiba +2Explainable Artificial Intelligence Automated Code Review
Technical University of Munich, Heilbronn, Germany · University of Hohenheim, Hohenheim, Germany
Beyond Local Inspection: Global, Guideline-Grounded Evaluation of Post-hoc XAI Methods for ECG Classification
Jul 27, 2026Nils Gumpfer, Michael Guckert, Samuel Sossalla +2Feature Attribution Explainable Artificial Intelligence
Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany · Technische Hochschule Mittelhessen, University of Applied Sciences, Friedberg (Hesse), Germany · Department of Internal Medicine I, Cardiology, Justus-Liebig-University Giessen, Giessen, Germany
Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations
Jul 24, 2026Kazi Kamruzzaman Rabbi, Md. Zami Al Zunaed Farabe, M. Sohel RahmanFeature Attribution Neural Network Interpretability
BRAC University, Dhaka, Bangladesh · Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh

Sep 2, 2026, cs.LGJ/K move · Enter open · S save

ObserverBench: Testing Mechanistic Estimates for Intervention and Control

Vijay Erramilli

Mechanistic Interpretability AI Safety Evaluation Inference-Time Intervention Explainability Evaluation

Open paper AskPDF arXiv

Mechanistic interpretability is increasingly used to guide interventions such as activation steering, circuit removal, and safety monitoring. Yet an internal estimate that is accurate on average can still choose a poor action. We present ObserverBench, a benchmark framework for testing whether an internal estimator---an observer---is adequate for the intervention, control, or safety task it directs. Each task fixes the model, information boundary, allowed actions, decision rule, held-out cases, and loss. The benchmark reports estimation accuracy separately from the loss caused by the chosen action. Theory and experiments show why both are needed. In closed-loop control, observer errors matter at the starting point and along directions the allowed intervention can reach. On circuit-intervention tasks in GPT-2-small and Qwen2.5-7B, pairwise observers predict unseen effects more accurately without always choosing better actions; observers trained on action loss choose lower-loss actions. In safety triage, a score that perfectly separates violations can allocate a fixed intervention budget poorly when violations have different costs. Across Qwen2.5-7B, Gemma-2-9B-it, and prospectively frozen Qwen3.5-9B APPS tasks, AUROC can rank monitors differently from deployment loss, and the best information source changes across models. Sparse SAE readouts also trail their layer-matched dense controls on the reported Qwen panels, under disclosed activation-density or checkpoint mismatches. ObserverBench provides fixed task contracts, runnable baselines, and table-based submissions for evaluating interpretability methods through the actions they enable.

Momentum

Related topics

Latest papers 218