Explainability Evaluation

Follow this topic Ask about this topic

Explainability evaluation studies whether an explanation reflects a model’s behavior and whether people can use it to understand, verify, or act on a decision. Common approaches measure faithfulness with feature removal, perturbations, counterfactuals, or causal interventions; assess stability across inputs, models, and data shifts; and compare explanations with ground-truth annotations or domain knowledge. Human studies test comprehension, decision performance, trust, and error detection, while benchmarks increasingly combine these outcomes with computational cost and robustness. No single metric works across tasks: scores can depend on architecture and perturbation choices, and mathematical quality or fluent explanations do not reliably predict human understanding or better decisions.

Latest papers

322 papers24 papers in the last four weeks, down 35% on the four weeks before. 0.1% of all new papers.

Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models
Feb 18, 2026Melkamu Abay Mersha, Jugal KalitaTransformer Interpretability Gradient-Based Attribution
College of Engineering and Applied Science, University of Colorado Colorado Springs, Colorado Springs, 80918, CO, USA
A unified framework for evaluating the robustness of machine-learning interpretability for prospect risking
Feb 16, 2026Prithwijit Chowdhury, Ahmad Mustafa, Mohit Prabhushankar +1Explainability Evaluation Explainable Artificial Intelligence
Center for Energy and Geo Processing (CeGP), Omni Lab for Intelligent Visual Engineering and Science (OLIVES), School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA.
A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model Behavior
Feb 2, 2026Harry Mayne, Justin Singh Kang, Dewi Gould +3Faithfulness of Language Model Explanations LLM Interpretability
University of Oxford · University of California, Berkeley · Independent +2
Explanation Multiplicity in SHAP: Characterization and Assessment
Jan 19, 2026Hyunseung Hwang, Seungeun Lee, Lucas Rosenblatt +2SHAP Feature Attribution Feature Attribution
KAIST, Daejeon, Republic of Korea · New York University, New York, NY, USA
Explainability of Complex AI Models with Correlation Impact Ratio
Jan 10, 2026Poushali Sengupta, Rabindra Khadka, Sabita Maharjan +5Feature Attribution Interpretable ML
Institute of Informatics, University of Oslo, Oslo, Norway · Department of Computer Science, Oslo Metropolitan University, Oslo, Norway · Aalborg University, Denmark
EvoXplain: When Machine Learning Models Agree on Predictions but Disagree on Why -- Measuring Mechanistic Multiplicity Across Training Runs
Dec 23, 2025Chama BensmailCancer Genomics Model Selection
University of Hertfordshire
TxSum: User-Centered Ethereum Transaction Understanding with Micro-Level Semantic Grounding
Dec 7, 2025Zifan Peng, Jingyi Zheng, Yule Liu +8Financial Services Grounded Text Generation
1Hong Kong University of Science and Technology (Guangzhou) · 3Independent Researcher · University of North Texas +2
CID: Measuring Feature Importance Through Counterfactual Distributions
Nov 19, 2025Eddie Conti, Álvaro Parafita, Axel BrandoFeature Attribution Explainability Evaluation
TAIES Group, HPES Lab, Barcelona Supercomputing Center (BSC)
Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations
Nov 15, 2025Eunkyu Park, Wesley Hanwen Deng, Vasudha Varadarajan +4VLM Reasoning Multimodal CoT Reasoning
Seoul National University · Human-Computer Interaction Institute, Carnegie Mellon University · Language Technologies Institute, Carnegie Mellon University
Distinguishing Performance From Competence in Evaluations of Humanlike Abstract Reasoning
Oct 2, 2025Claas Beger, Ryan Yi, Shuhao Fu +5Language Model Reasoning Evaluation Explainability Evaluation
Santa Fe Institute, New Mexico, USA · Sandia National Laboratories, New Mexico, USA · Advanced Micro Devices Inc., Texas, USA
Activation-Deactivation: A General Framework for Robust Post-hoc Explainable AI
Oct 1, 2025Akchunya Chanchal, David A. Kelly, Hana ChocklerExplainable Artificial Intelligence Explanation Stability
Department of Informatics King’s College London Strand, WC2R 2LS, London, UK
Explainable Information Processing in Particle Swarm Optimization through Landscape and Search Behavior Analysis
Sep 8, 2025Nitin Gupta, Bapi Dutta, Anupam YadavMetaheuristic Optimization Explainability Evaluation
Department of Mathematics and Computing Dr. B. R. Ambedkar National Institute of Technology Jalandhar, Jalandhar - 144008, INDIA · Department of Computer Science University of Jaén, Jaén, SPAIN
Ultra Strong Machine Learning: LLM-Generated Explanations Do Not Yet Suffice for Teaching Humans Active Learning Strategy
Aug 31, 2025Lun Ai, Johannes Langer, Ute Schmid +1Active Learning Human-in-the-Loop AI
Department of Computing, Imperial College London, UK · Faculty Information Systems and Applied Computer Science, University of Bamberg, Germany
Grad-CAM for Visualizing Attention Regions of PCA and SVM Layers in Convolutional Neural Networks
Aug 16, 2025Yuto Omae, Hirotaka TakahashiGradient-Based Attribution Convolutional Neural Networks
Nihon University
Rule2Text: A Framework for Generating and Evaluating Natural Language Explanations of Knowledge Graph Rules
Aug 14, 2025Nasim Shirvani-Mahdavi, Chengkai LiLLM Evaluation Explainability Evaluation
University of Texas at Arlington, Arlington, Texas 76019, USA
SurgXBench: Explainable Vision-Language Model Benchmark for Surgery
May 16, 2025Jiajun Cheng, Xianwu Zhao, Sainan Liu +5VLM Evaluation Surgical Video Understanding
Arizona State University · Intel Labs · University of California, Merced +2
Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps
Apr 23, 2025Felix Kares, Timo Speith, Hanwei Zhang +1Saliency Map Evaluation Explainability Evaluation
Saarland University, Germany · University of Bayreuth, Germany · University of Freiburg, Germany
Less is More: Efficient Black-box Attribution via Minimal Interpretable Subset Selection
Apr 1, 2025Ruoyu Chen, Siyuan Liang, Jingzhi Li +4Feature Attribution VLM Interpretability
Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China · University of Chinese Academy of Sciences, Beijing 100049, China · College of Computing and Data Science, Nanyang Technological University, 639798, Singapore +4
Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations
Mar 17, 2025Noah Y. Siegel, Nicolas Heess, Maria Perez-Ortiz +1Faithfulness of Language Model Explanations LLM Interpretability
Google DeepMind · Centre for AI, University College London · Imperial College London +1
VisionLogic: Discovering and Grounding Decision-Relevant Visual Concepts
Mar 13, 2025Chuqin Geng, Yuhe Jiang, Li Zhang +5Explainable Artificial Intelligence Concept-Based Explanations
School of Computer Science, McGill University, Montreal, Canada · Department of Computer Science, University of Toronto, Toronto, Canada
Causal Explanations for Image Classifiers
Nov 13, 2024Hana Chockler, David A. Kelly, Daniel Kroening +1Actual Causality Explainable Image Classification
King’s College London, London, UK · Amazon.com, Inc.,, USA · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE +1
CHAOSMINING: Benchmarking Post-Hoc Attribution with Sparse Informative Features in High Dimensions
Jun 17, 2024Ge Shi, Fangyi Liu, Ziwen Kan +1Gradient-Based Attribution Feature Attribution
Department of Computer Science University of California, Davis

Feb 18, 2026J/K move · Enter open · S save

Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models

Melkamu Abay Mersha, Jugal Kalita

College of Engineering and Applied Science, University of Colorado Colorado Springs, Colorado Springs, 80918, CO, USA

Transformer Interpretability Gradient-Based Attribution Integrated Gradients Explainability Evaluation

Open paper AskPDF arXiv

Transformer models achieve state-of-the-art performance across domains and tasks, yet their deeply layered representations make their predictions difficult to interpret. Existing explainability methods rely on final-layer attributions, capture either local token-level attributions or global attention patterns without unification, and lack context-awareness of inter-token dependencies and structural components. They also fail to capture how relevance evolves across layers and how structural components shape decision-making. To address these limitations, we proposed the Context-Aware Layer-wise Integrated Gradients (CA-LIG) Framework, a unified hierarchical attribution framework that computes layer-wise Integrated Gradients within each Transformer block and fuses these token-level attributions with class-specific attention gradients. This integration yields signed, context-sensitive attribution maps that capture supportive and opposing evidence while tracing the hierarchical flow of relevance through the Transformer layers. We evaluate the CA-LIG Framework across diverse tasks, domains, and transformer model families, including sentiment analysis and long and multi-class document classification with BERT, hate speech detection in a low-resource language setting with XLM-R and AfroLM, and image classification with Masked Autoencoder vision Transformer model. Across all tasks and architectures, CA-LIG provides more faithful attributions, shows stronger sensitivity to contextual dependencies, and produces clearer, more semantically coherent visualizations than established explainability methods. These results indicate that CA-LIG provides a more comprehensive, context-aware, and reliable explanation of Transformer decision-making, advancing both the practical interpretability and conceptual understanding of deep neural models.