Explainability Evaluation
Overview
Explainability evaluation studies whether an explanation reflects a model’s behavior and whether people can use it to understand, verify, or act on a decision. Common approaches measure faithfulness with feature removal, perturbations, counterfactuals, or causal interventions; assess stability across inputs, models, and data shifts; and compare explanations with ground-truth annotations or domain knowledge. Human studies test comprehension, decision performance, trust, and error detection, while benchmarks increasingly combine these outcomes with computational cost and robustness. No single metric works across tasks: scores can depend on architecture and perturbation choices, and mathematical quality or fluent explanations do not reliably predict human understanding or better decisions.
Latest papers
350 papers24 papers in the last four weeks, down 8% on the four weeks before. 0.2% of all new papers.
Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations
Surrogate Modeling and Explainable Artificial Intelligence for Complex Systems: A Workflow for Automated Simulation Exploration
Distinguishing Performance From Competence in Evaluations of Humanlike Abstract Reasoning
Activation-Deactivation: A General Framework for Robust Post-hoc Explainable AI
Explainable Information Processing in Particle Swarm Optimization through Landscape and Search Behavior Analysis
Ultra Strong Machine Learning: LLM-Generated Explanations Do Not Yet Suffice for Teaching Humans Active Learning Strategy
Grad-CAM for Visualizing Attention Regions of PCA and SVM Layers in Convolutional Neural Networks
PCA-Grad-CAM'', a method for visualizing attention regions in PCA feature vectors, and SVM-Grad-CAM'', a method for visualizing attention regions in an SVM classifier layer. Solving a closed-form Jacobian problem comprising partial derivatives from the last convolutional layer to the PCA and/or SVM layers is necessary to complete the proposed methods analytically. In this paper, we present the exact closed-form Jacobian and visualization results of the proposed methods applied to several major datasets. In addition, the insertion and deletion metrics, which are major evaluation metrics in explainable AI, were applied to PCA- and SVM-Grad-CAM. The results suggest that the proposed method can successfully visualize the attention regions of PCA and SVM.