Post Hoc Interpretability

Post-hoc interpretability aims to explain the decisions of already-trained machine learning models, particularly deep neural networks (DNNs), by analyzing their internal workings after training is complete. Current research focuses on developing and evaluating methods like LIME, Grad-CAM, SHAP, and novel approaches tailored to specific model architectures (e.g., graph neural networks, convolutional neural networks) and data types (e.g., images, text, time series). Improving the reliability and robustness of these methods, including addressing issues like perturbation artifacts and the faithful representation of feature interactions, is crucial for building trust and facilitating the adoption of AI in high-stakes applications such as healthcare and nuclear safety.

Papers

October 17, 2024

Reproducibility study of "LICO: Explainable Models with Language-Image Consistency"
Luan Fletcher, Robert van der Klis, Martin Sedláček, Stefan Vasilev, Christos Athanasiadis
Inherent Interpretability Strong Consistency Interpretability Method Explainable Model Interpretability Research Reproducibility Study Post Hoc Interpretability

August 30, 2024

Controllable Edge-Type-Specific Interpretation in Multi-Relational Graph Neural Networks for Drug Response Prediction
Xiaodi Li, Jianfeng Gui, Qian Gao, Haoyuan Shi, Zhenyu Yue
Graph Neural Network Inherent Interpretability Edge Inference Drug Response Prediction Interpretable Graph Post Hoc Interpretability

August 23, 2024

July 29, 2024

Revisiting the robustness of post-hoc interpretability methods
Jiawen Wei, Hugues Turbé, Gianmarco Mengaldo
Native Robustness Post Hoc Interpretability Hoc Interpretability

June 21, 2023

Feature Interactions Reveal Linguistic Structure in Language Models
Jaap Jumelet, Willem Zuidema
Language Model Feature Interaction Interpretability Research Linguistic Structure Post Hoc Interpretability

May 3, 2023

Explaining Language Models' Predictions with High-Impact Concepts
Ruochen Zhao, Shafiq Joty, Yongjie Wang, Tan Wang
Language Model Human Prediction High Impact Concept Concept Based Post Hoc Interpretability

March 2, 2023

Feature Perturbation Augmentation for Reliable Evaluation of Importance Estimators in Neural Networks
Lennart Brocki, Neo Christopher Chung
Neural Network Importance Estimation Perturbation Augmentation Local Interpretability Post Hoc Interpretability Perturbation Artifact

August 3, 2022

Post-hoc Interpretability based Parameter Selection for Data Oriented Nuclear Reactor Accident Diagnosis System
Chengyuan Li, Meifu Li, Zhifang Qiu
Raw Data Diagnostic Model Diagnosis System Parameter Selection Diagnostic Accuracy Post Hoc Interpretability Consistency Based Diagnosis

June 10, 2022

Explaining Neural Networks without Access to Training Data
Sascha Marton, Stefan Lüdtke, Christian Bartelt, Andrej Tschalzev, Heiner Stuckenschmidt
Neural Network Training Data Information Access Global Interpretability Post Hoc Interpretability Interpretable Function \Mathcal{I}$ Net

April 25, 2022

Integrating Prior Knowledge in Post-hoc Explanations
Adulam Jeyasothy, Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Marcin Detyniecki
Counterfactual Explanation Prior Knowledge Interpretability Method eXplainable Artificial Intelligence Post Hoc Explanation Post Hoc Interpretability

March 6, 2022

Fidelity of Interpretability Methods and Perturbation Artifacts in Neural Networks
Lennart Brocki, Neo Christopher Chung
Neural Network Deep Neural Network Supervised ImageNet Interpretability Method Fidelity Reward Fidelity Metric Post Hoc Interpretability Perturbation Artifact

November 28, 2021

Multicriteria interpretability driven Deep Learning
Marco Repetto
Deep Learning Deep Learning Model Inherent Interpretability Nonlinear Prediction Post Hoc Interpretability