Explanation Map

Explanation maps are visual representations designed to enhance the interpretability of complex deep learning models, particularly in computer vision. Current research focuses on developing methods to generate these maps for various architectures, including convolutional neural networks and vision transformers, often leveraging gradients, attention mechanisms, or iterative integration techniques to highlight the input features most influential in model predictions. These methods are evaluated using both reference-based metrics (comparing to human gaze patterns) and no-reference metrics (assessing stability and consistency), aiming to improve the trustworthiness and user understanding of AI systems. The ultimate goal is to bridge the "interpretation gap" between human understanding and the inner workings of these powerful, but often opaque, models.

Papers

August 4, 2024

Human-inspired Explanations for Vision Transformers and Convolutional Neural Networks
Mahadev Prasad Panda, Matteo Tiezzi, Martina Vilas, Gemma Roig, Bjoern M. Eskofier, Dario Zanca
Convolutional Neural Network Vision Transformer Deep Convolutional Neural Network High Explainability Explanation Map Human Like Explanation

July 29, 2024

MaskInversion: Localized Embeddings via Optimization of Explainability Maps
Walid Bousselham, Sofian Chaybouti, Christian Rupprecht, Vittorio Ferrari, Hilde Kuehne
Foundation Model Optimization Purpose Vision Language Foundation Model Vision Language Alignment Explanation Map Local Embeddings Query Driven Mask Transformer Region Representation

April 4, 2024

LeGrad: An Explainability Method for Vision Transformers via Feature Formation Sensitivity
Walid Bousselham, Angie Boggust, Sofian Chaybouti, Hendrik Strobelt, Hilde Kuehne
Vision Transformer Computer Vision Attention Map Explainability Method Sensitivity Aware Explanation Map

February 22, 2024

TIE-KD: Teacher-Independent and Explainable Knowledge Distillation for Monocular Depth Estimation
Sangwon Choi, Daejune Choi, Duksu Kim
Knowledge Distillation Monocular Depth Estimation Depth Map Explanation Map Tie KD

December 7, 2023

Improved Visual Grounding through Self-Consistent Explanations
Ruozhen He, Paola Cascante-Bonilla, Ziyan Yang, Alexander C. Berg, Vicente Ordonez
Visual Grounding Gradient Based Visual Explanation Vision and Language Explanation Map

November 27, 2023

Utilizing Explainability Techniques for Reinforcement Learning Model Assurance
Alexander Tapley, Kyle Gatesman, Luis Robaina, Brett Bissey, Joseph Weissman
Deep Reinforcement Learning Explainable Reinforcement Learning Safe Policy Explanation Map

October 28, 2023

Visual Explanations via Iterated Integrated Attributions
Oren Barkan, Yehonatan Elisha, Yuval Asher, Amit Eshel, Noam Koenigstein
Vision Model Internal Representation Visual Explanation Explanation Map

October 25, 2023

Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models
Oren Barkan, Yuval Asher, Amit Eshel, Yehonatan Elisha, Noam Koenigstein
LeArning Abstract Deep Convolutional Neural Network High Explainability Vision Model Black Box Model Model Agnostic Post Hoc Explanation Explanation Map

October 23, 2023

Deep Integrated Explanations
Oren Barkan, Yehonatan Elisha, Jonathan Weill, Yuval Asher, Amit Eshel, Noam Koenigstein
Vision Model Intermediate Representation Explanation Map

July 18, 2023

R-Cut: Enhancing Explainability in Vision Transformers with Relationship Weighted Out and Cut
Yingjie Niu, Ming Ding, Maoning Ge, Robin Karlsson, Yuxiao Zhang, Kazuya Takeda
Vision Transformer Transformer Based High Explainability ImageNet Dataset Human Relationship Class Specific Explanation Map

January 18, 2023

TAME: Attention Mechanism Based Feature Fusion for Generating Explanation Maps of Convolutional Neural Networks
Mariano Ntrougkas, Nikolaos Gkalelis, Vasileios Mezaris
Convolutional Neural Network Attention Mechanism Hierarchical Attention Explanation Map Morphological Task

December 2, 2022

Evaluation of Explanation Methods of AI -- CNNs in Image Classification Tasks with Reference-based and No-reference Metrics
A. Zhukov, J. Benois-Pineau, R. Giot
Artificial Intelligence CNN Model Explanation Method Post Hoc Explanation Image Classification Task Reference Free Metric Machine Learning Paradigm Artificial Intelligence Decision Explanation Map

July 26, 2022

Explaining Deep Neural Networks for Point Clouds using Gradient-based Visualisations
Jawad Tayyub, Muhammad Sarmad, Nicolas Schönborn
Deep Neural Network Point Cloud Visual Explanation Explanation Map

Explanation Map

Papers

Human-inspired Explanations for Vision Transformers and Convolutional Neural Networks

MaskInversion: Localized Embeddings via Optimization of Explainability Maps

LeGrad: An Explainability Method for Vision Transformers via Feature Formation Sensitivity

TIE-KD: Teacher-Independent and Explainable Knowledge Distillation for Monocular Depth Estimation

Improved Visual Grounding through Self-Consistent Explanations

Utilizing Explainability Techniques for Reinforcement Learning Model Assurance

Visual Explanations via Iterated Integrated Attributions

Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models

Deep Integrated Explanations

R-Cut: Enhancing Explainability in Vision Transformers with Relationship Weighted Out and Cut

TAME: Attention Mechanism Based Feature Fusion for Generating Explanation Maps of Convolutional Neural Networks

Evaluation of Explanation Methods of AI -- CNNs in Image Classification Tasks with Reference-based and No-reference Metrics

Explaining Deep Neural Networks for Point Clouds using Gradient-based Visualisations