Interpretable Transformer

Interpretable Transformers aim to enhance the transparency and understandability of Transformer models, renowned for their power but often criticized for their "black box" nature. Current research focuses on developing methods to visualize and interpret attention mechanisms, employing techniques like analyzing attention maps, modeling Transformer dynamics using partial differential equations, and designing architectures that inherently promote interpretability, such as decoder-only transformers and those incorporating hierarchical clustering. This work is significant because it addresses the critical need for trust and accountability in AI systems, enabling better debugging, improved model design, and facilitating the application of Transformers in sensitive domains like healthcare and neuroscience.

Papers

August 18, 2024

A Unified Framework for Interpretable Transformers Using PDEs and Information Theory
Yukun Zhang
Transformer Architecture Unified Framework Information Theory Transformer Performance Attention Based Transformer Interpretable Transformer

July 30, 2024

Interpretable Pre-Trained Transformers for Heart Time-Series Data
Harry J. Davies, James Monsen, Danilo P. Mandic
Pre Trained Model Decoder Only Transformer Beat Tracking Cardiac Model Interpretable Transformer

June 26, 2024

Clustering in pure-attention hardmax transformers and its role in sentiment analysis
Albert Alcalde, Giovanni Fantuzzi, Enrique Zuazua
Natural Language Processing Sentiment Analysis Integral Role Softmax Attention Interpretable Transformer

June 24, 2024

CausalFormer: An Interpretable Transformer for Temporal Causal Discovery
Lingbai Kong, Wengen Li, Hanchen Yang, Yichao Zhang, Jihong Guan, Shuigeng Zhou
Causal Discovery Method Causal Transformer Interpretable Transformer Temporal Causal Discovery

April 9, 2024

Does Transformer Interpretability Transfer to RNNs?
Gonçalo Paulo, Thomas Marshall, Nora Belrose
Contrastive Learning Transformer Language Model Traditional RNNs Recurrent Neural Network Architecture Interpretable Transformer

November 7, 2023

A Simple Interpretable Transformer for Fine-Grained Image Classification and Analysis
Dipanjyoti Paul, Arpita Chowdhury, Xinqi Xiong, Feng-Ju Chang, David Carlyn, Samuel Stevens, Kaiya L. Provost, Anuj Karpatne, Bryan Carstens, Daniel Rubenstein, Charles Stewart, Tanya Berger-Wolf, Yu Su, Wei-Lun Chao
General Analysis Image Classification Detection Transformer Fine Grained Image Classification Encoder Decoder Transformer Interpretable Transformer

May 6, 2023

Transformer-Based Hierarchical Clustering for Brain Network Analysis
Wei Dai, Hejie Cui, Xuan Kan, Ying Guo, Sanne van Rooij, Carl Yang
Brain Network Hierarchical Clustering Brain Network Analysis Interpretable Transformer Joint Cluster

July 12, 2022

eX-ViT: A Novel eXplainable Vision Transformer for Weakly Supervised Semantic Segmentation
Lu Yu, Wei Xiang, Juan Fang, Yi-Ping Phoebe Chen, Lianhua Chi
Vision Transformer Transformer Model Weakly Supervised Semantic Segmentation Novel Vision Transformer ViT LoS V2X Interpretable Transformer