Neural Network Interpretability

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 450% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 203

All topics

The Neglected Baseline in Model Interpretation
May 21, 2026Yongjin Cui, Xiaohui FanGradient-Based Attribution Integrated Gradients
Zhejiang University
Capability $\neq$ Interpretability: Human Interpretability of Vision Foundation Models
May 19, 2026Julien Colin, Lore Goetschalckx, Nuria Oliver +1Visual Representation Learning Vision Foundation Models
ELLIS Alicante, Spain · imec, Leuven, Belgium · Brown University, USA
FPED: A Functional-Network Prior-Guided Mixture-of-Experts Framework for Interpretable Brain Decoding
May 19, 2026Yudan Ren, Pengcheng Shi, Zihan Ma +2Neural Decoding Mixture-of-Experts Inference
School of Electronic Information (School of Artificial Intelligence), Northwest University, Xi’an 710127, China
Toy Combinatorial Interpretability Models Reveal Lottery Tickets in Early Feature Space
May 18, 2026Alon Bebchuk, Nir ShavitNeural Representation Geometry Neural Network Interpretability
Tel-Aviv University · MIT · Red Hat AI
Structured Neural Marked Point Processes for Interpretable Event Interaction Modeling
May 17, 2026Zhitong Xu, Qiwei Yuan, Yinghao Chen +2Temporal Point Processes Neural Network Interpretability
Kahlert School of Computing, University of Utah · Celonis AI
Beyond Accuracy: Robustness, Interpretability and Expressiveness of EEG Foundation Models
May 17, 2026Urban Širca, Maryam Alimardani, Stefanos Zafeiriou +1EEG Decoding Neural Network Interpretability
Vrije Universiteit Amsterdam · Imperial College London
Zero-Shot Faithful Textual Explanations via Directional-Derivative Influence on Predictions
May 16, 2026Toshinori Yamauchi, Hiroshi Kera, Kazuhiko KawamotoGradient-Based Attribution Neural Network Interpretability
Chiba University · National Institute of Informatics
$α$ -TCAV: A Unified Framework for Testing with Concept Activation Vectors
May 15, 2026Ekkehard Schnoor, Jawher Said, Malik Tiomoko +2Concept-Based Explanations Neural Network Interpretability
Department of Computer Science Aalto University Espoo, Finland · Department of Artificial Intelligence Fraunhofer Heinrich Hertz Institute Berlin, Germany · Huawei Noah’s Ark Lab Huawei Technologies Paris, France +1
A Unified Non-Parametric and Interpretable Point Cloud Analysis via t-FCW Graph Representation
May 14, 2026Haijian Lai, Bowen Liu, Man Xu +43D Point Cloud Segmentation 3D Part Segmentation
Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR, China · University of Coimbra, CISUC/LASI, DEI · Macao Polytechnic University, Macao SAR, China
From Weight Perturbation to Feature Attribution for Explaining Fully Connected Neural Networks
May 14, 2026Thodoris Lymperopoulos, Denia KanellopoulouFeature Attribution Perturbation-Based Feature Attribution
NCSR Demokritos Athens, Greece
Towards Fine-Grained and Verifiable Concept Bottleneck Models
May 14, 2026Yingying Fang, Haijie Xu, Shuang Wu +2Concept Bottleneck Models Neural Network Interpretability
Bioengineering Department and Imperial-X, Imperial College London, London, UK · Thoughtworks AI Labs, Singapore
Rethinking Layer Relevance in Large Language Models Beyond Cosine Similarity
May 13, 2026Cristian Hinostroza, Rodrigo Toro Icarte, Christ Devia +4LLM Pruning LLM Interpretability
1Pontificia Universidad Cat´olica de Chile · 2National Center for Artificial Intelligence (CENIA) · 3Universidad de Chile
AttnGen: Attention-Guided Saliency Learning for Interpretable Genomic Sequence Classification
May 13, 2026Rayhaneh Shabani Nia, Ali KarkehabadiNeural Network Interpretability Attention Mechanisms
University of California, Davis California, USA
Supervised Deep Multimodal Matrix Factorization for Interpretable Brain Network Analysis
May 13, 2026Amjad Seyedi, Lifang He, Songlin Zhao +2Neural Network Interpretability Multimodal Graph Learning
Dept. of Mathematics & Operational Research University of Mons, Mons, Belgium · Dept. of Computer Science & Engineering Lehigh University, Bethlehem, PA, USA · Dept. of Industrial & Systems Engineering Lehigh University, Bethlehem, PA, USA
Understanding Generalization through Decision Pattern Shift
May 13, 2026Huiqi Deng, Yibo Li, Quanshi Zhang +3Neural Network Generalization Neural Network Interpretability
Xi’an Jiaotong University · Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory
FiTS: Interpretable Spiking Neurons via Frequency Selectivity and Temporal Shaping
May 13, 2026Jongmin Choi, Joon Son ChungNeural Network Interpretability Spiking Neural Networks
Korea Advanced Institute of Science and Technology (KAIST)
Feature Visualization Recovers Known Cortical Selectivity from TRIBE v2
May 13, 2026Stuart Bladon, Brinnae BentNeural Network Interpretability Visual Neuroscience
Duke University, Durham, NC, USA.
Native Explainability for Bayesian Confidence Propagation Neural Networks: A Framework for Trusted Brain-Like AI
May 12, 2026Georgios Makridis, Georgios Fatouros, John Soldatos +2Bayesian Neural Networks Explainable Artificial Intelligence
ExpertAI-Lux S.a r.l · University of Piraeus, Greece, Department of Digital Systems · University of Glasgow School of Computing Science
A Composite Activation Function for Learning Stable Binary Representations
May 12, 2026Seokhun Park, Choeun Kim, Kwanho Lee +3Concept Bottleneck Models Binary Neural Networks
Department of Statistics, Seoul National University · Department of Applied Mathematics, University of Twente
Interpretability Can Be Actionable
May 11, 2026Hadas Orgad, Fazl Barez, Tal Haklay +9Explainable Artificial Intelligence Neural Network Interpretability
Kempner Institute at Harvard University · University of Oxford · Martian +10
Probing Cross-modal Information Hubs in Audio-Visual LLMs
May 11, 2026Jihoo Jung, Chaeyoung Jung, Ji-Hoon Kim +1Cross-Modal Learning Multimodal Large Language Models
Department of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · The Graduate School of Advanced Imaging Science, Multimedia & Film, Chung-Ang University, Seoul, Republic of Korea
Deep Arguing
May 11, 2026Adam Gould, Francesca ToniNeural Network Interpretability Computational Argumentation
Department of Computing Imperial College London
Causal Explanations from the Geometric Properties of ReLU Neural Networks
May 11, 2026Hector Woods, Philippa Ryan, Rob AlexanderNeural Network Interpretability Causal Explanation
Department of Computer Science University of York York, United Kingdom
E-TCAV: Formalizing Penultimate Proxies for Efficient Concept Based Interpretability
May 11, 2026Hasib Aslam, Muhammad Ali Chattha, Muhammad Taha Mukhtar +3Concept-Based Explanations Neural Network Interpretability
1National University of Sciences and Technology (NUST) · 2German Research Centre for Artificial Intelligence (DFKI)
Scaling Vision Models Does Not Consistently Improve Localisation-Based Explanation Quality
May 11, 2026Mateusz Cedro, Marcin ChlebusNeural Network Interpretability Explainability Evaluation
University of Antwerp, Antwerp, 2000, Belgium · University of Warsaw, Warsaw, 00-241, Poland
Explainability of Recurrent Neural Networks for Enhancing P300-based Brain-Computer Interfaces
May 11, 2026Christian Oliva, Vinicio Changoluisa, Francisco B Rodríguez +1Brain-Computer Interfaces EEG Decoding
Grupo de Neurocomputación Biológica, Departamento de Ingeniería Informática, Escuela Politécnica Superior, Universidad Autónoma de Madrid, c/ Francisco Tomas y Valiente, 11, Madrid, 28049, Madrid, Spain · Grupo de Investigación en Electrónica y Telemática, Universidad Politécnica Salesiana, Campus el Girón, Ave 12 de Octubre 24-22, Quito, 170143, Quito, Ecuador
A Game Theoretic Free Energy Analysis of Higher Order Synergy in Attention Heads of Large Language Models
May 10, 2026Djamel BouchaffraLLM Pruning Attention Head Analysis
DAVID Lab, University of Paris-Saclay, UVSQ Campus, 78035 Versailles, France
From Mechanistic to Compositional Interpretability
May 9, 2026Ward Gauderis, Thomas Dooms, Steven T. Homer +2Mechanistic Interpretability Neural Network Interpretability
Vrije Universiteit Brussel, Brussels, Belgium · Independent · UK AI Security Institute, London, United Kingdom.
ProDG: Prototypes for Data-Free Generative Post-Hoc Explainability
May 9, 2026Piotr Borycki, Magdalena Trędowicz, Jacek Tabor +2Explainable Artificial Intelligence Neural Network Interpretability
Jagiellonian University · IDEAS Research Institute · Centre for Credible AI +1
Biological Plausibility and Representational Alignment of Feedback Alignment in Convolutional Networks
May 8, 2026Jake Lance, Larry KieuRepresentational Similarity Analysis Backpropagation
University of Toronto

May 21, 2026, cs.CVJ/K move · Enter open · S save

The Neglected Baseline in Model Interpretation

Yongjin Cui, Xiaohui Fan

Zhejiang University

Gradient-Based Attribution Integrated Gradients Feature Attribution Neural Network Interpretability Explainability Evaluation

Open paper AskPDF arXiv

We observe that existing model interpretation methods generally ignore the baseline, and such neglect often results in imprecise or even incorrect interpretation. In this paper, we reformulate the task of model interpretation and the interpretation principles for model interpretation results to demonstrate the importance of the baseline. For the first time, we unify gradient-based methods, Integrated Gradients (IG), and Taylor expansion, clarify the relationships among the three, and explicitly identify the corresponding baseline for each method. This may have a significant impact on the further performance improvement of some gradient-based schemes. On this basis, we analyze the flaws and errors in related model interpretation methods (IG, LayerCAM, ODAM, Difference Map). We advocate evaluating the quality of model interpretation results precisely through the attribution error between the attribution result and the attribution target, rather than adopting flawed evaluation methods, such as those based on marginal-effect or the assumption of perfect model performance. We revise IG and develope a model interpretation method with a clear and reasonable baseline, achieving better results. Our method supports model interpretation based on features from any layer. Interpretation based on features from different layers are all reasonable, and the differences among these results reflect varying degrees of feature extraction at different feature extraction stages.

Momentum

Related topics

Latest papers 203