Neural Network Interpretability

Ask about this topic

Overview

Neural network interpretability asks what a model has learned, which internal computations support its predictions, and whether those computations are understandable and reliable. Post-hoc methods include saliency and attribution, concept-based explanations, counterfactuals, and formal logical explanations; mechanistic work uses probes, sparse autoencoders, circuit analysis, and interventions on neurons or features. Another approach builds interpretability into models through sparse or additive architectures, prototypes, symbolic rules, and concept bottlenecks, with evaluations spanning image, language, time-series, and scientific tasks. A central challenge is showing that an explanation is both human-meaningful and faithful to the model: readable features or sparse circuits alone do not establish causal importance, and methods can vary in robustness and reliability across settings.

Latest papers

CardsList

333 papers37 papers in the last four weeks, up 23% on the four weeks before. 0.2% of all new papers.

  1. The Query Channel: Information-Theoretic Limits of Masking-Based Explanations

    Apr 17, 2026Erciyes Karakaya, Ozgur ErcetinFeature AttributionExplainable Artificial Intelligence

    *Department of Electrical and Computer Engineering, University of Maryland, College Park, USA · †Faculty of Engineering and Natural Sciences, Sabanci University, Turkiye

  2. ProtoTTA: Prototype-Guided Test-Time Adaptation

    Apr 16, 2026Mohammad Mahdi Abootorabi, Parvin Mousavi, Purang Abolmaesumi +1Prototype-Based LearningTest-Time Adaptation

    University of British Columbia · Queen’s University · Vector Institute

  3. Improving Sparse Autoencoder with Dynamic Attention

    Apr 16, 2026Dongsheng Wang, Jinsen Zhang, Dawei Su +1Sparse AutoencodersNeural Network Interpretability

    College of Computer Science and Software Engineering, Shenzhen University, China

  4. Zero-Ablation Overstates Register Content Dependence in DINO Vision Transformers

    Apr 15, 2026Felipe Parodi, Jordan Matelsky, Melanie SegadoVision TransformerNeural Network Interpretability

    University of Pennsylvania · Johns Hopkins Applied Physics Laboratory

  5. Geometric Routing Enables Causal Expert Control in Mixture of Experts

    Apr 15, 2026Ivan Ternovtsii, Yurii BilakMixture-of-Experts InterpretabilityMixture-of-Experts Expert Specialization

    Department of Software Systems, Uzhhorod National University, Narodna sq. 3, Uzhhorod, Ukraine, 88000 · HengeBytes

  6. Hardware-Efficient Neuro-Symbolic Networks with the Exp-Minus-Log Operator

    Apr 15, 2026Eymen IpekNeurosymbolic LearningEfficient Neural Network Inference

    Institute of Automotive Engineering, Graz University of Technology, Austria

  7. The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts

    Apr 13, 2026Thomas Walker, Ahmed Imtiaz Humayun, Randall Balestriero +1Neural Network InterpretabilityMechanistic Interpretability

    Rice University · Google Research · Brown University

  8. THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture

    Apr 13, 2026Augustus Haoyang LiNeural Network InterpretabilityCompositional Generalization

    Irvine Valley College Irvine, CA, USA

  9. Visualising the Attractor Landscape of Neural Cellular Automata

    Apr 12, 2026James Stovold, Mia-Katrin Kvalsund, Harald Michael Ludwig +2Neural Cellular AutomataNeural Network Interpretability

    University of York, UK · University of Oslo, Norway · Machine Learning Research Unit, TU Wien, Austria +2

  10. On-board Telemetry Monitoring in Autonomous Satellites: Challenges and Opportunities

    Apr 9, 2026Lorenzo Capelli, Leandro de Souza Rosa, Maurizio De Tommasi +8Interpretable Anomaly DetectionFault Detection

    University of Bologna, Italy · Thales Alenia Space Italia, Rome, Italy · European Space Agency (ESA-ESTEC), Netherlands

  11. Neural-Symbolic Knowledge Tracing: Injecting Educational Knowledge into Deep Learning for Responsible Learner Modelling

    Apr 9, 2026Danial Hooshyar, Gustav Šír, Yeongwook Yang +6Knowledge TracingAI in Education

    School of Digital Technologies Tallinn University, Estonia · Faculty of Information Technology University of Jyväskylä Finland · Faculty of Education and Psychology University of Jyväskylä Finland +7

  12. LINE: LLM-based Iterative Neuron Explanations for Vision Models

    Apr 9, 2026Vladimir Zaigrajew, Michał Piechota, Gaspar Sekula +2Neuron-Level InterpretabilityNeural Network Interpretability

    Centre for Credible AI Warsaw University of Technology · Warsaw University of Technology Warsaw, Poland · Centre for Credible AI, Warsaw University of Technology University of Warsaw, Poland

  13. ConceptTracer: Interactive Analysis of Concept Saliency and Selectivity in Neural Representations

    Apr 8, 2026Ricardo Knauer, Andre Beinrucker, Erik RodnerNeural Network InterpretabilityNeuron-Level Interpretability

    KI-Werkstatt, University of Applied Sciences Berlin, Berlin, Germany

  14. Explaining Neural Networks in Preference Learning: a Post-hoc Inductive Logic Programming Approach

    Apr 8, 2026Daniele Fossemò, Filippo Mignosi, Giuseppe Placidi +3Neural Network InterpretabilityPreference Learning

    Department of Information Engineering, Computer Science and Mathematics, University of L’Aquila, Italy · ICAR-CNR, Palermo, Italy · Department of Life, Health and Environmental Sciences, University of L’Aquila, Italy +2

  15. Beyond Semantics: Disentangling Information Scope in Sparse Autoencoders for CLIP

    Apr 7, 2026Yusung Ro, Jaehyun Choi, Junmo KimNeural Network InterpretabilityVLM Interpretability

    KAIST · Korea AI Safety Institute, ETRI

  16. PH-KAN: Port-Hamiltonian Kolmogorov-Arnold Network

    Apr 7, 2026Achraf El Messaoudi, Karim Cherifi, Yann Le Gorrec +1Nonlinear System IdentificationPort-Hamiltonian Systems

    UMLP, ENSMM, FEMTO-ST · Université Marie et Louis Pasteur, SUPMICROTECH, CNRS, institut FEMTO-ST, F-25000 Besançon, France

  17. Functional Similarity Metric for Neural Networks: Overcoming Parametric Ambiguity via Activation Region Analysis

    Apr 4, 2026Kutomanov HennadiiRepresentational Similarity AnalysisNeural Network Interpretability

  18. Interpretable Fuzzy Modeling Reveals Population-Level Representation Differences in P300 Brain Computer Interfaces Across Neurodivergent and Neurotypical Cohorts

    Apr 4, 2026Xiaowei Jiang, Sudong Shang, Adrian Wilkinson +4Brain-Computer InterfacesEEG Decoding

    Computational Intelligence and Brain-Computer Interfaces lab, Australian AI Institute, Faculty of Engineering and Information Technology, University of Technology Sydney, 2007, Australia · Department of Management, Griffith Business School, 4215, Australia · Department of Neuroscience, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA +2

  19. The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Level

    Apr 2, 2026Jeremy Herbst, Stefan Wermter, Jae Hee LeeMixture-of-Experts InterpretabilityMixture-of-Experts Models

    Department of Informatics, University of Hamburg, Hamburg, Germany.

  20. PhiNet: Speaker Verification with Phonetic Interpretability

    Apr 2, 2026Yi Ma, Shuai Wang, Tianchi Liu +1Speaker VerificationNeural Network Interpretability

    Department of Electrical and Computer Engineering, National University of Singapore, 119077, Singapore · School of Intelligence Science and Technology, Nanjing University, Suzhou, 215163, China · Shenzhen Loop Area Institute, Shenzhen, China +2

  21. Real-Time Explanations for Tabular Foundation Models

    Mar 31, 2026Luan Borges Teodoro Reis Sena, Francisco Galuppo AzevedoNeural Network InterpretabilityTabular Foundation Models

    Kunumi Institute, Belo Horizonte, Minas Gerais, Brazil · Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

  22. From Density Matrices to Phase Transitions in Deep Learning: Spectral Early Warnings and Interpretability

    Mar 31, 2026Max Hennick, Guillaume CorlouerPhase Transitions in Neural NetworksNeural Network Training Dynamics

  23. Geometry-aware similarity metrics for neural representations on Riemannian and statistical manifolds

    Mar 30, 2026N Alex Cayco-Gajic, Arthur PellegrinoNeural Representation GeometryRepresentational Similarity Analysis

    Gatsby Unit, University College London

  24. Post-hoc Self-explanation of CNNs

    Mar 30, 2026Ahcène Boubekki, Line H. ClemmensenPrototype-Based ExplanationNeural Network Interpretability

    Department of Mathematical Sciences, University of Copenhagen, Denmark

  25. TTE-CAM: Self-Explainable Class Activation Maps for Pretrained Black-Box CNNs

    Mar 27, 2026Kerol Djoumessi, Philipp BerensExplainable Image ClassificationNeural Network Interpretability

    Hertie Institute for AI in Brain Health, University of Tübingen, Germany · Tübingen AI Center, University of Tübingen, Germany

  26. End-to-end Feature Alignment: A Simple CNN with Intrinsic Class Attribution

    Mar 26, 2026Parniyan Farvardin, David ChapmanNeural Network InterpretabilityExplainable Image Classification

    University of Miami

  27. Vision Hopfield Memory Networks for Image Recognition

    Mar 26, 2026Jianfeng Wang, Amine M'Charrak, Luk Koska +6Memory-Augmented Neural NetworksHopfield Networks

    Department of Computer Science, University of Oxford, United Kingdom · Faculty of Informatics, Vienna University of Technology, Austria

  28. Select, Hypothesize and Verify: Towards Verified Neuron Concept Interpretation

    Mar 26, 2026ZeBin Ji, Yang Hu, Xiuli Bi +2Neuron-Level InterpretabilityNeural Network Interpretability

    Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, China · Jinan Inspur Data Technology Co., Ltd., Jinan, China

  29. A Practical Guide Towards Interpreting Time-Series Deep Clinical Predictive Models: A Reproducibility Study

    Mar 25, 2026Yongda Fan, John Wu, Andrea Fitzpatrick +3Explainable Time Series ForecastingNeural Network Interpretability

    University of Illinois Urbana-Champaign, Champaign, IL 61801, USA · PyHealth · University of Illinois College of Medicine, Chicago, IL 60612, USA

  30. From Weights to Concepts: Data-Free Interpretability of CLIP via Singular Vector Decomposition

    Mar 25, 2026Francesco Gentile, Nicola Dall'Asen, Francesco Tonini +3Transformer InterpretabilityVLM Interpretability

    University of Trento · University of Pisa · Fondazione Bruno Kessler