Neural Network Interpretability

Ask about this topic

Overview

Neural network interpretability asks what a model has learned, which internal computations support its predictions, and whether those computations are understandable and reliable. Post-hoc methods include saliency and attribution, concept-based explanations, counterfactuals, and formal logical explanations; mechanistic work uses probes, sparse autoencoders, circuit analysis, and interventions on neurons or features. Another approach builds interpretability into models through sparse or additive architectures, prototypes, symbolic rules, and concept bottlenecks, with evaluations spanning image, language, time-series, and scientific tasks. A central challenge is showing that an explanation is both human-meaningful and faithful to the model: readable features or sparse circuits alone do not establish causal importance, and methods can vary in robustness and reliability across settings.

Latest papers

CardsList

333 papers37 papers in the last four weeks, up 23% on the four weeks before. 0.2% of all new papers.

  1. Neural Network Models for Contextual Regression

    Mar 25, 2026Seksan Kiatsupaibul, Pakawan ChansiripasModular Neural NetworksNeural Network Interpretability

    Department of Statistics, Chulalongkorn University, Bangkok 10330, Thailand

  2. Minimal Sufficient Representations for Self-interpretable Deep Neural Networks

    Mar 25, 2026Zhiyao Tan, Liu Li, Huazhen LinNeural Network InterpretabilityRepresentation Learning

    Center of Statistical Research, School of Statistics, Southwestern University of Finance and Economics, Chengdu, China · School of Mathematics and Statistics, Wuhan University, Wuhan, China · New Cornerstone Science Laboratory, Southwestern University of Finance and Economics, Chengdu, China

  3. Latent Algorithmic Structure Precedes Grokking: A Mechanistic Study of ReLU MLPs on Modular Arithmetic

    Mar 24, 2026Anand SwaroopGrokkingNeural Network Interpretability

  4. Layer-wise Target Propagation: Efficient Component Attribution through Target Centric Propagation

    Mar 20, 2026Lasse Marten Jantsch, Dong-Jae Koh, Seonghyeon Lee +1Transformer InterpretabilityLLM Interpretability

    School of Computer Science and Engineering, Kyungpook National University, South Korea

  5. Ensembles-based Feature Guided Analysis

    Mar 20, 2026Federico Formica, Stefano Gregis, Andrea Rota +3Neural Network InterpretabilityFeature Attribution

    McMaster University, Canada · University of Bergamo, Italy

  6. Anatomical Heterogeneity in Transformer Language Models

    Mar 19, 2026Tomasz WietrzykowskiTransformerEfficient Language Model Training

    Independent Researcher, Wroclaw, Poland

  7. SINDy-KANs: Sparse identification of non-linear dynamics through Kolmogorov-Arnold networks

    Mar 19, 2026Amanda A. Howard, Nicholas Zolman, Bruno Jacob +2Nonlinear System IdentificationSparse Identification of Nonlinear Dynamics

    Pacific Northwest National Laboratory, Richland, WA 99354 · University of Washington, Seattle, WA 98195

  8. Towards Interpretable Framework for Neural Audio Codecs via Sparse Autoencoders: A Case Study on Accent Information

    Mar 18, 2026Shih-Heng Wang, Tiantian Feng, Aditya Kommineni +4Neural Audio CodecsSparse Autoencoders

    University of Southern California, USA

  9. ARTEMIS: A Neuro Symbolic Framework for Economically Constrained Market Dynamics

    Mar 18, 2026Rahul D RayQuantitative FinanceFinancial Forecasting

    Department of Electronics and Electrical Engineering, BITS Pilani, Hyderabad Campus

  10. Fundamental Limits of Neural Network Sparsification: Evidence from Catastrophic Interpretability Collapse

    Mar 18, 2026Dip Roy, Rajiv Misra, Sanjay Kumar SinghActivation SparsityNeural Network Interpretability

    Department of Computer Science and Engineering, Indian Institute of Technology, Patna, India · Department of Computer Science, Rajarshi School of Management Technology, Varanasi, India

  11. Sparse but not Simpler: A Multi-Level Interpretability Analysis of Vision Transformers

    Mar 16, 2026Siyu ZhangNeural Network InterpretabilityTransformer Interpretability

    University of Texas at Austin, Austin TX 78712, USA

  12. Building Trust in PINNs: Error Estimation through Finite Difference Methods

    Mar 16, 2026Aleksander Krasowski, René P. Klausen, Aycan Celik +3Physics-Informed MLPDE Solving

    Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute · Centre of eXplainable Artificial Intelligence, Technological University Dublin · Department of Electrical Engineering and Computer Science, Technische Universität Berlin +1

  13. GradCFA: A Hybrid Gradient-Based Counterfactual and Feature Attribution Explanation Algorithm for Local Interpretation of Neural Networks

    Mar 16, 2026Jacob Sanderson, Hua Mao, Wai Lok WooCounterfactual ExplanationsNeural Network Interpretability

    Department of Computer and Information Sciences, Northumbria University, Newcastle, UK, NE1 8ST

  14. L2GTX: From Local to Global Time Series Explanations

    Mar 13, 2026Ephrem Tibebe Mekonnen, Luca Longo, Lucas Rizzo +1Explainable Artificial IntelligenceInterpretable Time-Series Classification

    School of Computer Science, Technological University Dublin Dublin, Ireland · Artificial Intelligence and Cognitive Load Research Lab, University College Cork Cork, Ireland · School of Computer Science and Information Technology, University College Cork Cork, Ireland +1

  15. The Latent Color Subspace: Emergent Order in High-Dimensional Chaos

    Mar 12, 2026Mateusz Pach, Jessica Bader, Quentin Bouniot +2Controllable Image GenerationNeural Network Interpretability

    Technical University of Munich · Helmholtz Munich · Munich Center for Machine Learning +2

  16. A Unified Latent Space Disentanglement VAE Framework with Robust Disentanglement Effectiveness Evaluation

    Mar 11, 2026Xiaoan Lang, Md Mostafizer Rahman, Fang LiuVariational AutoencodersDisentangled Representation Learning

    Department of Applied and Computational Mathematics and Statistics University of Notre Dame, Notre Dame, IN, USA · Lucy Family Institute for Data & Society University of Notre Dame, Notre Dame, IN, USA

  17. On the Reliability of Cue Conflict and Beyond

    Mar 11, 2026Pum Jun Kim, Seung-Ah Lee, Seongho Park +2Neural Network Interpretability

    Ulsan National Institute of Science & Technology · College of Medicine at Hanyang University · NAVER AI Lab

  18. Task-Conditioned Routing Signatures in Sparse Mixture-of-Experts Transformers

    Mar 11, 2026Mynampati Sri Ranganadha AvinashMixture-of-Experts InterpretabilityMixture-of-Experts Language Models

    Independent Researcher | Asthra Labs

  19. FAME: Formal Abstract Minimal Explanation for Neural Networks

    Mar 11, 2026Ryma Boumazouza, Raya Elsaleh, Melanie Ducoffe +2Abstract InterpretationNeural Network Interpretability

    Airbus SAS, France · IRT Saint-Exupery, France · The Hebrew University of Jerusalem, Israel

  20. Why Does It Look There? Structured Explanations for Image Classification

    Mar 10, 2026Jiarui Li, Zixiang Yin, Samuel J Landry +2Prototype-Based ExplanationExplainable Image Classification

    Department of Computer Science, Tulane University, New Orleans LA 70118, USA · Department of Biochemistry and Molecular Biology, Tulane University School of Medicine, New Orleans LA 70112, USA

  21. Rethinking the Harmonic Loss via Non-Euclidean Distance Layers

    Mar 10, 2026Maxwell Miller-Golub, Collin Coil, Kamil Faber +4Neural Network InterpretabilityEnergy-Efficient ML

    American University, Washington, DC, USA · University of Vermont, Burlington, USA · AGH University, Krakow, Poland +2

  22. From Data Statistics to Feature Geometry: How Correlations Shape Superposition

    Mar 10, 2026Lucas Prieto, Edward Stevinson, Melih Barsbey +2Feature SuperpositionNeural Network Interpretability

  23. BrainSTR: Spatio-Temporal Contrastive Learning for Interpretable Dynamic Brain Network Modeling

    Mar 10, 2026Guiliang Guo, Guangqi Wen, Lingwen Liu +6Dynamic Functional ConnectivityBrain Network Analysis

    Computer Science and Engineering, Northeastern University, Shenyang, China · School of Computer Science and Artificial Intelligence, Shandong Normal University, Jinan, China · Graduate School of Information, Production and Systems, Waseda University, Kitakyushu, Fukuoka, Japan +3

  24. What is Missing? Explaining Neurons Activated by Absent Concepts

    Mar 10, 2026Robin Hesse, Simone Schaub-Meyer, Janina Hesse +2Feature VisualizationNeural Network Interpretability

    Max Planck Institute for Informatics, SIC · Department of Computer Science, Technical University of Darmstadt · Leibniz Institute for Resilience Research +2

  25. UNBOX: Unveiling Black-box visual models with Natural-language

    Mar 9, 2026Simone Carnemolla, Chiara Russo, Simone Palazzo +5Neural Network InterpretabilityExplainable Image Classification

    University of Catania. · Technical University of Munich. · Helmholtz Munich.

  26. TA-RNN-Medical-Hybrid: A Time-Aware and Interpretable Framework for Mortality Risk Prediction

    Mar 9, 2026Zahra Jafari, Azadeh Zamanifar, Amirfarhad FarhadiMortality PredictionLongitudinal EHR Modeling

    Department of Computer Engineering, SRC.,Islamic Azad university, Tehran, Iran · School of Computer Engineering, Iran university of Science and technology, Tehran, Iran

  27. Interpretable-by-Design Transformers via Architectural Stream Independence

    Mar 8, 2026Clayton Kerce, Alexis FoxTransformer InterpretabilityNeural Network Interpretability

    Georgia Tech Research Institute

  28. What on Earth is AlphaEarth? Hierarchical structure and functional interpretability for global land cover

    Mar 8, 2026Ivan Felipe Benavides-Martinez, Justin Guthrie, Jhon Edwin Arias +6Geospatial Foundation ModelsGeospatial Representation Learning

    Artificial Intelligence for Climate and Sustainability The Institute for Experiential Artificial Intelligence Northeastern University, Portland, ME, USA · Gulf of Maine Research Institute Portland, ME, USA · Sustainability and Data Sciences Laboratory Northeastern University, Boston, MA, USA +2

  29. A Cortically Inspired Architecture for Modular Perceptual AI

    Mar 7, 2026Prerna LuthraModular Neural NetworksNeural Network Interpretability

    Independent Researcher

  30. Causal Interpretation of Neural Network Computations with Contribution Decomposition

    Mar 6, 2026Joshua Brendan Melander, Zaki Alaoui, Shenghua Liu +2Neural Network InterpretabilityMechanistic Interpretability

    Department of Neurobiology, Stanford University · Department of Physics, Stanford University · Department of Applied Physics, Stanford University