Neural Network Interpretability

Overview

Neural network interpretability asks what a model has learned, which internal computations support its predictions, and whether those computations are understandable and reliable. Post-hoc methods include saliency and attribution, concept-based explanations, counterfactuals, and formal logical explanations; mechanistic work uses probes, sparse autoencoders, circuit analysis, and interventions on neurons or features. Another approach builds interpretability into models through sparse or additive architectures, prototypes, symbolic rules, and concept bottlenecks, with evaluations spanning image, language, time-series, and scientific tasks. A central challenge is showing that an explanation is both human-meaningful and faithful to the model: readable features or sparse circuits alone do not establish causal importance, and methods can vary in robustness and reliability across settings.

Latest papers

CardsList

367 papers25 papers in the last four weeks, down 43% on the four weeks before. 0.2% of all new papers.

  1. Who Moved My ZZ Boson? Revealing Resonance Features Inside a Neural Network

    Nov 19, 2026Kevin Alberto Urquía Calderón, Midori Kato, Inar Timiryasov +1Neural Network InterpretabilityGenerative Modeling

  2. Separating Interpretability and Reconstruction in Sparse Autoencoders

    Nov 18, 2026Anh Tien Nguyen, Dimitris Samaras, Mahdi S. HosseiniSparse AutoencodersNeural Network Interpretability

  3. Auditing Learned Dynamics in Weather Models

    Nov 18, 2026Skylar Gu, Christina LastCausal InterpretabilityCausal Intervention

  4. Learning Trajectories Leave Decodable Traces in Neural Network Weight Space and Internal Representations

    Nov 18, 2026Patrick Krauss, Achim Schilling, Andreas Maier +2Neural Network Training DynamicsWeight-Space Learning

  5. The geometry of neural network decision boundaries

    Nov 18, 2026Dylan M Paiton, David L Schultheiss, Matthias Kümmerer +2Decision Boundary AnalysisNeural Network Interpretability

  6. From Constraints to Causality: Petri net Encodings for Explaining Binary Neural Network Inference

    Nov 17, 2026Alex Chan, Mohamed Tarraf, Alex Yakovlev +1Binary Neural NetworksNeural Network Interpretability

  7. ONAM: Fitting Orthogonal Neural Additive Models for Post-Hoc Interpretability of Black-Box Predictions

    Nov 16, 2026David Köhler, Matthias SchmidNeural Network InterpretabilityFeature Attribution

  8. Deep Sequence Model for Genome Wide Discovery of Coding and Regulatory Element Signatures

    Nov 15, 2026Rayhaneh Shabani Nia, Ali KarkehabadiGenomicsSequence Modeling

  9. Evaluating and Improving Natural Language Autoencoders

    Nov 14, 2026Marco Bazzani, Dan MossingAutoencodersNeural Network Interpretability

  10. Size-Matched Nulls Reveal What Block Ablation Measures in MambaVision

    Nov 14, 2026Tahmina Kousar, Tehseen ZiaNeural Network InterpretabilityHybrid Mamba-Transformer Architectures

  11. Learning nonlinear interactions and cross-feature couplings with product-unit residual networks

    Nov 13, 2026Ziyuan Li, Uwe Jaekel, Babette DellenFeature Interaction ModelingNeural Network Interpretability

  12. Data-Free Metrics Don't Work: The Missing Invariance Under Functionally Preserving Reparameterisations

    Nov 11, 2026Gabryel Mason-Williams, Israel Mason-Williams, Fredrik DahlqvistRandom Matrix TheoryNeural Network Interpretability

  13. Mechanistic Interpretability Needs a Logic of Inference

    Nov 10, 2026Zhimin Hu, Chen Jiang, Hanbo Xie +2Mechanistic InterpretabilityNeural Network Interpretability

  14. Causal Concept Explanations for Deep Neural Models

    Nov 10, 2026Josh Rountree, Pulkit Verma, Oswin So +2Concept-Based ExplanationsNeural Network Interpretability

  15. Representational Atoms in Neural Networks

    Nov 10, 2026Chenhui Hu, Pengfei Cao, Kang Liu +1Neural Network InterpretabilitySemantic Representation Learning

  16. Do Feature Interactions Break Explainability? Provable Explanations for Any-Order Neural Additive Models

    Nov 8, 2026Idan Refaeli, Yizhak Yisrael Elboher, Guy Katz +1Neural Network InterpretabilityFeature Interaction Modeling

  17. Projected Neural Additive Models as Universal Approximators

    Nov 6, 2026Nhon N. Phan, Qiang Du, John D. Clayton +1Function ApproximationNeural Network Approximation Theory

  18. MEDALLION: Manifold Embedding Distillation and Attributions for Low-dimensional Latent InterpretatiON

    Nov 5, 2026Vicky Jung, Claire M. He, Irene Chang +1Neural Network InterpretabilityDimensionality Reduction

  19. Function-Preserving Audits of Visual Weight Readers

    Nov 4, 2026Sungguk Cha, Mintae Kim, Youngsub Han +2Neural Network Interpretability

  20. From Activation to Intervention: Interpreting Sparse Visual Features through Reconstruction

    Nov 3, 2026Jianyang Gu, Jacob Beattie, Mridul Khurana +7Neural Network InterpretabilityMechanistic Interpretability

  21. Opening the Black Box of Neural Auction Mechanisms

    Nov 1, 2026Li Kong, Zilong Liu, Haoyang Zhou +2Mechanism DesignMechanistic Interpretability

  22. Interpretable structural temperature scaling for calibrating graph neural networks and point-cloud segmentation

    Oct 30, 2026Chenyi HuangModel CalibrationGraph Neural Networks

  23. Interpretable neural equation learning for high-dimensional materials data

    Oct 27, 2026Philip Jacobson, Prasad Iyer, Remi Dingreville +1Symbolic RegressionNeural Network Interpretability