Neural Network Interpretability

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 450% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 203

All topics

ICON Decomposition: Auditing deep neural networks for shortcuts by decomposing layer-wise representations using concepts
Aug 26, 2026Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer +7Concept-Based Explanations Neural Network Interpretability
Hertie Institute for AI in Brain Health, University of Tübingen, Tübingen, Germany. · Department of Psychiatry and Neurosciences, Charité - Universitätsmedizin Berlin, Berlin, Germany. · Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany. +5
Perturbation-based Regional Interpretability through Subtraction Mapping (PRISM): naming-error dissociations in language models and post-stroke aphasia
Aug 13, 2026Xiang Guan, Roger D. Newman-Norlund, Yong Yang +8Transformer Interpretability LLM Interpretability
Department of Computer Science and Engineering, University of South Carolina · Department of Communication Sciences and Disorders, University of South Carolina · ALLT.AI, LLC, Columbia, SC +3
The Neural Division of Labor: Biologically-Inspired Modular Architectures for Robust Neuromorphic Computing
Aug 8, 2026Maksim Bazhenov, Serafim Grubas, Vakhtang PutkaradzeEfficient Neural Network Inference Neural Network Interpretability
Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA · Department of Physics, University of Alberta, Edmonton, AB T6G 2G1, Canada · Department of Mathematical and Statistical Sciences, University of Alberta, Edmonton, AB T6G 2G1, Canada +1
Tools to Explain Neural Networks for Power System Dynamics
Aug 8, 2026Petros Ellinas, Johanna Vorwerk, Spyros ChatzivasileiadisNeural Surrogate Modeling Neural Network Interpretability
Department of Wind and Energy Systems, Technical University of Denmark, Lyngby, Denmark
The Spectral Neuron
Aug 8, 2026Alex ShtoffNeural Network Robustness Neural Network Interpretability
Technology Innovation Institute
Mathematical Principles and Experimental Discoveries of the Emergence of Symbolic Patterns in Artificial Neural Networks
Aug 7, 2026Quanshi Zhang, Qihan Ren, Siyu LouNeural Network Generalization Neural Network Interpretability
School of Computer Science, Shanghai Jiao Tong University, Shanghai, China
The Neural Echo: A Signal Processing Perspective for Understanding Neural Networks
Aug 5, 2026Chongbiao Wang, Daniel Gaa, Joachim Weickert +1Neural Network Interpretability Convolutional Neural Networks
Mathematical Image Analysis Group Faculty of Mathematics and Computer Science, Saarland University Campus E1.7, 66041 Saarbrücken, Germany · Zuse School ELIZA Hochschulstr. 10, 64289 Darmstadt, Germany
Computing Actual Causes for Neural Network Predictions under Structured Causal Inputs
Aug 4, 2026Jannick Strobel, Muqsit Azeem, Stefan LeueBranch and Bound Actual Causality
University of Konstanz
Beyond Routing Weights: Faithful Response-Level Interpretation of Mixture-of-Experts Reward Models via Contribution Contrast
Jul 31, 2026Yifan Wang, Jinyi Mu, Mayank Jobanputra +4Reward Modeling Neural Network Interpretability
Saarland University · University of California San Diego · Bielefeld University +2
Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification
Jul 31, 2026Stanislaw Janik, Michal ByraImplicit Neural Representations Neural Network Interpretability
Institute of Fundamental Technological Research, Polish Academy of Sciences, Warsaw, Poland · Samsung AI Center, Warsaw, Poland
ECG-InterpBench: Benchmarking the Interpretability of ECG Foundation Models with Matched-Scale Sparse Autoencoders
Jul 29, 2026Yixuan Duan, Wei QiuElectrocardiogram Foundation Models Neural Network Interpretability
Department of Electrical and Computer Engineering Rice University Houston, Texas, USA
From Conceptual Hydrologic Models to Conceptually Interpretable Neural Networks: A Snow-Water Mass-Conserving-Perceptron Framework for Discovering Catchment-Scale Precipitation-Storage-Runoff Representations
Jul 29, 2026Yuan-Heng Wang, Hoshin V. GuptaNeural Network Interpretability
Department of Hydrology and Atmospheric Science, University of Arizona, Tucson, AZ
Are the High-weight Neurons the Important Ones in Image Classification Neural Networks?
Jul 28, 2026Qitao Chen, Dongfu Yin, F. Richard YuNeural Network Interpretability Weight-Space Analysis
WeBank Institute of FinTech, ShenZhen University, YueHai, ShenZhen, 518060, GuangDong, China. · Artificial Intelligence and Digital Economy Guangdong Provincial Laboratory, YuTang, ShenZhen, 518132, GuangDong, China.
LAWFUL: Law-Aligned Witness for Faithful Use of Latents
Jul 26, 2026Kevin Chen, Kenneth W. Parker, Anish AroraTransformer Interpretability Mechanistic Interpretability
Computer Science & Engineering, The Ohio State University, Columbus, Ohio, USA · Independent Researcher, Waterford, Virginia, USA
Variable Importance Identification Through Lazy Training for Binary Classification
Jul 25, 2026Anand Singh, Luke Pennella, Eshan Kabir +1Feature Attribution Neural Network Interpretability
Department of Mathematics, Georgia Institute of Technology · Department of Mathematics, University of Connecticut · Department of Mathematics, Columbia University +1
Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations
Jul 24, 2026Kazi Kamruzzaman Rabbi, Md. Zami Al Zunaed Farabe, M. Sohel RahmanFeature Attribution Neural Network Interpretability
BRAC University, Dhaka, Bangladesh · Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh
Do emulated quantum circuits change what CNNs look at? Performance and explainability comparison in medical image classification
Jul 23, 2026Guillermo Rubiños Rodríguez, Martín Ottavianelli, Mateo Alonso +4Quantum Convolutional Neural Networks Quantum-Inspired ML
Instituto Tecnol´ogico de Galicia, Cant´on Grande 9, Planta 3, 15003 A Coru˜na, Spain · Centro Singular de Investigaci´on en Tecnolox´ıas Intelixentes (CiTIUS), Universidade de Santiago de Compostela, Santiago de Compostela, 15782, Spain · Department of Neuroradiology, LMU University Hospital, Ludwig Maximilian University of Munich, Marchioninistraße 15, 81377 Munich, Germany +1
Neural Feature Governance: Extending Atom Prevalence
Jul 23, 2026Idris Karel Seunda Ekwe, Patrick Tenga Shako, Ernest Parfait FokouéBayesian Neural Networks Uncertainty Quantification
African Institute for Mathematical Sciences (AIMS), Cameroon · Rochester Institute of Technology (RIT), US
Scaling Interpretable Transformers with Parity Bottleneck Layers
Jul 22, 2026Andrew Mack, Kraig Yuheng Tou, Mark Henry +2Feature Superposition Transformer Interpretability
Principles of Intelligence · Independent
The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks
Jul 22, 2026Antonio Di CeccoFeature Interaction Modeling Neural Network Interpretability
Universit`a "G. d'Annunzio" Chieti–Pescara, and School of AI.
Do Sheaf Neural Networks Use Holonomy? A Measure--Intervene--Control Study
Jul 21, 2026Ankit Grover, Rémi BourgerieNeural Network Interpretability Sheaf Neural Networks
KTH Royal Institute of Technology, Stockholm, Sweden
Tensor Network Machine Learning for Wildfire Susceptibility Mapping: from Grokking Dynamics to Quantum Mixedness of Class Representations
Jul 21, 2026Domenico Pomarico, Alessandra Costantino, Gabriel Ramirez Sanchez +12Tensor Networks Neural Network Interpretability
1Dipartimento Interuniversitario di Fisica, Universit`a degli Studi di Bari Aldo Moro, Bari, I-70126, Italy · 2Istituto Nazionale di Fisica Nucleare, Sezione di Bari, Bari, I-70126, Italy · 3Dipartimento di Biomedicina Traslazionale e Neuroscienze (DiBraiN), Universit`a degli Studi di Bari Aldo Moro, Bari, I-70124, Italy +2
Retrieval is Enough: Training-Free Interpretability with a Tool-Using Agent
Jul 17, 2026Sriram Balasubramanian, Soheil FeiziMechanistic Interpretability Neural Network Interpretability
Department of Computer Science University of Maryland
Neural spectroscopy of AlphaFold2 reveals encoded protein conformational landscapes
Jul 17, 2026Kaustav MehtaProtein Structure Prediction Neural Network Interpretability
Independent researcher, India
Kolmogorov--Arnold Networks for Small Language Models
Jul 17, 2026Felippe Alves, Renato VicenteSmall Language Models Neural Network Interpretability
TELUS Digital Research Hub – CIAAM Institute of Mathematics, Statistics and Computer Science University of São Paulo
TIDE: Trustworthy and Interpretable Battery Degradation Estimation with Contextual Learning and Symbolic Distillation
Jul 16, 2026Wen Yang Tan, Jiawei Li, Fang Liu +4RUL Estimation Neural Network Interpretability
Singapore Institute of Technology (SIT), Singapore 828608 · Singapore University of Social Sciences (SUSS), Singapore 599494
From Preimage Search To Source-Grounded Feature Inversion
Jul 14, 2026Kaixiang ShuNeural Network Interpretability Image Inverse Problems
Independent Researcher
Understanding Structured Health Data through Interaction-Aware Mixture-of-Experts
Jul 14, 2026Ji Hwan Park, Ying Ding, Tianjin GuoMulti-View Learning Clinical Outcome Prediction
Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, Texas, USA · School of Information, The University of Texas at Austin, Austin, Texas, USA · Kelley School of Business, Indiana University, Bloomington, Indiana, USA
Sparse Inter-Layer Dependencies of Transformer FFN Neurons
Jul 13, 2026Johannes Knittel, Hanspeter PfisterTransformer Interpretability Transformer FFNs
Harvard University
From Neural Network Decisions to Training Cases: An Exact Account via Case-Based Decision Theory
Jul 13, 2026Manli Yan, Yuebin Lin, Yaowen Yu +1Algorithmic Auditing Neural Network Interpretability
Huazhong University of Science and Technology Wuhan, China

Aug 26, 2026, cs.LGJ/K move · Enter open · S save

ICON Decomposition: Auditing deep neural networks for shortcuts by decomposing layer-wise representations using concepts

Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer, Marc-Andre Schulz+6

Hertie Institute for AI in Brain Health, University of Tübingen, Tübingen, Germany. · Department of Psychiatry and Neurosciences, Charité - Universitätsmedizin Berlin, Berlin, Germany. · Department of Psychology, Humboldt-Universität zu Berlin, Berlin, Germany.+5

Concept-Based Explanations Neural Network Interpretability Explainable Medical Image Analysis Shortcut Learning

Open paper AskPDF arXiv

Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Before deployment, models should be audited for reliance on a set of concepts, such as acquisition artifacts or demographics. Current methods, such as linear probes and concept activation vectors, measure reliance by asking whether each concept, in isolation, is decodable from a layer. Their scores therefore reflect not only reliance but also correlations in the audit dataset. We introduce Independent Canonical cONcept (ICON) decomposition, which quantifies the share of a layer's variance each concept explains, conditional on all other concepts and the outcome. ICON scores are variance shares, comparable across layers and between continuous and categorical concepts. ICON also reports the share the set leaves unexplained. On simulated data, ICON recovers the true importance more accurately than seven baselines. On skin-cancer and neuroimaging models, ICON distinguishes learned shortcuts from correlated concepts, confirmed by retraining and out-of-distribution tests.

Momentum

Related topics

Latest papers 203