Neural Network Interpretability

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 450% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 203

All topics

Interpreting Reinforcement Learning Agents with Susceptibilities
May 8, 2026Chris Elliott, Einar Urdshals, David Quarel +1Reinforcement Learning Neural Network Interpretability
Timaeus
Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning
May 8, 2026Chris Elliott, Daniel MurfetStatistical Physics of Learning Bayesian Inference
Timaeus
Is Class Signal Clustered or Routed in Task-Induced Implicit Neural Representation Weight Spaces?
May 8, 2026Xinyi Guo, Mingyi He, Haobin Ding +7Neural Representation Geometry Implicit Neural Representations
South China Normal University · Beijing University of Chemical Technology · Tsinghua University +1
Concept-Based Abductive and Contrastive Explanations for Behaviors of Vision Models
May 7, 2026Ronaldo Canizales, Divya Gopinath, Corina Păsăreanu +1Abductive Reasoning Concept-Based Explanations
Colorado State University, Fort Collins CO, USA · KBR Inc., NASA Ames, Moffett Field CA, USA · Carnegie Mellon University, Pittsburgh PA, USA
Hyperbolic Concept Bottleneck Models
May 7, 2026Daniel Uyterlinde, Swasti Shreya Mishra, Pascal MettesConcept Bottleneck Models Hierarchical Representation Learning
Informatics Institute, University of Amsterdam, The Netherlands
The Metagame of Interpretability and Meta-Attributions
May 7, 2026Hubert Baniecki, Przemyslaw Biecek, Fabian FumagalliTransformer Interpretability Feature Interaction Modeling
University of Warsaw · Centre for Credible AI, Warsaw University of Technology · Bielefeld University +1
AffineLens: Capturing the Continuous Piecewise Affine Functions of Neural Networks
May 7, 2026Yi Wei, Xuan Qi, Furao Shen +3Neural Network Interpretability Neural Network Approximation Theory
State Key Laboratory of Novel Software Technology School of Intelligence Science and Technology Nanjing University, Jiangsu, China · AI for Good Istituto Italiano di Tecnologia, Genoa, Italy · DITEN University of Genoa, Genoa, Italy +2
Temporal Functional Circuits: From Spline Plots to Faithful Explanations in KAN Forecasting
May 7, 2026Naveen MysoreExplainable Time Series Forecasting Explainable Artificial Intelligence
University of California, Santa Barbara
Spectral Lens: Activation and Gradient Spectra as Diagnostics of LLM Optimization
May 7, 2026Andy Zeyi Liu, Elliot Paquette, John SousRepresentation Learning Decoder-Only Language Models
Department of Applied Physics, Yale University, New Haven, CT 06511, United States · Department of Mathematics and Statistics, McGill University · Department of Applied Physics, Yale University, New Haven, Connecticut 06511, USA +1
Interpreting V1 Population Activity via Image-Neural Latent Representation Alignment
May 5, 2026Xin Wang, Zhuangzhi Gao, Hongyi Qin +3Contrastive Learning Neural Processes
School of Psychological and Cognitive Sciences, Peking University, Beijing, China · Department of Primary Care and Mental Health, University of Liverpool, Liverpool, United Kingdom · Liverpool Centre for Cardiovascular Science, University of Liverpool, Liverpool, United Kingdom +2
KANs need curvature: penalties for compositional smoothness
May 4, 2026James BagrowNeural Network Activation Functions Neural Network Interpretability
Mathematics & Statistics, University of Vermont, Burlington, VT, United States · Vermont Complex Systems Center, University of Vermont, Burlington, VT, United States
A framework for analyzing concept representations in neural models
May 2, 2026Burin Naowarat, Hao Tang, Sharon GoldwaterRepresentation Disentanglement Neural Network Interpretability
The Centre for Speech Technology Research School of Informatics, University of Edinburgh, United Kingdom
HyCOP: Hybrid Composition Operators for Interpretable Learning of PDEs
May 1, 2026Jinpai Zhao, Nishant Panda, Yen Ting Lin +3Neural Surrogate Modeling Neural Network Interpretability
Oden Institute University of Texas at Austin Austin, TX, USA · Information Sciences, CAI-3 Los Alamos National Laboratory Los Alamos, NM 87545, USA · Norwegian University of Life Sciences Ås, Norway / Simula Research Lab. Oslo, Norway
From Local to Global to Mechanistic: An iERF-Centered Unified Framework for Interpreting Vision Models
May 1, 2026Yearim Kim, Sangyu Han, Nojun KwakSaliency Map Evaluation Mechanistic Interpretability
Seoul National University, Seoul 08826, South Korea
Do Sparse Autoencoders Capture Concept Manifolds?
Apr 30, 2026Usha Bhalla, Thomas Fel, Can Rager +9Neural Representation Geometry Sparse Autoencoders
aHarvard University · bNortheastern University · cTechnion IIT +1
Adjoint Inversion Reveals Holographic Superposition and Destructive Interference in CNN Classifiers
Apr 30, 2026Kaixiang ShuFeature Superposition Neural Network Interpretability
Independent Researcher · Chongqing, China
Towards interpretable AI with quantum annealing feature selection
Apr 28, 2026Francesco Aldo Venturelli, Emanuele Costa, Sikha O K +3Quantum Annealing Neural Network Interpretability
1BCN Medtech, Universitat Pompeu Fabra, Barcelona, Spain · 2Barcelona Supercomputing Center (BSC) · 3Departament de F´ısica Qu`antica i Astrof´ısica, Facultat de F´ısica, Universitat de Barcelona, E-08028 Barcelona, Spain +2
SaliencyDecor: Enhancing Neural Network Interpretability through Feature Decorrelation
Apr 28, 2026Ali Karkehabadi, Jamshid Hassanpour, Houman Homayoun +1Gradient-Based Attribution Neural Network Interpretability
University of California, Davis, USA · Georgia Institute of Technology, USA
On the explainability of max-plus neural networks
Apr 27, 2026Ikhlas Enaieh, Olivier Fercoq, García ÁngelFeature Attribution Neural Network Interpretability
S2A, LTCI · LTCI, T´el´ecom Paris · Institut Polytechnique de Paris +5
Explainable AI in Speaker Recognition -- Making Latent Representations Understandable
Apr 25, 2026Yanze Xu, Wenwu Wang, Mark D. PlumbleyClustering Hierarchical Clustering
Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, GU2 7XH, UK · Department of Informatics, King’s College London, London, WC2R 2LS, UK
Explanation of Dynamic Physical Field Predictions using WassersteinGrad: Application to Autoregressive Weather Forecasting
Apr 24, 2026Younes Essafouri, Laure Raynaud, Luciano Drozda +1Gradient-Based Attribution Neural Network Interpretability
Univ. Toulouse, INSA Toulouse, CNRS UMR 5219, IMT, Toulouse, France · Météo-France, CNRS, Univ. Toulouse, CNRM, Toulouse, France · Cerfacs, CNRS/Cerfacs/IRD, CECI, Toulouse, France
Contrastive Semantic Projection: Faithful Neuron Labeling with Contrastive Examples
Apr 24, 2026Oussama Bouanani, Jim Berend, Wojciech Samek +2Neural Network Interpretability
Fraunhofer Heinrich Hertz Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany +1
H-Sets: Hessian-Guided Discovery of Set-Level Feature Interactions in Image Classifiers
Apr 23, 2026Ayushi Mehrotra, Dipkamal Bhusal, Michael Clifford +1Gradient-Based Attribution Feature Attribution
Rochester Institute of Technology · Toyota InfoTech Labs
Foundation models for discovering robust biomarkers of neurological disorders from dynamic functional connectivity
Apr 23, 2026Deepank Girish, Yi Hao Chan, Sukrit Gupta +2Medical Imaging Foundation Models Functional Connectivity
College of Computing and Data Science, Nanyang Technological University, Singapore
An explicit operator explains end-to-end computation in the modern neural networks used for sequence and language modeling
Apr 22, 2026Anif N. Shikder, Ramit Dey, Sayantan Auddy +6Dynamical Systems Neural Network Interpretability
Department of Mathematics, Western University, London ON, Canada · Fields Lab for Network Computation, Fields Institute, Toronto ON, Canada · King’s University College at Western University, London ON, Canada +1
Rethinking Intrinsic Dimension Estimation in Neural Representations
Apr 22, 2026Rickmer Schulte, David RügamerIntrinsic Dimensionality Neural Representation Geometry
LMU Munich, MCML
Bridging Traditional Explainability Methods and Multimodal Multilingual Models: An XAI-Based Analysis
Apr 21, 2026Paweł Pozorski, Jakub Muszyński, Maria GanzhaMultimodal Large Language Models Neural Network Interpretability
student record book number 327304 · student record book number 327246 · thesis supervisor
Decision-Aware Attention Propagation for Vision Transformer Explainability
Apr 20, 2026Sehyeong Jo, Gangjae Jang, Haesol ParkTransformer Interpretability Vision Transformer
Aarhus University · University of Colorado Boulder · KIST
The Topological Dual of a Dataset: A Logic-to-Topology Encoding for AlphaGeometry-Style Data
Apr 20, 2026Anthony BordgNeural Network Interpretability Neuro-Symbolic Reasoning
Huawei Lagrange Center, Paris, France
A Sugeno Integral View of Binarized Neural Network Inference
Apr 20, 2026Ismaïl Baaj, Henri PradeBinary Neural Networks Neural Network Interpretability
LEMMA, Paris-Panthéon-Assas University · IRIT - Toulouse University

May 8, 2026, cs.LGJ/K move · Enter open · S save

Interpreting Reinforcement Learning Agents with Susceptibilities

Chris Elliott, Einar Urdshals, David Quarel, Daniel Murfet

Timaeus

Reinforcement Learning Neural Network Interpretability Interpretability in RL

Open paper AskPDF arXiv

Susceptibilities are a technique for neural network interpretability that studies the response of posterior expectation values of observables to perturbations of the loss. We generalize this construction to the setting of the regret in deep reinforcement learning and investigate the utility of susceptibilities in a simple gridworld model that nevertheless exhibits non-trivial stagewise development. We argue that susceptibilities reveal internal features of the development of the model in parameter space that one cannot detect purely by studying the development of the learned policy. We validate these results with activation-steering, and discuss the framework's extension to RLHF post-training.

Momentum

Related topics

Latest papers 203