Neuron-Level Interpretability

Follow this topic Ask about this topic

Latest papers

20 papers3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Comparing Sparsified MLPs and Transcoders: Many Transcoder Features Are Neurons in Disguise
Oct 17, 2026Stepan Shabalin, Mateusz PiotrowskiTransformer Interpretability Neuron-Level Interpretability
Anthropic · Georgia Institute of Technology
Sparse Transformer: a Controlled Testbed for Mechanistic Interpretability
Oct 14, 2026Anton Korznikov, Alexey Dontsov, Elena TutubalinaMechanistic Interpretability Neuron-Level Interpretability
Hallucination Neurons and Where to Find Them: An Investigation into the existence of Hallucination Neurons
Sep 24, 2026Huseyin Cavus, Sebin Sabu, Joshua Spear +2LLM Interpretability Hallucination in Language Models
Trakya University, Edirne, Türkiye · DRIVE, Great Ormond Street Hospital for Children NHS Foundation Trust, London, UK
NeuroActiSep: Detecting Factual Hallucinations from Feed-Forward Neurons in a Single Pass
Sep 13, 2026Ali Derogar Odolou, Reza Nazari, Mostafa SalehiFactual QA LLM Hallucination Detection
School of Intelligent Systems Engineering College of Interdisciplinary Science and Technology University of Tehran, Tehran, Iran
Through the Looking Glass: Directly Reading and Writing Transformers
Sep 9, 2026Mark OskinKnowledge Editing Transformer Interpretability
School of Computer Science and Engineering University of Washington
RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons
Aug 25, 2026Runyu Wang, Bo Liu, Xiaxin Zhang +6Transformer Interpretability LLM Interpretability
School of Transportation and Civil Engineering, Nantong University · Chongqing University of Post and Telecommunications · China Southern Power Grid Company Limited +1
Do CNNs Internally Represent Real and Fake Images Differently? A Hidden-Layer Analysis
Aug 12, 2026Moumita Sen Sarma, Pascal Hitzler, Eugene Y. VassermanNeuron-Level Interpretability Convolutional Neural Networks
Department of Computer Science, Kansas State University, Manhattan, Kansas, USA
Multilingual Emotion Neurons in Large Audio-Language Models
Aug 9, 2026Xiutian Zhao, Philipp Koehn, Björn Schuller +1Cross-Lingual Representation Alignment Neuron-Level Interpretability
Center for Language and Speech Processing (CLSP), Johns Hopkins University, USA · Group on Language, Audio & Music (GLAM), Imperial College London, UK
Sparse Inter-Layer Dependencies of Transformer FFN Neurons
Jul 13, 2026Johannes Knittel, Hanspeter PfisterTransformer Interpretability Transformer FFNs
Harvard University
Normalized Relevance Measure as a Unifying Framework to Explain Neural Network Latent Structures
May 30, 2026Ping Xiong, Thomas Schnake, Grégoire Montavon +2Feature Attribution Neural Network Interpretability
Berlin Institute for the Foundations of Learning and Data – BIFOLD, 10623 Berlin, Germany · Machine Learning Group, Technical University of Berlin, Berlin, Germany · Department of Chemistry, Chemical Physics Theory Group, University of Toronto, Toronto, Canada +6
Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performance
May 29, 2026Doğukan Bağcı, Bernt Schiele, Simone Schaub-Meyer +2Disentangled Representation Learning Representation Disentanglement
Max Planck Institute for Informatics, SIC · Department of Computer Science, TU Darmstadt
Tiny Brains, Giant Impact: Uncovering the Keystone Neurons of LLM with Just a Few Prompts
May 24, 2026Xiangtian Ji, Yuxin Chen, Zhengzhou Cai +3Fine-Tuning LLM Interpretability
National University of Singapore · Beijing University of Posts and Telecommunications · University of Science and Technology of China
LINE: LLM-based Iterative Neuron Explanations for Vision Models
Apr 9, 2026Vladimir Zaigrajew, Michał Piechota, Gaspar Sekula +2Neuron-Level Interpretability Neural Network Interpretability
Centre for Credible AI Warsaw University of Technology · Warsaw University of Technology Warsaw, Poland · Centre for Credible AI, Warsaw University of Technology University of Warsaw, Poland
ConceptTracer: Interactive Analysis of Concept Saliency and Selectivity in Neural Representations
Apr 8, 2026Ricardo Knauer, Andre Beinrucker, Erik RodnerNeural Network Interpretability Neuron-Level Interpretability
KI-Werkstatt, University of Applied Sciences Berlin, Berlin, Germany
Select, Hypothesize and Verify: Towards Verified Neuron Concept Interpretation
Mar 26, 2026ZeBin Ji, Yang Hu, Xiuli Bi +2Neuron-Level Interpretability Neural Network Interpretability
Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, China · Jinan Inspur Data Technology Co., Ltd., Jinan, China
What is Missing? Explaining Neurons Activated by Absent Concepts
Mar 10, 2026Robin Hesse, Simone Schaub-Meyer, Janina Hesse +2Feature Visualization Neural Network Interpretability
Max Planck Institute for Informatics, SIC · Department of Computer Science, Technical University of Darmstadt · Leibniz Institute for Resilience Research +2
SYNAPSE: Framework for Neuron Analysis and Perturbation in Sequence Encoding
Mar 9, 2026Jesús Sánchez Ochoa, Enrique Tomás Martínez Beltrán, Alberto Huertas CeldránNeuron-Level Interpretability Transformer Interpretability
Department of Information and Communications Engineering, University of Murcia, Murcia, 30100, Spain
Causal Interpretation of Neural Network Computations with Contribution Decomposition
Mar 6, 2026Joshua Brendan Melander, Zaki Alaoui, Shenghua Liu +2Neural Network Interpretability Mechanistic Interpretability
Department of Neurobiology, Stanford University · Department of Physics, Stanford University · Department of Applied Physics, Stanford University
GLUScope: A Tool for Analyzing GLU Neurons in Transformer Language Models
Feb 27, 2026Sebastian Gerstner, Hinrich SchützeNeuron-Level Interpretability Transformer Interpretability
LMU Munich and Munich Center for Machine Learning (MCML), Munich, Germany
A Case Study on Concept Induction for Neuron-Level Interpretability in CNN
Feb 27, 2026Moumita Sen Sarma, Samatha Ereshi Akkamahadevi, Pascal HitzlerNeuron-Level Interpretability Neural Network Interpretability
Department of Computer Science, Kansas State University, Manhattan, Kansas, USA

Oct 17, 2026J/K move · Enter open · S save

Comparing Sparsified MLPs and Transcoders: Many Transcoder Features Are Neurons in Disguise

Stepan Shabalin, Mateusz Piotrowski

Anthropic · Georgia Institute of Technology

Transformer Interpretability Neuron-Level Interpretability Transformer FFNs

Open paper AskPDF arXiv

MLP neurons in transformer language models are widely understood to be polysemantic, motivating transcoders - sparse, learned approximations of MLP layers - as interpretable replacement models for mechanistic analysis. However, recent work shows that MLP neurons can be used to recover sparse, interpretable circuits (Arora et al. 2026, Ye et al. 2025), raising the question of whether transcoders are necessary for circuit analysis. We define neuron features as transcoder features whose weights are strongly aligned with individual MLP neurons, and show that they account for a substantial fraction of active features in late layers across 13 open-source transcoders. Only neuron features have a stable correspondence with something in the underlying MLP: other features vary more between training runs, do not match the MLP's Jacobian, and change less significantly from their random initialization. Our findings reframe MLP neurons as a stronger basis for interpretability than commonly assumed, and call for better methods of attributing the non-neuron structure that is used by the model.