LLM Interpretability

LLM: Large Language Model

Large language model interpretability asks what information models represent, how it moves through layers and components, and which computations cause particular answers or behaviors. Researchers use probes and representation geometry, sparse autoencoders, attribution and circuit tracing, activation patching or ablation, and steering to study factual recall, mathematical reasoning, hallucination, refusal, and in-context learning. These methods can reveal distributed circuits and layer-wise signals, and sometimes support targeted interventions without retraining. A central challenge is that information being decodable—or a direction being steerable—does not prove the model uses it; causal faithfulness, robustness across prompts and model families, and reliable evaluation remain open tensions.

Latest papers

781 papers82 papers in the last four weeks, up 12% on the four weeks before. 0.5% of all new papers.

Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMs
Mar 15, 2026Michael KeemanEmotion Representation in Language Models LLM Interpretability
Keido Labs, Liverpool, UK
Understanding the Emergence of Seemingly Useless Features in Next-Token Predictors
Mar 14, 2026Mark Rofin, Jalal Naghiyev, Michael HahnTransformer Interpretability LLM Interpretability
School of Computer and Communication Sciences, EPFL · Technical University of Munich · Zuse School ELIZA +1
On the Tip of the Tongue: Why LLMs Hallucinate Answers They Can Decode
Mar 14, 2026Valeria Ruscio, Keiran ThompsonHallucination in Language Models LLM Interpretability
Intuition Machines Inc.
Causal Tracing of Audio-Text Fusion in Large Audio Language Models
Mar 14, 2026Wei-Chih Chen, Chien-yu Huang, Hung-yi LeeLLM Interpretability Audio Understanding
National Taiwan University, Taiwan · Carnegie Mellon University, United States
As Language Models Scale, Low-order Linear Depth Dynamics Emerge
Mar 13, 2026Buddhika Nettasinghe, Geethu JosephLLM Interpretability Language Model Scaling Laws
University of Iowa, Iowa City, USA · 2Delft University of Technology, Delft, Netherlands
Word Recovery in Large Language Models Enables Character-Level Tokenization Robustness
Mar 11, 2026Zhipeng Yang, Shu Yang, Lijie Hu +1Language Model Robustness Tokenization in Language Models
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Provable Responsible AI and Data Analytics (PRADA) Lab · King Abdullah University of Science and Technology
Quantifying the Necessity of Chain of Thought through Opaque Serial Depth
Mar 10, 2026Jonah Brown-Cohen, David Lindner, Rohin ShahImplicit Reasoning in Language Models CoT Reasoning
Google DeepMind
Emotion is Not Just a Label: Latent Emotional Factors in LLM Processing
Mar 10, 2026Benjamin Reichman, Adar Avsian, Samuel Webster +1Emotion Representation in Language Models LLM Interpretability
Georgia Institute of Technology
One Language, Two Scripts: Probing Script-Invariance in LLM Concept Representations
Mar 9, 2026Sripad KarneLLM Interpretability Sparse Autoencoders
Columbia University
Sensivity of LLMs' Explanations to the Training Randomness:Context, Class & Task Dependencies
Mar 9, 2026Romain Loncour, Jérémie Bogaert, François-Xavier StandaertLLM Interpretability Transformer Interpretability
UCLouvain, Louvain-la-Neuve, Belgium.
Navigating the Concept Space of Language Models
Mar 6, 2026Wilson E. Marcílio-Jr, Danilo M. ElerLLM Interpretability Representation Geometry in Language Models
Adaption Labs, Curitiba, PR, Brazil · São Paulo State University (UNESP), Presidente Prudente, SP, Brazil
SPOT: Span-level Pause-of-Thought for Efficient and Interpretable Latent Reasoning in Large Language Models
Mar 6, 2026Yunlong Chu, Minglai Shao, Yuhang Liu +4Efficient Language Model Reasoning CoT Compression
School of New Media and Communication, Tianjin University, Tianjin, China · Independent Contributor, CA, USA · School of Computer Science and Engineering, Beihang University, Beijing, China
Evaluating LLM Alignment With Human Trust Models
Mar 6, 2026Anushka Debnath, Stephen Cranefield, Bastin Tony Roy Savarimuthu +1Trust in AI LLM Interpretability
School of Computing, University of Otago, New Zealand · IRIT, CNRS, Toulouse University, France
HART: Data-Driven Hallucination Attribution and Evidence-Based Tracing for Large Language Models
Mar 6, 2026Shize Liang, Hongzhi WangHallucination in Language Models LLM Interpretability
Faculty of Computing, Harbin Institute of Technology, Harbin, China
World Properties without World Models: Distributional Associations and the Interpretation of Decoding Results from Language Models
Mar 4, 2026Elan BarenholtzLLM World Models Word Embeddings
Department of Psychology & Center for Complex Systems and Brain Sciences Florida Atlantic University
Stable and Steerable Sparse Autoencoders with Weight Regularization
Mar 4, 2026Piotr Jedryszek, Oliver M. CrookSparse Autoencoders LLM Interpretability
Department of Biology, University of Oxford, Oxford, UK · Kavli Institute for Nanoscience Discovery, University of Oxford, Oxford, UK · Department of Chemistry, University of Oxford, Oxford, UK
Understanding the Dynamics of Demonstration Conflict in In-Context Learning
Mar 3, 2026Difan Jiao, Di Wang, Lijie HuIn-Context Learning Implicit Reasoning in Language Models
University of Toronto · King Abdullah University of Science and Technology · Mohamed bin Zayed University of Artificial Intelligence
Tracing Pharmacological Knowledge In Large Language Models
Mar 3, 2026Basil Hasan Khwaja, Dylan Chen, Guntas Toor +1LLM Interpretability Representation Geometry in Language Models
School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA · Viterbi School of Engineering, University of Southern California, Los Angeles, CA, USA · Stephen J. R. Smith Faculty of Engineering and Applied Science, Queen’s University, Kingston, ON, Canada +1
Step-Level Sparse Autoencoder for Reasoning Process Interpretation
Mar 3, 2026Xuan Yang, Jiayu Liu, Yuhang Lai +3LLM Interpretability Sparse Autoencoders
Department of Data Science, City University of Hong Kong · Hong Kong Institute of AI for Science, City University of Hong Kong · Li Auto Inc +1
SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training
Mar 3, 2026Qi Zhang, Yifei Wang, Xiaohan Wang +4Task Transferability Estimation Cross-Domain Transfer Learning
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Meituan · Amazon AGI SF Lab +1
Emerging Human-like Strategies for Semantic Memory Foraging in Large Language Models
Mar 2, 2026Eric Lacosse, Mariana Duarte, Peter M. Todd +1LLM Interpretability
Champalimaud Research, Lisbon, Portugal · Centre for Restorative Neurotechnology, Lisbon, Portugal · Indiana University, Bloomington, IN, USA
Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers
Mar 2, 2026Kyle Cox, Darius Kianersi, Adrià Garriga-AlonsoFaithfulness of Language Model Explanations LLM Interpretability
Independent · Cambridge Boston Alignment Initiative (CBAI)
Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning
Mar 1, 2026Hamed Damirchi, Ignacio Meza De la Jara, Ehsan Abbasnejad +3LLM Interpretability Representation Geometry in Language Models
Australian Institute for Machine Learning, Adelaide University · Monash University · Concordia University
A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations
Feb 28, 2026Hossein JavidniaLLM Interpretability Representation Geometry in Language Models
School of Computing, Dublin City University, Ireland
From Stochastic Answers to Verifiable Reasoning: Interpretable Decision-Making with LLM-Generated Code
Feb 28, 2026Anirudh Jaidev Mahesh, Ben Griffin, Fuat Alican +8Rare-Event Prediction LLM Decision-Making
Georgia Institute of Technology · University of Oxford · Vela Research +1
The GRADIEND Python Package: An End-to-End System for Gradient-Based Feature Learning
Feb 27, 2026Jonathan Drechsel, Steffen HerboldLLM Interpretability Language Modeling
Faculty of Computer Science and Mathematics, University of Passau
GLUScope: A Tool for Analyzing GLU Neurons in Transformer Language Models
Feb 27, 2026Sebastian Gerstner, Leonor Veloso, Hinrich SchützeNeuron-Level Interpretability Transformer Interpretability
LMU Munich and Munich Center for Machine Learning (MCML), Munich, Germany
Induction Meets Biology: Mechanisms of Repeat Detection in Protein Language Models
Feb 26, 2026Gal Pomerants, Yaniv Nikankin, Anja Reusch +3Protein Language Models LLM Interpretability
Technion - Israel Institute of Technology · The Hebrew University of Jerusalem, Israel · Kempner Institute, Harvard University
Probing for Knowledge Attribution in Large Language Models
Feb 26, 2026Ivo Brink, Alexander Boer, Dennis UlmerLLM Interpretability Hallucination in Language Models
KPMG NL / University of Amsterdam Amsterdam, The Netherlands · KPMG NL Amsterdam, The Netherlands · University of Amsterdam Amsterdam, The Netherlands
Interpreting and Steering State-Space Models via Activation Subspace Bottlenecks
Feb 26, 2026Vamshi Sunku Mohan, Kaustubh Gupta, Aneesha Das +1State Space Language Models State Space Models
AppViewX, Bangalore · Independent Researcher · Microsoft Research, Redmond

Mar 15, 2026J/K move · Enter open · S save

Whether, Not Which: Mechanistic Interpretability Reveals Dissociable Affect Reception and Emotion Categorization in LLMs

Michael Keeman

Keido Labs, Liverpool, UK

Emotion Representation in Language Models LLM Interpretability

Open paper AskPDF arXiv

Large language models appear to develop internal representations of emotion -- "emotion circuits," "emotion neurons," and structured emotional manifolds have been reported across multiple model families. But every study making these claims uses stimuli signalled by explicit emotion keywords, leaving a fundamental question unanswered: do these circuits detect genuine emotional meaning, or do they detect the word "devastated"? We present the first clinical validity test of emotion circuit claims using mechanistic interpretability methods grounded in clinical psychology -- clinical vignettes that evoke emotions through situational and behavioural cues alone, emotion keywords removed. Across six models (Llama-3.2-1B, Llama-3-8B, Gemma-2-9B; base and instruct variants), we apply four convergent mechanistic interpretability methods -- linear probing, causal activation patching, knockout experiments, and representational geometry -- and discover two dissociable emotion processing mechanisms. Affect reception -- detecting emotionally significant content -- operates with near-perfect accuracy (AUROC 1.000), consistent with early-layer saturation, and replicates across all six models. Emotion categorization -- mapping affect to specific emotion labels -- is partially keyword-dependent, dropping 1-7% without keywords and improving with scale. Causal activation patching confirms keyword-rich and keyword-free stimuli share representational space, transferring affective salience rather than emotion-category identity. These findings falsify the keyword-spotting hypothesis, establish a novel mechanistic dissociation, and introduce clinical stimulus methodology as a rigorous standard for testing emotion processing claims in large language models -- with direct implications for AI safety evaluation and alignment. All stimuli, code, and data are released for replication.