LLM Interpretability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

57 papers in the last four weeks, up 73% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28

Latest papers 546

All topics

From Directions to Regions: Decomposing Activations in Language Models via Local Geometry
Feb 2, 2026Or Shafran, Shaked Ronen, Omri Fahn +3Language Model Steering LLM Interpretability
Blavatnik School of Computer Science and AI, Tel Aviv University, Israel · New York University, New York, NY, USA · Goodfire.
There Is More to Refusal in Large Language Models than a Single Direction
Feb 2, 2026Faaiz Joad, Majd Hawasly, Sabri Boughorbel +2LLM Interpretability LLM Refusal Behavior
Qatar Computing Research Institute, HBKU, Doha, Qatar
Functional Subspace, where language models can use vector algebra to solve problems
Feb 2, 2026Jung H. Lee, Sujith VijayanLLM Interpretability In-Context Learning
Pacific Northwest National Laboratory Seattle, WA · School of Neuroscience Virginia Tech Blacksburg, VA
Understanding LLM Failures: A Multi-Tape Turing Machine Analysis of Systematic Errors in Language Model Reasoning
Jan 27, 2026Magnus BomanLLM Interpretability Language Modeling
Karolinska Institutet, Stockholm, Sweden
YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models
Jan 22, 2026Junyu Lin, Meizhen Liu, Xiufeng Huang +12LLM Guardrails LLM Interpretability
Alibaba-AAIG
Tracing the Latent Threads: A Mechanistic Study of How LLMs Represent and Operationalize Race and Ethnicity Cues
Jan 19, 2026Shiyue Hu, Ruizhe Li, Yanjun GaoLLM Interpretability Mechanistic Interpretability
University of Colorado Anschutz · University of Colorado Boulder · University of Aberdeen
To Copy or Not to Copy: Copying Is Easier to Induce Than Recall
Jan 17, 2026Mehrdad Farahani, Franziska Penzkofer, Richard JohanssonKnowledge Conflicts in Language Models LLM Interpretability
1Chalmers University of Technology · University of Gothenburg
Relational Linearity is a Predictor of Hallucinations
Jan 16, 2026Yuetian Lu, Yihong Liu, Sebastian Gerstner +3LLM Interpretability Hallucination in Language Models
Center for Information and Language Processing (CIS), LMU Munich · Ubiquitous Knowledge Processing Lab (UKP Lab), Technical University of Darmstadt · Munich Center for Machine Learning (MCML)
Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs
Jan 16, 2026Lecheng Yan, Ruizhe Li, Guanhua Chen +5Reward Hacking Memorization in Language Models
Southern University of Science and Technology · University of Science and Technology of China · University of Aberdeen +5
Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
Jan 12, 2026Zhenghao He, Guangzhi Xiong, Bohan Liu +2LLM Interpretability CoT Reasoning
University of Virginia, USA
Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control
Jan 6, 2026Harshvardhan Saini, Yiming Tang, Dianbo LiuLanguage Model Steering LLM Interpretability
Indian Institute of Technology (ISM), Dhanbad · National University of Singapore
Base Models Know How to Reason, Thinking Models Learn When
Oct 8, 2025Constantin Venhoff, Iván Arcuschin, Philip Torr +2LLM Interpretability RL for Language Model Reasoning
University of Oxford, UK · MATS · Poseidon Research.
Vis-CoT: A Human-in-the-Loop Framework for Interactive Visualization and Intervention in LLM Chain-of-Thought Reasoning
Sep 1, 2025Kaviraj Pather, Elena Hadjigeorgiou, Arben Krasniqi +4LLM Interpretability Human-in-the-Loop AI
Department of Computer Science & Engineering University of Mauritius R´eduit 80837, Mauritius · Department of Electrical & Computer Engineering Cyprus University of Technology 3036 Limassol, Cyprus · Faculty of Electrical & Computer Engineering University of Prishtina ”Hasan Prishtina” 10000 Prishtina, Kosovo +4
Unraveling the cognitive patterns of Large Language Models through module communities
Aug 25, 2025Kushal Raj Bhandari, Pin-Yu Chen, Jianxi GaoComputational Cognitive Modeling LLM Interpretability
Department of Computer Science Rensselaer Polytechnic Institute Troy, NY, USA · IBM Research Yorktown Heights, NY, USA
Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
Aug 22, 2025David Chanin, Adrià Garriga-AlonsoActivation Sparsity LLM Interpretability
University College London · MATS Research
BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Injection
Aug 12, 2025Sekh Mainul Islam, Nadav Borenstein, Siddhesh Milind Pawar +3Social Bias in Language Models LLM Interpretability
University of Copenhagen
Too Categorical to be Human: Emotion Concepts in LLMs and Humans
Aug 7, 2025Sree Bhattacharyya, Evgenii Kuriabov, Lucas Craig +4LLM Interpretability Emotion Representation in Language Models
Department of Informatics and Intelligent Systems, College of Information Sciences and Technology, The Pennsylvania State University · Department of Statistics, Eberly College of Science, The Pennsylvania State University · Department of Computer Science and Engineering, College of Engineering, The Pennsylvania State University +1
LLMs Encode Harmfulness and Refusal Separately
Jul 16, 2025Jiachen Zhao, Jing Huang, Zhengxuan Wu +2LLM Interpretability LLM Refusal Behavior
Northeastern University · Stanford University
Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages
Jul 15, 2025Lyzander Marciano Andrylie, Inaya Rahmanisa, Mahardika Krisna Ihsani +3Multilingual Language Models Representation Learning
Faculty of Computer Science, Universitas Indonesia · Department of Natural Language Processing, MBZUAI
On the Effect of Uncertainty on Layer-wise Inference Dynamics
Jul 9, 2025Sunwoo Kim, Haneul Yoo, Alice OhLLM Inference LLM Interpretability
School of Computing, KAIST, Daejeon, South Korea.
The Trilemma of Truth in Large Language Models
Jun 30, 2025Germans Savcisens, Tina Eliassi-RadLLM Interpretability LLM Reliability
Khoury College of Computer Sciences Northeastern University, Boston, MA, USA · Network Science Institute Northeastern University, Boston, MA, USA · Santa Fe Institute, Santa Fe, NM, USA
Cross-Layer Discrete Concept Discovery for Interpreting Language Models
Jun 24, 2025Ankur Garg, Xuemin Yu, Hassan Sajjad +1Variational Autoencoders LLM Interpretability
InverseScope: Scalable Activation Inversion for Interpreting Large Language Models
Jun 9, 2025Yifan Luo, Zhennan Zhou, Bin DongRepresentation Geometry LLM Interpretability
School of Mathematical Science, Peking University, Beijing, China · School of Science, Westlake University, Hangzhou, China · Beijing International Center for Mathematical Research and the New Cornerstone Science Laboratory, Peking University, Beijing, China +1
When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning
May 21, 2025Rongzhi Zhu, Yi Liu, Jiancheng Wang +6Overthinking in Language Models LLM Interpretability
State Key Laboratory for Novel Software Technology, Nanjing University, China · China Unicom Software Research Institute, China · National Institute of Healthcare Data Science, Nanjing University, China +1
Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations
Apr 7, 2025Pedro Ferreira, Wilker Aziz, Ivan TitovCoT Faithfulness Reward Hacking
Institute for Logic, Language and Computation (ILLC), University of Amsterdam · Institute for Language, Cognition and Computation (ILCC), University of Edinburgh
Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations
Mar 17, 2025Noah Y. Siegel, Nicolas Heess, Maria Perez-Ortiz +1Faithfulness of Language Model Explanations LLM Interpretability
Google DeepMind · Centre for AI, University College London · Imperial College London +1
LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers
Feb 20, 2025Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev +4Transformer Interpretability Long-Context Language Modeling
AIRI · Skoltech · Lomonosov Moscow State University +1
Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning
Dec 14, 2024Julia Witte Zimmerman, Denis Hudon, Kathryn Cramer +9LLM Interpretability Distributional Semantics
Computational Story Lab, Vermont Complex Systems Institute, University of Vermont, Burlington, VT 05405, USA · Vermont Complex Systems Institute, University of Vermont, Burlington, VT 05405, USA · Department of Computer Science, St. Michael’s College, Colchester, VT 05439, USA +3
Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models
Oct 3, 2024Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva +5Self-Attention Multiple-Choice Question Answering
Skolkovo Institute of Science and Technology · AI Foundation and Algorithm Lab · Moscow Institute of Physics and Technology +2
How LLMs Follow Instructions: Skillful Coordination, Not a Universal Mechanism
Date pendingElisabetta Rocchetti, Alfio FerraraLLM Interpretability Instruction Following
Department of Computer Science, Università degli Studi di Milano

Feb 2, 2026, cs.CLJ/K move · Enter open · S save

From Directions to Regions: Decomposing Activations in Language Models via Local Geometry

Or Shafran, Shaked Ronen, Omri Fahn, Shauli Ravfogel+2

Blavatnik School of Computer Science and AI, Tel Aviv University, Israel · New York University, New York, NY, USA · Goodfire.

Language Model Steering LLM Interpretability Gaussian Mixture Models Language Modeling Representation Geometry in Language Models

Open paper AskPDF arXiv

Activation decomposition methods in language models are tightly coupled to geometric assumptions on how concepts are realized in activation space. Existing approaches search for individual global directions, implicitly assuming linear separability, which overlooks concepts with nonlinear or multi-dimensional structure. In this work, we leverage Mixture of Factor Analyzers (MFA) as a scalable, unsupervised alternative that models the activation space as a collection of Gaussian regions with their local covariance structure. MFA decomposes activations into two compositional geometric objects: the region's centroid in activation space, and the local variation from the centroid. We train large-scale MFAs for Llama-3.1-8B and Gemma-2-2B, and show they capture complex, nonlinear structures in activation space. Moreover, evaluations on localization and steering benchmarks show that MFA outperforms unsupervised baselines, is competitive with supervised localization methods, and often achieves stronger steering performance than sparse autoencoders. Together, our findings position local geometry, expressed through subspaces, as a promising unit of analysis for scalable concept discovery and model control, accounting for complex structures that isolated directions fail to capture.

Momentum

Related topics

Latest papers 546