LLM Interpretability

LLM: Large Language Model

Large language model interpretability asks what information models represent, how it moves through layers and components, and which computations cause particular answers or behaviors. Researchers use probes and representation geometry, sparse autoencoders, attribution and circuit tracing, activation patching or ablation, and steering to study factual recall, mathematical reasoning, hallucination, refusal, and in-context learning. These methods can reveal distributed circuits and layer-wise signals, and sometimes support targeted interventions without retraining. A central challenge is that information being decodable—or a direction being steerable—does not prove the model uses it; causal faithfulness, robustness across prompts and model families, and reliable evaluation remain open tensions.

Latest papers

781 papers82 papers in the last four weeks, up 12% on the four weeks before. 0.5% of all new papers.

YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models
Jan 22, 2026Junyu Lin, Meizhen Liu, Xiufeng Huang +12LLM Guardrails LLM Interpretability
Alibaba-AAIG
Tracing the Latent Threads: A Mechanistic Study of How LLMs Represent and Operationalize Race and Ethnicity Cues
Jan 19, 2026Shiyue Hu, Ruizhe Li, Yanjun GaoLLM Interpretability Mechanistic Interpretability
University of Colorado Anschutz · University of Colorado Boulder · University of Aberdeen
To Copy or Not to Copy: Copying Is Easier to Induce Than Recall
Jan 17, 2026Mehrdad Farahani, Franziska Penzkofer, Richard JohanssonKnowledge Conflicts in Language Models LLM Interpretability
1Chalmers University of Technology · University of Gothenburg
Relational Linearity is a Predictor of Hallucinations
Jan 16, 2026Yuetian Lu, Yihong Liu, Sebastian Gerstner +3LLM Interpretability Hallucination in Language Models
Center for Information and Language Processing (CIS), LMU Munich · Ubiquitous Knowledge Processing Lab (UKP Lab), Technical University of Darmstadt · Munich Center for Machine Learning (MCML)
Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs
Jan 16, 2026Lecheng Yan, Ruizhe Li, Guanhua Chen +5Reward Hacking Memorization in Language Models
Southern University of Science and Technology · University of Science and Technology of China · University of Aberdeen +5
Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
Jan 12, 2026Zhenghao He, Guangzhi Xiong, Bohan Liu +2LLM Interpretability CoT Reasoning
University of Virginia, USA
Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencoders
Jan 6, 2026Ruikang Zhang, Shuo Wang, Qi SuLLM Interpretability Activation Steering
Peking University, Beijing, China
Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control
Jan 6, 2026Harshvardhan Saini, Yiming Tang, Dianbo LiuLanguage Model Steering LLM Interpretability
Indian Institute of Technology (ISM), Dhanbad · National University of Singapore
AlignSAE: Concept-Aligned Sparse Autoencoders
Dec 1, 2025Minglai Yang, Xinyu Guo, Zhengliang Shi +4Sparse Autoencoders LLM Interpretability
University of Arizona · Shandong University · LMU Munich +1
Rep2Text: Decoding Full Text from a Single LLM Token Representation
Nov 9, 2025Haiyan Zhao, Zirui He, Yiming Tang +4LLM Interpretability LLM Information Extraction
New Jersey Institute of Technology · National University of Singapore · Wake Forest University +2
Fine-grained Analysis of Brain-LLM Alignment through Input Attribution
Oct 14, 2025Michela Proietti, Roberto Capobianco, Mariya TonevaLLM Interpretability Next-Token Prediction
Sapienza University of Rome, Italy · Sony AI, Zurich, Switzerland · Max Planck Institute for Software Systems, Germany
Base Models Know How to Reason, Thinking Models Learn When
Oct 8, 2025Constantin Venhoff, Iván Arcuschin, Philip Torr +2LLM Interpretability RL for Language Model Reasoning
University of Oxford, UK · MATS · Poseidon Research.
Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic Architectures
Sep 29, 2025Marco Bronzini, Carlo Nicolini, Bruno Lepri +2LLM Interpretability Language Model Probing
Vis-CoT: A Human-in-the-Loop Framework for Interactive Visualization and Intervention in LLM Chain-of-Thought Reasoning
Sep 1, 2025Kaviraj Pather, Elena Hadjigeorgiou, Arben Krasniqi +4LLM Interpretability Human-in-the-Loop AI
Department of Computer Science & Engineering University of Mauritius R´eduit 80837, Mauritius · Department of Electrical & Computer Engineering Cyprus University of Technology 3036 Limassol, Cyprus · Faculty of Electrical & Computer Engineering University of Prishtina ”Hasan Prishtina” 10000 Prishtina, Kosovo +4
Unraveling the cognitive patterns of Large Language Models through module communities
Aug 25, 2025Kushal Raj Bhandari, Pin-Yu Chen, Jianxi GaoComputational Cognitive Modeling LLM Interpretability
Department of Computer Science Rensselaer Polytechnic Institute Troy, NY, USA · IBM Research Yorktown Heights, NY, USA
Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders
Aug 22, 2025David Chanin, Adrià Garriga-AlonsoActivation Sparsity LLM Interpretability
University College London · MATS Research
BiasGym: A Simple and Generalizable Framework for Analyzing and Removing Biases through Injection
Aug 12, 2025Sekh Mainul Islam, Nadav Borenstein, Siddhesh Milind Pawar +3Social Bias in Language Models LLM Interpretability
University of Copenhagen
Too Categorical to be Human: Emotion Concepts in LLMs and Humans
Aug 7, 2025Sree Bhattacharyya, Evgenii Kuriabov, Lucas Craig +4LLM Interpretability Emotion Representation in Language Models
Department of Informatics and Intelligent Systems, College of Information Sciences and Technology, The Pennsylvania State University · Department of Statistics, Eberly College of Science, The Pennsylvania State University · Department of Computer Science and Engineering, College of Engineering, The Pennsylvania State University +1
LLMs Encode Harmfulness and Refusal Separately
Jul 16, 2025Jiachen Zhao, Jing Huang, Zhengxuan Wu +2LLM Interpretability LLM Refusal Behavior
Northeastern University · Stanford University
Sparse Autoencoders Can Capture Language-Specific Concepts Across Diverse Languages
Jul 15, 2025Lyzander Marciano Andrylie, Inaya Rahmanisa, Mahardika Krisna Ihsani +3Multilingual Language Models Representation Learning
Faculty of Computer Science, Universitas Indonesia · Department of Natural Language Processing, MBZUAI
On the Effect of Uncertainty on Layer-wise Inference Dynamics
Jul 9, 2025Sunwoo Kim, Haneul Yoo, Alice OhLLM Inference LLM Interpretability
School of Computing, KAIST, Daejeon, South Korea.
The Trilemma of Truth in Large Language Models
Jun 30, 2025Germans Savcisens, Tina Eliassi-RadLLM Interpretability LLM Reliability
Khoury College of Computer Sciences Northeastern University, Boston, MA, USA · Network Science Institute Northeastern University, Boston, MA, USA · Santa Fe Institute, Santa Fe, NM, USA
Cross-Layer Discrete Concept Discovery for Interpreting Language Models
Jun 24, 2025Ankur Garg, Xuemin Yu, Hassan Sajjad +1Variational Autoencoders LLM Interpretability
InverseScope: Scalable Activation Inversion for Interpreting Large Language Models
Jun 9, 2025Yifan Luo, Zhennan Zhou, Bin DongRepresentation Geometry LLM Interpretability
School of Mathematical Science, Peking University, Beijing, China · School of Science, Westlake University, Hangzhou, China · Beijing International Center for Mathematical Research and the New Cornerstone Science Laboratory, Peking University, Beijing, China +1
Localizing Persona Representations in LLMs
May 30, 2025Celia Cintas, Miriam Rateike, Erik Miehling +2LLM Interpretability Personality Modeling in Language Models
IBM Research Africa · IBM Research Europe
When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning
May 21, 2025Rongzhi Zhu, Yi Liu, Jiancheng Wang +6Overthinking in Language Models LLM Interpretability
State Key Laboratory for Novel Software Technology, Nanjing University, China · China Unicom Software Research Institute, China · National Institute of Healthcare Data Science, Nanjing University, China +1
Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations
Apr 7, 2025Pedro Ferreira, Wilker Aziz, Ivan TitovCoT Faithfulness Reward Hacking
Institute for Logic, Language and Computation (ILLC), University of Amsterdam · Institute for Language, Cognition and Computation (ILCC), University of Edinburgh
Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations
Mar 17, 2025Noah Y. Siegel, Nicolas Heess, Maria Perez-Ortiz +1Faithfulness of Language Model Explanations LLM Interpretability
Google DeepMind · Centre for AI, University College London · Imperial College London +1
LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers
Feb 20, 2025Anton Razzhigaev, Matvey Mikhalchuk, Temurbek Rahmatullaev +4Transformer Interpretability Long-Context Language Modeling
AIRI · Skoltech · Lomonosov Moscow State University +1
Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning
Dec 14, 2024Julia Witte Zimmerman, Denis Hudon, Kathryn Cramer +9LLM Interpretability Distributional Semantics
Computational Story Lab, Vermont Complex Systems Institute, University of Vermont, Burlington, VT 05405, USA · Vermont Complex Systems Institute, University of Vermont, Burlington, VT 05405, USA · Department of Computer Science, St. Michael’s College, Colchester, VT 05439, USA +3

Jan 22, 2026J/K move · Enter open · S save

YuFeng-XGuard: A Reasoning-Centric, Interpretable, and Flexible Guardrail Model for Large Language Models

Junyu Lin, Meizhen Liu, Xiufeng Huang, Jinfeng Li+11

Alibaba-AAIG

LLM Guardrails LLM Interpretability LLM Safety

Open paper AskPDF arXiv

As large language models (LLMs) are increasingly deployed in real-world applications, safety guardrails are required to go beyond coarse-grained filtering and support fine-grained, interpretable, and adaptable risk assessment. However, existing solutions often rely on rapid classification schemes or post-hoc rules, resulting in limited transparency, inflexible policies, or prohibitive inference costs. To this end, we present YuFeng-XGuard, a reasoning-centric guardrail model family designed to perform multi-dimensional risk perception for LLM interactions. Instead of producing opaque binary judgments, YuFeng-XGuard generates structured risk predictions, including explicit risk categories and configurable confidence scores, accompanied by natural language explanations that expose the underlying reasoning process. This formulation enables safety decisions that are both actionable and interpretable. To balance decision latency and explanatory depth, we adopt a tiered inference paradigm that performs an initial risk decision based on the first decoded token, while preserving ondemand explanatory reasoning when required. In addition, we introduce a dynamic policy mechanism that decouples risk perception from policy enforcement, allowing safety policies to be adjusted without model retraining. Extensive experiments on a diverse set of public safety benchmarks demonstrate that YuFeng-XGuard achieves stateof-the-art performance while maintaining strong efficiency-efficacy trade-offs. We release YuFeng-XGuard as an open model family, including both a full-capacity variant and a lightweight version, to support a wide range of deployment scenarios.