LLM Interpretability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

57 papers in the last four weeks, up 73% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28

Latest papers 546

All topics

Rational Sparse Autoencoder
Jun 12, 2026Naiyu Yin, Yue YuActivation Sparsity LLM Interpretability
Department of Mathematics · Lehigh University · Bethlehem, PA 18015
A Definition of Good Explanations and the Challenges Explaining LLM Outputs
Jun 12, 2026Louis Mahon, Elliot Ford, Callum HackettExplainable Artificial Intelligence Counterfactual Explanations
UnlikelyAI, London, UK
A Low-Rank Subspace Analysis of LLM Interventions
Jun 12, 2026Angira Sharma, Christian Schroeder de Witt, Philip Torr +2LLM Interpretability LLM Refusal Behavior
University of Oxford
When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs
Jun 12, 2026Boris Marinov, Angira Sharma, Christian Schroeder de Witt +3Multilingual Language Models LLM Interpretability
1UCL · University of Oxford
Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry
Jun 11, 2026Jennifer Meng Lu, Ruochen Zhang, Isabelle Lee +3LLM Interpretability Representation Geometry in Language Models
Brown University · University of Southern California · Harvard University +1
Reasoning as Pattern Matching: Shared Mechanisms in Human and LLM Everyday Reasoning
Jun 11, 2026Zach Studdiford, Gary LupyanCognitive Modeling LLM Interpretability
University of Wisconsin–Madison Department of Psychology Department of Computer Science
Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
Jun 11, 2026Daniel Scalena, Sara Candussio, Luca Bortolussi +3CoT Faithfulness LLM Interpretability
CLCG, University of Groningen · University of Milano-Bicocca · University of Trieste +1
Localizing Anchoring Pathways in Language Models
Jun 11, 2026Hillary N. Owusu, Sarah Wiegreffe, Naomi H. FeldmanNumerical Reasoning in Language Models LLM Interpretability
University of Maryland, College Park
Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal
Jun 10, 2026Leon Bergen, Usha Bhalla, Sidharth Baskaran +14Reward Shaping LLM Interpretability
Core contributors
ICA Lens: Interpreting Language Models Without Training Another Dictionary
Jun 10, 2026Sida Liu, Feijiang HanIndependent Component Analysis LLM Interpretability
Independent Researcher · University of Maryland
Bergson: An Open Source Library for Data Attribution
Jun 10, 2026Lucia Quirke, Louis Jaburi, David Johnston +6Gradient-Based Attribution LLM Interpretability
EleutherAI · Independent
Forecasting Future Behavior as a Learning Task
Jun 9, 2026Mosh Levy, Yoav Goldberg, Asa Cooper SticklandLLM Interpretability Large Reasoning Models
Bar-Ilan University · Constellation · UK AI Security Institute +1
How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation
Jun 9, 2026Ian van Dort, Maria HeussRetrieval-Augmented Generation LLM Interpretability
University of Amsterdam
MIRAGE: A Polarity-Flipping Encoding Subspace in LLM Agents
Jun 9, 2026Pratibha Revankar, Kargi Chauhan, Jihye Kim +3LLM Interpretability AI Agent Security
University of California, Santa Cruz.
Interpreting and Steering a Text-to-Speech Language Model with Sparse Autoencoders
Jun 8, 2026Nikita Koriagin, Georgii Aparin, Nikita Balagansky +1TTS Synthesis Language Model Steering
♣T-Tech · ♦AI Foundation and Algorithm Lab
The Neutral Mask: How Alignment Training Provides Shallow Alignment while Leaving Partisan Structure Intact in a Large Language Model
Jun 8, 2026Wendy K. TamLLM Alignment LLM Interpretability
Professor and Stevenson Chair in the Departments of Political Science, Computer Science, Biomedical Informatics, and the Law School at Vanderbilt University, and an affiliate of the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign
PRISM: Recovering Instruction Sets from Language Model Activations
Jun 8, 2026Gilad Gressel, Rahul Pankajakshan, Julia Diament +3Neural Decoding LLM Interpretability
Center for Cybersecurity Systems & Networks, Amrita Vishwa Vidyapeetham · Ben-Gurion University of the Negev · Microsoft
The Amplifying Mirror: Locating and Steering the Partisan Direction inside a Large Language Model
Jun 7, 2026Wendy K. TamLLM Interpretability Political Bias in Language Models
Professor and Stevenson Chair in the Departments of Political Science, Computer Science, Biomedical Informatics, and the Law School at Vanderbilt University
Analyzing the Correlation Between Hallucinations and Knowledge Conflicts in Large Language Models
Jun 7, 2026Lucrezia Laraspata, Giovanna Castellano, Gennaro VessioKnowledge Conflicts in Language Models LLM Interpretability
Department of Computer Science, University of Bari Aldo Moro, Bari, Italy
Inside the LLM Word Factory
Jun 7, 2026Benzi Busigin, Yuval PinterLanguage Model Decoding LLM Interpretability
Stein Faculty of Computer and Information Science Ben-Gurion University of the Negev Beer Sheva, Israel
Explaining Black-Box Language Models: Learning to Optimize Linguistically-Structured Word Subsets
Jun 7, 2026Minyoung Hwang, Seokhyun Lee, Changhee LeeFeature Attribution LLM Interpretability
Korea University Republic of Korea
Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects
Jun 6, 2026Evan DuanLanguage Model Steering LLM Interpretability
Cross-LLM Consistency in Inference: Evidence from Shared Interactions
Jun 6, 2026Siyu Lou, Yao Yan, Yuntian Chen +1LLM Inference LLM Interpretability
School of Computer Science Shanghai Jiao Tong University Ningbo Key Laboratory of Advanced Manufacturing Simulation Eastern Institute of Technology, Ningbo · College of Computer and Information Science Chongqing Normal University · Ningbo Key Laboratory of Advanced Manufacturing Simulation Eastern Institute of Technology, Ningbo
MechLens: Late Crystallization of Factual Knowledge Explains Intervention Effectiveness in Language Models
Jun 6, 2026Xueping GaoMemorization in Language Models LLM Interpretability
Alibaba Cloud
Beyond Post-hoc Explanation: Toward Glassbox AI via Probabilistic Mediation
Jun 5, 2026Manuele LeonelliExplainable Artificial Intelligence AI Accountability
School of Science and Technology, IE University, Madrid, Spain
When Attribution Patching Lies: Diagnosis and a Second-Order Correction
Jun 5, 2026Luyang Zhang, Jialu WangGradient-Based Attribution LLM Interpretability
1Carnegie Mellon University · University of California, Santa Cruz
Interpreting Brain Responses to Language with Sparse Features from Language Models
Jun 5, 2026Michael A. Lepori, Kendrick Kay, Greta TuckuteLLM Interpretability Sparse Autoencoders
Dept. of Computer Science Brown University · Dept. of Radiology University of Minnesota · Kempner Institute Harvard University
LLM Self-Recognition: Steering and Retrieving Activation Signatures
Jun 4, 2026Thibaud Ardoin, Jonas Schäfer, Gerhard WunderAI-Generated Text Detection LLM Interpretability
Department of Computer Science, Freie Universitaet Berlin, Germany.
The Tell-Tale Norm: $\ell_2$ Magnitude as a Signal for Reasoning Dynamics in Large Language Models
Jun 4, 2026Jinyang Zhang, Hongxin Ding, Yue Fang +4LLM Interpretability Test-Time Scaling
School of Computer Science, Peking University · National Engineering Research Center of Software Engineering, Peking University · Key Laboratory of High Confidence Software Technologies, Ministry of Education +2
LLM Explainability with Counterfactual Chains and Causal Graphs
Jun 4, 2026Nirit Nussbaum-Hoffer, Nitay Calderon, Liat Ein-Dor +1LLM Reasoning with Graphs LLM Interpretability
Faculty of Data and Decision Sciences, Technion · IBM Research

Jun 12, 2026, cs.LGJ/K move · Enter open · S save

Rational Sparse Autoencoder

Naiyu Yin, Yue Yu

Department of Mathematics · Lehigh University · Bethlehem, PA 18015

Activation Sparsity LLM Interpretability Sparse Autoencoders Sparse Coding

Open paper AskPDF arXiv

Sparse autoencoders (SAEs) are standard tools for mechanistic interpretability, but current SAE families are constrained by fixed encoder nonlinearities such as ReLU, JumpReLU, and TopK. This hard-codes a particular sparsity mechanism into the model and can distort the reconstruction-versus-sparsity trade-off. We introduce the Rational Sparse Autoencoder (RSAE), which replaces the fixed encoder activation with a trainable rational function. Rational activations are flexible enough to uniformly approximate the activation primitives used by existing SAE families on compact domains (for TopK, the thresholded gate obtained after a separating top-k threshold is supplied), while also providing a richer function class for adapting to the observed pre-activation geometry. We realise this idea through a two-stage pipeline: an initialisation procedure that copies the pre-trained baseline SAE weights, plugs in rational coefficients obtained by the relaxed Remez exchange on synthetic data, and calibrates the scale parameters along with the rational coefficients; followed by a fine-tuning step under the standard sparsity-regularised reconstruction objective. Empirically, on residual-stream activations of three open-weight language models and across all three baseline activation families, the RSAE strictly improves on it after the fine-tuning step, both on reconstruction-side metrics and on downstream-behaviour metrics, without sacrificing feature-level interpretability under sparse probing. These gains are consistent across host language models, across baseline activation families, and across the full range of baseline sparsity we tested, while the upgrade itself adds only a handful of scalar parameters per autoencoder and runs in minutes on a single consumer GPU.

Momentum

Related topics

Latest papers 546