LLM Interpretability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

57 papers in the last four weeks, up 73% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28

Latest papers 546

All topics

Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Models
May 29, 2026Zhiwen You, Nafiseh Nikeghbal, Jana DiesnerGender Bias in Language Models LLM Interpretability
University of Illinois Urbana-Champaign · Technical University of Munich · Munich Center for Machine Learning
Human-Alignment, Calibration, and Activation Patterns in Large Language Model Uncertainty
May 29, 2026Kyle Moore, Jesse Roberts, Daryl Watson +2Confidence Estimation in Language Models LLM Interpretability
1Vanderbilt University · 2Tennessee Technological University
Do Language Models Track Entities Across State Changes?
May 28, 2026Zilu Tang, Qiao Zhao, Gabriel Franco +4LLM Interpretability State Tracking
Department of Computer Science, Boston University, Boston, USA · Department of Data Science, Monash University, Indonesia · Faculty of Computer Science, University of Vienna, Austria +1
Latent Performance Profiling of Large Language Models
May 28, 2026Tanmoy Chakraborty, Ayan Sengupta, Suparna Bhattacharya +7LLM Evaluation LLM Interpretability
Department of Electrical Engineering, Indian Institute of Technology Delhi, New Delhi India · Yardi School of Artiﬁcial Intelligence, Indian Institute of Technology Delhi, New Delhi India · Hewlett Packard Enterprise, India +5
Dissecting the Black Box: Circuit-Level Analysis of LLM Vulnerability Detection
May 28, 2026Syafiq Al Atiiq, Chun Zhou, Christian GehrmannLLM Interpretability Software Vulnerability Detection
Lund University Lund, Sweden
Internal Representation, Not Clinical Knowledge: Where Apparent LLM Triage Failures Originate
May 28, 2026David Fraile Navarro, Berardino Como, Jialei Sheng +2Healthcare Clinical Triage
Macquarie University, Sydney, Australia · Politecnico di Bari, Bari, Italy · NSW Health, Sydney, Australia +1
Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
May 28, 2026Adly Templeton, Tom Conerly, Jonathan Marcus +23LLM Interpretability Sparse Autoencoders
Anthropic
When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception
May 28, 2026Vahideh ZolfaghariLinear Probing Language Model Safety Evaluation
Algoverse AI Research, Algoverse AI Research Mentorship Program, USA. · Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran. · Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text
May 27, 2026Tianyang Zhou, Wenbo Chen, Pierre Jinghong Liang +1LLM Interpretability Text Classification
1Carnegie Mellon University · 2Amazon
Interpretability-Guided Layer Selection over Subspace Projection: SAEs as Stethoscopes, Not Scalpels, for Raw Task Vector Model Editing
May 27, 2026Li Lei, Madalina Ciobanu, Qingqing Mao +1Task Vectors LLM Interpretability
Incept Labs Houston, TX · Incept Labs, Houston, TX · Titan Holdings, San Francisco, CA
Refusal Before Decoding: Detecting and Exploiting Refusal Signals in Intermediate LLM Activations
May 27, 2026Matteo Gioele Collu, Riccardo Conte, Alberto Giaretta +4LLM Interpretability Adversarial Attacks on LLMs
University of Padua, Italy · 2Örebro University, Sweden · 3Fondazione Bruno Kessler, Italy
Cultural Binding Heads in Language Models
May 27, 2026Avrile Floro, Luca BenedettoLLM Interpretability Cultural Bias in Language Models
Institut Polytechnique de Paris · Télécom SudParis, Institut Polytechnique de Paris
Feature Geometry of LoRA Adapters: A Sparse Autoencoder Analysis of Representational Divergence in Fine-Tuned Language Models
May 27, 2026Prasanth K KLLM Interpretability Sparse Autoencoders
Independent AI Safety Researcher
Sign-Aware Gated Sparse Autoencoders: Modeling Anticorrelated Features with Bi-Jump-ReLU Activations
May 27, 2026Bartosz Wieciech, Zmnako Awrahman, Marcin Czelej +2Activation Sparsity LLM Interpretability
Amazon Web Services
Integrated and Cross-Architecture Interpretation of LLM Reasoning
May 27, 2026Leonardo Matthew Yauw, Wei-Bin Kou, Yujiu YangTransformer Interpretability LLM Interpretability
Tsinghua Shenzhen International Graduate School · Harbin Institute of Technology, Shenzhen
Where Does Toxicity Live? Mechanistic Localization and Targeted Suppression in Language Models
May 27, 2026Himanshu Beniwal, Mayank SinghLanguage Model Safety Evaluation LLM Interpretability
Indian Institute of Technology Gandhinagar
Geometry of Human Perceptual Domains Emerges Transiently in LLM Representations
May 27, 2026Simardeep Singh, Paras ChopraNeural Representation Geometry LLM Interpretability
Indian Institute of Technology Roorkee, Roorkee, India · Lossfunk, India
Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
May 27, 2026Sachin KumarLLM Evaluation Language Model Safety Evaluation
LexisNexis, USA
Do Agents Think Deeper? A Mechanistic Investigation of Layer-Wise Dynamics in Sequential Planning
May 27, 2026Zhenyu Cui, Xiangzhong LuoLLM Interpretability Mechanistic Interpretability
CAREF: Calibration-Aware Regularization for Explanation Faithfulness Without Rationale Supervision
May 27, 2026Naphat Nithisopa, Teerapong PanboonyuenFaithfulness of Language Model Explanations LLM Interpretability
†MARSAIL · ‡Chulalongkorn University · §PBYAIL (Panboonyuen AI Lab)
Revealing Algorithmic Deductive Circuits for Logical Reasoning
May 27, 2026Phuong Minh Nguyen, Tien Huu Dang, Naoya InoueAttention Head Analysis LLM Interpretability
Japan Advanced Institute of Science and Technology
Do Models Know Why They Changed Their Mind? Interpretability and Faithfulness of Chain-of-Thought Under Knowledge Conflict
May 26, 2026Pruthvinath Jeripity VenkataCoT Faithfulness Faithfulness of Language Model Explanations
Independent Researcher
Learning to Translate from Soft to Hard LLM Prompts
May 26, 2026Pitipat Kongsomjit, Suryansh Goyal, Jacob WhitehillLLM Prompting LLM Interpretability
Worcester Polytechnic Institute
What Makes Chain-of-Thought Work at Probe Time? Local Co-occurrence Rather Than Global Derivation
May 26, 2026Xiang Wang, Wei WeiLLM Interpretability CoT Reasoning
Huazhong University of Science and Technology
The Attribution Blind Spot: Detecting When Language Models Rely on Memory Rather Than Retrieved Context
May 26, 2026Zhe Yu, Wenpeng Xing, Yunzhao Wei +4Memorization in Language Models LLM Interpretability
Binjiang Institute of Zhejiang University · Zhejiang University · National Fintech Evaluation Center +2
Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations
May 25, 2026Shanghao Li, Jinda Han, Yibo Wang +5LLM Grounding LLM Interpretability
University of Illinois Chicago · University of Illinois Urbana-Champaign
Can LLMs Introspect? A Reality Check
May 25, 2026Shashwat Singh, Tal Linzen, Shauli RavfogelLLM Evaluation LLM Interpretability
Center for Data Science New York University
Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals
May 25, 2026Federico Torrielli, Peter Schneider-Kamp, Lukas Galke PoechConfidence Estimation in Language Models LLM Interpretability
University of Turin · University of Southern Denmark
Universal Activation Verbalizer: A Unified Framework for Cross-Model Activation Explanation
May 25, 2026Haiyan Zhao, Zirui He, Guanchu Wang +3Transformer Interpretability LLM Interpretability
New Jersey Institute of Technology · University of North Carolina at Charlotte · Cisco Research +1
Priors Persist Through Suppression: A Stroop Paradigm for Lexical Override
May 25, 2026Han-yu WangKnowledge Conflicts in Language Models LLM Interpretability
The University of Hong Kong

May 29, 2026, cs.CLJ/K move · Enter open · S save

Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Models

Zhiwen You, Nafiseh Nikeghbal, Jana Diesner

University of Illinois Urbana-Champaign · Technical University of Munich · Munich Center for Machine Learning

Gender Bias in Language Models LLM Interpretability Controllable Text Generation Causal Interventions in Language Models

Open paper AskPDF arXiv

Language models (LMs) can produce gendered language and stereotypes even when given neutral prompts. Most prior work on gender bias in LMs primarily examines gender through a binary lens (feminine vs. masculine), with limited attention to gender-neutral forms, such as they/them pronouns or neutrally phrased job titles. How gender-related signals are encoded in the internal representations of LMs remains an open question. In this work, we study gender-specific neurons in LMs across three categories: feminine, masculine, and gender-neutral. We propose a neuron-level intervention method to identify neurons that are strongly tied to each gender category. We then test these neurons through controlled generation, showing that activating or masking gender-related neurons can steer a sentence toward a target gender form while preserving its original meaning. To evaluate the effectiveness of our gender-intervention approach, we curate two datasets with controlled sentences labeled across all three gender categories and validate the data quality through human evaluation. Experiments on two open-source LMs show that gender-specific neurons are not evenly distributed across model layers; instead, they concentrate heavily in the earliest layers with smaller contributions from later layers. Compared to existing methods, our method achieves more precise gender control, with less leakage into non-target gender categories and stable output quality through two evaluation criteria. Overall, our work examines how gender is encoded in LMs and provides a simple yet effective approach toward controlled gender intervention for both neuron intervention evaluation and gender bias mitigation. Code and datasets are available at: https://github.com/zhiwenyou103/Gender-Neuron-Intervention

Momentum

Related topics

Latest papers 546