LLM Interpretability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

57 papers in the last four weeks, up 73% on the four weeks before. 0.6% of all new papers.

Jul 13Week of Sep 28

Latest papers 546

All topics

Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
Jul 30, 2026Pere Martra, Eugenio Martínez Cámara, Alfonso Ureña LópezLLM Pruning LLM Interpretability
Universidad Internacional Men´endez Pelayo Barcelona, Spain · Departamento de Inform´atica Universidad de Ja´en Ja´en, Spain
RepBench: Compiling Benchmarks into Capability Representations for Large Language Models
Jul 30, 2026Yanshi Li, Xueru Bai, Shuman Liu +1LLM Evaluation LLM Interpretability
Shopee, Shanghai, China · Shopee, Beijing, China
Reasoning Consensus: Structural Ensembling of LLM Reasoning via Weighted DAG Aggregation
Jul 30, 2026Amruta Parulekar, Jinu Lee, Dilek Hakkani-Tür +1LLM Reasoning with Graphs LLM Interpretability
University of Illinois Urbana-Champaign
Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures
Jul 30, 2026SiYuan Ma, Yiqin Luo, Zhangji +8LLM Interpretability Causal Abstraction
1Nanyang Technological University · 2Southern University of Science and Technology · 3Tianjin University +7
From Representations to Behaviors: Exploring the Person-Situation-Behavior Triad in LLMs
Jul 29, 2026Ruikang Zhang, Shuo Wang, Qi SuLLM Interpretability Personality Modeling in Language Models
1Peking University, Beijing, China · 2Beijing Institute of Technology, Beijing, China
From Found to Designed: Concepts as a Design Axis for Large Language Models
Jul 29, 2026Chen ShaniLLM Grounding LLM Interpretability
Tel Aviv University
Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs. SFT Fine-Tuned Models
Jul 28, 2026Antyabha Rahman, Akshaj Gurugubelli, Omar Ankit +2Numerical Reasoning in Language Models LLM Interpretability
University of New South Wales · Algoverse AI Research · University of Waterloo +1
Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
Jul 28, 2026Deepanshu Mody, Samarth Agarwal, Utkarsh Mittal +1Language Model Safety Evaluation LLM Interpretability
Center for Data Science, New York University
How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
Jul 28, 2026Ramtin Ehsani, Irene Manotas, Saurabh Pujar +2Automated Program Repair LLM Interpretability
Drexel University Philadelphia, PA, USA · IBM Research Yorktown, NY, USA
How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories
Jul 28, 2026Hui Wei, Junda Wu, Sheldon Yu +8LLM Interpretability Representation Geometry in Language Models
UC Merced · UC San Diego · UIUC +1
LLMs Can Annotate Attribution Graphs
Jul 28, 2026Ameen Patel, Max Zhang, Nathan HuLLM Interpretability LLM-Assisted Annotation
Stanford University
LEX-EC: A Lexical Evidence-Channel Audit Framework for Zero-Shot LLM Personality Classification in Black-Box Settings
Jul 27, 2026Brittany Harbison, Ashok K. GoelZero-Shot Text Classification LLM Auditing
Georgia Institute of Technology Atlanta, Georgia, USA
Context Is King: How In-Context Specification Shapes the Geometry of Concepts
Jul 27, 2026Elad David, Max FominLLM Interpretability In-Context Learning
Zenity
How Context Attribution Handles What the Model Already Knows
Jul 26, 2026Quoc-Huy Trinh, Lin Zhu, Sebastian SzyllerLLM Evaluation LLM Interpretability
Aalto University
Do LLMs Know Their Vulnerable Scenarios?
Jul 26, 2026Ziheng Peng, Huiqi Deng, Haoran Jing +5LLM Interpretability Adversarial Scenario Generation
Xi’an Jiaotong University Shanghai, China · Shanghai Artificial Intelligence Laboratory Shanghai, China
When Activation Oracles Learn Not to Read: Concept-Specific Blind Spots in Fine-Tuned Oracles
Jul 25, 2026Tobias Bersia, Tatiana GaintsevaLLM Interpretability Language Model Auditing
BAISH · Queen Mary University of London
Not All LLM Reasoning is Visible in the Chain-of-Thought
Jul 24, 2026Vatsal Baherwani, Tom Goldstein, Ashwinee PandaCoT Faithfulness LLM Interpretability
New York University · University of Maryland · TogetherAI
When Language Models Meet NeuroGraphs: Exploring Enhanced Agentic LLM Framework Towards Brain Network Analysis
Jul 24, 2026Jiaxing Li, Rui Dong, Muyao Tang +1LLM Interpretability LLM Agents
School of Computer Science and Engineering, Southeast University
Probing Latent Colombian Identity Inferences in Qwen2.5-7B with Natural Language Autoencoders
Jul 23, 2026Pablo Santiago Potes Velasco, María del Mar García Matabanchoy, Óscar Julián Pérez Ladino +3LLM Interpretability Cultural Bias in Language Models
Universidad Autónoma de Occidente Cali, Colombia · Universidad Autónoma de Occidente GobLab-UAI Cali, Colombia
Training Large Language Models for Self-Explanation Faithfulness
Jul 23, 2026Yeoktatt Cheah, María Pérez-Ortiz, Noah Y. Siegel +1Faithfulness of Language Model Explanations LLM Interpretability
Centre for AI, Department of Computer Science University College London · Imperial College London
Where Animacy Lives in Large Language Models: Tracing the Circuits of the Animacy Concept
Jul 23, 2026Samuele Punzo, Giovanni Cinà, Sandro PezzelleLLM Interpretability Circuits in Language Models
Graduate School of Informatics, University of Amsterdam · Department of Medical Informatics, Amsterdam University Medical Center · ILLC, University of Amsterdam
Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects
Jul 22, 2026Seonglae Cho, Zekun Wu, Kleyton Da Costa +3LLM Interpretability Sparse Autoencoders
Holistic AI · University College London
Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanations
Jul 22, 2026Hiskias DingetoFaithfulness of Language Model Explanations LLM Interpretability
StackOne Technologies
Gotta Catch them all: the modes of Sycophancy
Jul 22, 2026Shreyans Jain, Alexandra Yost, Amirali AbdullahLLM Sycophancy LLM Interpretability
Thoughtworks · Southern Utah University (SUU)
Understanding the Impact of Linguistic Realization Choices on LLM Stance with Causal Tracing
Jul 22, 2026Langchen Huang, Sebastian Padó, Franziska WeeberPrompt Sensitivity LLM Interpretability
Institute for Natural Language Processing, University of Stuttgart
Causal dictionary learning reveals and validates transcription-factor binding features in genomic language models
Jul 21, 2026Sarwan AliLLM Interpretability Genomics
Columbia University Irving Medical Center, New York, USA
Semantic Primes as Explanans for Emotion in Large Language Models
Jul 21, 2026Frank XingLLM Interpretability Mechanistic Interpretability
University of Reading
Diagnosing Correctness Probes under Self-Judgement Confounding
Jul 18, 2026Yi-Long LuLLM Interpretability Language Model Self-Assessment
Key Laboratory of Adolescent Cyberpsychology and Behavior (CCNU), Ministry of Education, Wuhan 430079, China · Key Laboratory of Human Development and Mental Health of Hubei Province, School of Psychology, Central China Normal University, Wuhan 430079, China
The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models
Jul 18, 2026Francesco Karim VicidominiLLM Interpretability Factual Knowledge in Language Models
Independent researcher
Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs
Jul 18, 2026Sharath Naganna, Tanvir Ahmed Sijan, Uddipta KalitaLLM Interpretability Circuits in Language Models
Indian Institute of Technology Bombay, Mumbai, India · Jahangirnagar University, Dhaka, Bangladesh · National Institute of Technology, Silchar, India

Jul 30, 2026, cs.CLJ/K move · Enter open · S save

Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

Pere Martra, Eugenio Martínez Cámara, Alfonso Ureña López

Universidad Internacional Men´endez Pelayo Barcelona, Spain · Departamento de Inform´atica Universidad de Ja´en Ja´en, Spain

LLM Pruning LLM Interpretability Demographic Bias in Language Models Causal Interventions in Language Models

Open paper AskPDF arXiv

This work presents Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias in large language models (LLMs). As a foundational empirical validation of this method, this work focuses on causal bias localization. Using minimally contrastive prompt pairs and inference-time activation capture, the method identifies neurons that react differentially when processing demographic attributes in GLU architectures, evaluating the signal at the down_proj input. Empirical evaluation was conducted on models of up to 3 billion parameters (Llama-3.2 family and Salamandra-2B), combining standardized benchmark evaluation with qualitative text generation experiments. Results demonstrate that zeroing the identified neurons alters how the model responds to associated demographic variables. However, rather than producing flat mitigation, the intervention causes bidirectional bias destabilization: because BiasScore is unsigned, candidate sets mix neurons that push toward and against the stereotype, and the net effect on aggregate bias depends on which sign dominates. The intervention is extremely surgical: zeroing at most 40 neurons in Llama-3.2-1B (less than 0.031% of total MLP width) achieves a mean retention of 99.49% in reasoning and general knowledge capabilities. These findings empirically confirm that demographic bias processing and model capabilities operate on dissociable circuits, establishing the methodological foundations for transitioning from blind zeroing toward directional behavior modulation.

Momentum

Related topics

Latest papers 546