Mechanistic Interpretability

Overview

The field seeks to explain how a model’s internal computations produce particular outputs, rather than relying only on input-output behavior. Researchers use sparse autoencoders and transcoders to identify features, and probes, attribution graphs, activation patching, ablation, and steering to trace or test their roles. Studies examine language-model tasks such as factual recall, reasoning, and refusal, as well as vision, audio, and scientific models. A central challenge is that a signal can be easy to decode without being causally used, while circuit explanations and interventions can vary across prompts, models, and analysis choices.

Latest papers

CardsList

427 papers58 papers in the last four weeks, up 23% on the four weeks before. 0.4% of all new papers.

  1. Discovering Interpretable Acoustic Features in Autoregressive Voice-Cloning TTS Models

    Oct 29, 2026Anuj Diwan, David Harwath, Eunsol ChoiMechanistic InterpretabilityTTS Synthesis

  2. Can LLMs Faithfully Solve Ontology Engineering Tasks? A Mechanistic Interpretability Study

    Oct 28, 2026Bohui Zhang, Albert Meroño-Peñuela, Elena SimperlMechanistic InterpretabilityCausal Interventions in Language Models

  3. Hidden Commitment: When Language Models Silently Pick a Side and How Steering Can Surface It

    Oct 28, 2026Samuel Dawit Assefa, Jae Won ChoKnowledge Conflicts in Language ModelsLanguage Model Auditing

  4. Beyond the Readout: A Latent Numerical Frame Shared Across Modality and Operation

    Oct 27, 2026Oriol Juan Sabater, Alex Ferrando de las Morenas, Pau Rodríguez +1Multimodal Model InterpretabilityMultimodal Reasoning

  5. Uncovering Why LLMs Fail: From Internal Mechanisms to Adversarial Reasoning Trajectories

    Oct 27, 2026Zirui Ding, Xinxin Fan, Hao Yuan +4LLM AuditingMechanistic Interpretability

  6. What Survives Distillation? A Mechanistic Interpretability Study of GPT-2 and DistilGPT-2

    Oct 27, 2026Nabeelah Faumi, Deshan SumanathilakaLanguage Model DistillationKnowledge Distillation

  7. Mechanistic Insights into LoRA: Layer Sparsity for Adaptive Fine-Tuning via Path Patching

    Oct 26, 2026Fei Zuo, Yizhou Huang, Kezhi WangLow-Rank AdaptationParameter-Efficient Fine-Tuning

  8. Cross-Loop Query Handoff: How Later Loops Redirect Two-Hop Retrieval in the Ouro Looped Language Models

    Oct 25, 2026Ziyu Mu, Zehua WangMulti-Hop RetrievalMulti-Hop QA

  9. Beyond the Opening: Tracing the First Token's Persistent Influence on Reasoning Length

    Oct 25, 2026Ruochen Liu, Hao Yan, Ziluowen Luo +2Mechanistic Interpretability

  10. MI-MIDI 2: From Observation to Control with Sparse Autoencoders in Text-to-MIDI Generation

    Oct 23, 2026Jakub Poćwiardowski, Mateusz ModrzejewskiMechanistic InterpretabilitySparse Autoencoders

  11. Interpreting Model Reasoning through Behavioral Counterfactuals

    Oct 22, 2026Shawn Im, James Oldfield, Adel Bibi +1Causal Interventions in Language ModelsFaithfulness of Language Model Reasoning

  12. Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books

    Oct 20, 2026Valentin Mohl, Alexandre Bismuth, Aramis Marti-Shahandeh +6Limit Order Book ModelingState Representation Learning

  13. Convergent Performance, Divergent Mechanisms in LoRA-Adapted Vision Foundation Models for Brain MRI

    Oct 20, 2026Cecilia Diana-Albelda, Fiona Kekwick, Raghav Mehta +6Vision Foundation Model AdaptationLow-Rank Adaptation

  14. Uncovering Audio-Text Alignment in Large Audio-Language Models through Cross-Modal Concept Neurons

    Oct 20, 2026Tzu-Quan Lin, Chih-Kai Yang, Yi-Cheng Lin +2Audio-Text AlignmentAudio-Language Models

  15. Shaped SAEs: Expanding the Toolkit for Mechanistic Interpretability

    Oct 19, 2026Oveys Delafrooz Noroozi, Bhagyashree Puranik, Upamanyu MadhowLLM InterpretabilitySparse Autoencoders

  16. Inside the Scorer: Mechanistic Evidence on Rubric Control in LLM-Based Automated Essay Scoring from 25 Base Models

    Oct 19, 2026Chungche Huang, Po-Ya Angela Wang, Shu-Kai HsiehAutomated Essay ScoringRubric-Based Evaluation

  17. Text Dominance in Paralinguistic Conflict: Tracing Suppression Through Audio LLM Circuits

    Oct 19, 2026Ankur Banerjee, Vishnu Guntakala, Danity Pike +3Audio-Language ModelsAudio-Language Model Evaluation

  18. Cognitive Maps in Large Language Models

    Oct 19, 2026Albert Qin, Jonathan D. Cohen, Gautam ReddyCognitive ModelingMechanistic Interpretability

  19. Transformers Learn Two Competing Sorting Algorithms, and Only One Length-Generalizes

    Oct 18, 2026Khashayar Gatmiry, Nathan W. Henry, Chloe H. Su +5Length GeneralizationTransformer Training Dynamics

  20. Safety Kernel: Mechanistic Alignment for Cross-Lingual Defense in LLMs

    Oct 17, 2026Chengyu Xie, Fei Shen, Shuyi Miao +5LLM SafetyLanguage Model Robustness

  21. Beyond SAEs: Consistent Concept Discovery with Sparse Coding

    Oct 16, 2026Jinho Choi, Hyesu Lim, Jaegul Choo +1Sparse AutoencodersSparse Coding

  22. Verbalising Model's Computation for Chain-of-Thought Faithfulness Detection

    Oct 16, 2026Yujing Wang, Yulan He, Wenya Wang +1CoT FaithfulnessFaithfulness of Language Model Reasoning

  23. Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics

    Oct 15, 2026Jian Gu, Hongyu Zhang, Chunyang Chen +1LLM InterpretabilityMechanistic Interpretability

  24. Human Interventions as Labels: Localizing, Steering, and (Sometimes) Editing the Computations Behind LLM Agent Failures

    Oct 15, 2026Cesare Zavattari, Alessandro Tommasi, Giuseppe PrencipeLLM AgentsFailure Attribution for LLM Agents

  25. Translation Circuit Discovery and Targeted Optimization for Low-Resource Multilingual Translation

    Oct 15, 2026Atai Waris Khan, Abdullah Mazhar, Raghu Kumar +1Multilingual Language ModelsLow-Resource MT