Circuits in Language Models

Overview

Circuits in language models are groups of attention heads, MLP neurons, and learned features whose interactions support behaviors such as induction, factual recall, arithmetic, refusal, and multilingual routing. Researchers locate them with activation patching and ablation, attribution graphs, and sparse autoencoders or transcoders, then test their causal roles on controlled tasks such as indirect object identification and greater-than. Some computations appear compact and reusable, while others are distributed, context-dependent, or reorganized by architecture and post-training. A central challenge is showing that a discovered circuit is causally faithful and task-specific rather than an artifact of prompts, analysis choices, or intervention side effects.

Latest papers

CardsList

130 papers14 papers in the last four weeks, down 52% on the four weeks before. 0.1% of all new papers.

  1. Localization Does Not Imply Steerability: A Mechanistic Study of Sign Errors in LLM Mathematics

    Nov 5, 2026Shradha Agarwal, Tariq J., Deepak RajbharLLM InterpretabilityCircuits in Language Models

  2. When Every Choice Is Correct: A Causal Study of Numerical Preference in Language Models

    Nov 4, 2026Hongnian Huang, Hongliang LiuMechanistic InterpretabilityCircuits in Language Models

  3. A Circuit for In-Context Learning

    Nov 3, 2026Andrew Gao, William GaoIn-Context LearningCircuits in Language Models

  4. Bridging Singular Learning Theory and Mechanistic Interpretability at the Billion-Parameter Scale

    Oct 31, 2026Ethan Ji, Wei Wang, Yizhou SunMechanistic InterpretabilitySingular Learning Theory

  5. Circuit Oracle: Automating Attribution Graph Analysis via Natural-Language Queries

    Oct 30, 2026Shariar Kabir, Hong Kiat Tan, Swastik Agrawal +2LLM InterpretabilityMechanistic Interpretability

  6. Mechanistic Failure Modes of Weight Pruning in Transformer Language Models

    Oct 27, 2026Nabeelah Faumi, Deshan SumanathilakaLLM PruningAttention Head Pruning

  7. The Emergence and Sparsification of a Syntactic Circuit

    Oct 23, 2026Qing Yao, Sasha Boguraev, Tiago Pimentel +1Circuits in Language ModelsSyntactic Generalization in Language Models

  8. Interpretable sparse fine-tuning with TopKLoRA enables faithful circuit discovery

    Oct 23, 2026Marek Masiak, Andrzej Szablewski, Nicola Cancedda +1Sparse Fine-TuningLLM Interpretability

  9. Concentrated Causal Influence, Distributed Diagnostic Information: Cognitive Impairment Decisions in Large Audio-Language Models

    Oct 23, 2026Xiutian Zhao, Luqi Sun, Björn Schuller +1Audio-Language ModelsCircuits in Language Models

  10. How Depth Growth and Looping Shape In-Context Learning in Language Models

    Oct 22, 2026Ferdinand Kapl, Kaitlin Maile, Stefan Bauer +1In-Context LearningParameter Sharing

  11. Language Impairment as a Lens onto Large Language Models

    Oct 20, 2026Muxuan He, Wang Bill Zhu, Yue Chen +5LLM InterpretabilityCircuits in Language Models

  12. Are Linguistic and Arithmetic Circuits Dissociated in LLMs?

    Oct 20, 2026Riku Kisako, Tatsuki Kuribayashi, Ryohei Sasanomodular language model architecturesCircuits in Language Models

  13. Many Biases, One Circuit: Post-Training Tunes a Pretrained Cue-Routing Mechanism

    Oct 16, 2026Prakhar Gupta, Terry Jingchen Zhang, Zhijing JinCognitive Bias in Language ModelsCircuits in Language Models

  14. Translation Circuit Discovery and Targeted Optimization for Low-Resource Multilingual Translation

    Oct 15, 2026Atai Waris Khan, Abdullah Mazhar, Raghu Kumar +1Multilingual Language ModelsLow-Resource MT

  15. Do Emergent Misalignment Models Share a Circuit?

    Oct 14, 2026Muhammad Haseeb, Muhammad TahirEmergent Misalignment in Language ModelsCircuits in Language Models

  16. Interpreting Language Model Parameters

    Oct 14, 2026Lucius Bushnaq, Dan Braun, Oliver Clive-Griffin +5Mechanistic InterpretabilityTransformer Interpretability

  17. Causal Syntax Circuits in CodeLMs: Budget and Probe Decide What Looks Localised

    Oct 14, 2026Harsh Kumar, Tanmay Shailesh Joshi, Rahul Maity +1Circuits in Language ModelsCode Language Models

  18. Task Context Shapes Component Roles in Language Models

    Oct 9, 2026Sovesh Mohapatra, Dani S BassettLLM InterpretabilityCircuits in Language Models

  19. Language Models Have Interpretable Narrative Mechanisms

    Oct 9, 2026David Yiyang Liu, Aditya Joshi, Paul Dawson +1LLM InterpretabilityCircuits in Language Models

  20. SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models

    Oct 7, 2026Miao Yu, Hao Huang, Lu Yuan +3Refusal Mechanisms in Language ModelsLLM Safety Alignment

  21. Where Do LLMs Retrieve Network Information? Identifying the Responsible Heads and Neurons

    Oct 6, 2026Md Abdul Kadir, Md Mohasin Hossain, Omair Shahzad Bhatti +2Attention Head AnalysisLLM Interpretability

  22. Copying Before Suppression: What Drives a Below-Chance Dip During Language Model Training?

    Oct 2, 2026Tejas Dahiya, Cole BlondinCircuits in Language ModelsLLM Training

  23. Slaying the Hydra: Interaction-Aware Circuit Discovery in Language Models

    Oct 2, 2026Sankaran Vaidyanathan, Rafal Urbaniak, Emily Bunnapradist +2Circuits in Language ModelsMechanistic Interpretability

  24. Component and Dimension Sparsity in Transformer Refusal Mechanisms

    Sep 30, 2026Vincent Siu, Glenn Grant-Richards, Vlad Pavlovich +3Refusal Behavior in Language ModelsTransformer Interpretability