Circuit Discovery

Follow this topic Ask about this topic

Circuit discovery studies how groups of neurons, attention heads, features, and connections work together to produce a model’s behavior, helping researchers explain and intervene on its computations. Common approaches use activation patching and ablation, attribution graphs, sparse autoencoders or transcoders, and optimization or search to find candidate circuits. Researchers test these methods on language-model tasks such as induction and factual recall, as well as vision, speech, and code, measuring causal effects, behavioral fidelity, and circuit size. A central challenge is that results can depend on prompts, data, and analysis choices, and structurally different circuits may implement the same behavior, so reliable claims require careful stability and causal validation.

Latest papers

39 papers3 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

Causal Syntax Circuits in CodeLMs: Budget and Probe Decide What Looks Localised
Oct 14, 2026Harsh Kumar, Tanmay Shailesh Joshi, Rahul Maity +1Circuits in Language Models Code Language Models
National Institute of Technology Agartala
CircuitTree: Composing Circuits into A Syllable Transcriber
Oct 7, 2026Jiankun Wei, Ewan Dunbar, Gerald PennCircuit Discovery Automatic Speech Recognition
123Univerity of Toronto
Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability
Oct 1, 2026Chuqin Geng, Li Zhang, Haolin Ye +3Circuit Discovery Mechanistic Interpretability
University of Toronto · McGill University
Attention as a Routing Graph: Live Circuit Extraction from a Single Forward Pass
Sep 21, 2026Ash Manvi, Samreena TajreenCircuit Discovery Attention Mechanisms
Aquin Labs
Circuit-Diff: Factual Edit-based Intervention Method for Localizing Knowledge in Attribution Graphs
Sep 20, 2026Edward G. Friedman, Xiangchen SongCircuit Discovery Mechanistic Interpretability
School of Computer Science Machine Learning Department Carnegie Mellon University Pittsburgh, PA 15213
K-TRAIL: Simulator-Guided Generative Design of EM/RF Circuits
Sep 19, 2026Piyush Saha, Evan Newell, Hanna O'Leary +2Inverse Design Electronic Design Automation
School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR 97331 USA · Department of Electrical and Computer Engineering, Yale University, New Haven, CT 06511 USA
The Refutation Gap: Certifying Both Halves of an Optimality Claim
Sep 16, 2026Rohan PandeyFormal Verification Program Synthesis
Sparse Weight and Edge Circuit Discovery in Transformer-based Acoustic Models
Sep 9, 2026Jiankun Wei, Ewan Dunbar, Gerald PennTransformer Interpretability Circuit Discovery
Department of Computer Science, University of Toronto, Toronto, Canada
Analog-DB: An Agent-First Analog Integrated Circuit Database, From Blocks to Systems
Sep 1, 2026Danial Noori Zadeh, Mohamed B. ElamienCircuit Discovery Electronic Design Automation
The authors are with the Department of Electrical and Computer Engineering, McMaster University, Hamilton, ON, Canada
Explanation Multiplicity: Circuit-Level Interpretability Evidence Does Not Survive Defensible Analytic Variation
Aug 13, 2026Ajay Pravin MahaleMechanistic Interpretability Explainability Evaluation
Hochschule Trier
What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representations
Jul 23, 2026Piotr WilamCircuit Discovery Language Model Introspection
University College London
CircuitKIT : Circuit Discovery, Evaluation, and Application Toolkit for Mechanistic Interpretability
Jul 21, 2026Pratinav Seth, Hem Gosalia, Aditya Kasliwal +1Circuit Discovery Mechanistic Interpretability
Lexsi Labs
Circuit Claims Depend on What Is Extracted and How It Is Compared
Jul 21, 2026Yang Sheng, Jie FuTransformer Interpretability Circuit Discovery
Fudan University · Shanghai Innovation Institute · IQuest Research
A Shared Subcircuit Lets LLMs Count Down Across Tasks
Jul 14, 2026Jacob Dunefsky, Wes Gurnee, Emmanuel AmeisenCircuit Discovery Circuits in Language Models
Yale University · Anthropic
Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits
Jul 2, 2026Zhiren Gong, He Lu, Tiantong Wang +6Circuit Discovery Mechanistic Interpretability
College of Computing and Data Science, Nanyang Technological University, Singapore · Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore
FactorLibrary: From Polynomials to Circuits via Recursive Subgoals
Jun 24, 2026Rohan Pandey, Michael Ruofan Zeng, Weikun K. Zhang +5Circuit Discovery Reinforcement Learning
University of Washington, Seattle, WA, USA.
Targeted Recovery of Weight-Space Mechanisms From Neural Networks
Jun 19, 2026Antoine Vigouroux, Lee SharkeyTransformer Interpretability Circuit Discovery
1MATS · 2Goodfire
Scalable Circuit Learning for Interpreting Large Language Models
Jun 15, 2026Naiyu Yin, Dennis Wei, Tian Gao +3Circuit Discovery LLM Interpretability
Lehigh University, Mathematics Department, Bethlehem, USA · IBM Research, Yorktown Heights, USA
Demystifying Variance in Circuit Discovery of LLMs
Jun 15, 2026Frank Zhengqing Wu, Francesco Tonin, Volkan CevherPrompt Sensitivity Circuit Discovery
Laboratory for Information and Inference Systems (LIONS) École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland · Laboratory for Informationand Inference Systems (LIONS) École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
Circuit Tracing in Autoregressive Protein Language Models
Jun 14, 2026Darin Tsui, William Deinzer, Daniel Saeedi +1Circuit Discovery Protein Fitness Prediction
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atltanta, GA.
Closure-Validated Circuit Discovery in Attention Heads: Co-activation Proposes, Ablation Disposes
Jun 8, 2026Yongzhong XuTransformer Interpretability Circuit Discovery
Many Circuits, One Mechanism: Input Variation and Evaluation Granularity in Circuit Discovery
Jun 4, 2026Alireza Bayat Makou, Jingcheng Niu, Subhabrata Dutta +1Circuit Discovery Mechanistic Interpretability
UKP Lab, Technical University of Darmstadt · National Research Center for Applied Cybersecurity ATHENE
MechRL: Reinforcement Learning Agents Perform Circuit Discovery for Mechanistic Interpretability
May 25, 2026Barsat KhadkaTransformer Interpretability Circuit Discovery
School of Computing Sciences and Engineering The University of Southern Mississippi
CSP-Atlas: Concept-Specific Neural Circuits in a Sparse Python Transformer
May 23, 2026Piotr WilamTransformer Interpretability Circuit Discovery
University College London
Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers
May 22, 2026Yongzhong XuTransformer Interpretability Circuit Discovery
From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach
May 20, 2026Nura Aljaafari, Danilo S. Carvalho, Andre FreitasCircuit Discovery Mechanistic Interpretability
Department of Computer Science, University of Manchester, United Kingdom · CRUK National Biomarker Centre, University of Manchester, United Kingdom · Idiap Research Institute, Switzerland
Transformers Linearly Represent Highly Structured World Models
May 13, 2026Roman Kniazev, Nathanaël FijalkowCircuit Discovery World Models
LaBRI, CNRS · University of Bordeaux · France
All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs
May 12, 2026Xi Chen, Mingyu Jin, Jingcheng Niu +7Circuit Discovery LLM Interpretability
University of Toronto · The Hong Kong University of Science and Technology (Guangzhou) · Rutgers University +2
Data-driven Circuit Discovery for Interpretability of Language Models
May 9, 2026Daking Rai, Mor Geva, Ziyu YaoCircuit Discovery LLM Interpretability
George Mason University · Tel Aviv University
Architecture, Not Scale: Circuit Localization in Large Language Models
May 9, 2026Sohan VenkateshTransformer Interpretability Grouped-Query Attention
Manipal Institute of Technology Bengaluru.

Oct 14, 2026J/K move · Enter open · S save

Causal Syntax Circuits in CodeLMs: Budget and Probe Decide What Looks Localised

Harsh Kumar, Tanmay Shailesh Joshi, Rahul Maity, Anusa Saha

National Institute of Technology Agartala

Circuits in Language Models Code Language Models Circuit Discovery Causal Interventions in Language Models

Open paper AskPDF arXiv

Code models write Python when the prompt does not say which language to use. We ask a mechanistic question about that habit: is it produced by a small, findable group of components inside the model? We run one pipeline---mean ablation, activation patching, greedy circuit extraction, a random-control significance test, and activation steering---over 55 model-language settings spanning ten open code models (five base and five instruction-tuned), three target languages (Rust, C#, PHP), two prompt suites, and two component budgets. We report five findings. First, the preference is large, grows with model size, and survives instruction tuning: it is larger on every instruction-tuned checkpoint than on its base version, by 12% on average. Second, its measured size is mostly a property of the probe; replacing our aligned prompts with XLCoST parallel functions takes the same measurement on the same models from about 1.6 to about 0.01. Third, whether the preference looks localised depends on how many components the search may keep: with a budget of 12, 1 of 15 settings yields a circuit that beats size-matched random controls after multiple-comparison correction, and with a budget of 40, 6 of 15 do. Fourth, interventions that a smaller-budget study reports as uniformly failing do sometimes succeed: patching makes a Python prompt prefer target syntax in 2 of 45 settings, and steering changes the most likely next token in 9 of 45. Fifth, every circuit whose effect is significant but points away from the target language comes from a setting where circuit extraction failed, which makes such results a diagnostic rather than a phenomenon. The practical lesson is that a claim about whether a behaviour is localised can turn on analysis choices that papers rarely report. We release the full grid so those choices can be varied.