Circuit Discovery

Follow this topic Ask about this topic

Circuit discovery studies how groups of neurons, attention heads, features, and connections work together to produce a model’s behavior, helping researchers explain and intervene on its computations. Common approaches use activation patching and ablation, attribution graphs, sparse autoencoders or transcoders, and optimization or search to find candidate circuits. Researchers test these methods on language-model tasks such as induction and factual recall, as well as vision, speech, and code, measuring causal effects, behavioral fidelity, and circuit size. A central challenge is that results can depend on prompts, data, and analysis choices, and structurally different circuits may implement the same behavior, so reliable claims require careful stability and causal validation.

Latest papers

39 papers3 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits
May 8, 2026Michael Li, Nishant SubramaniCircuit Discovery Mechanistic Interpretability
Northeastern University · Carnegie Mellon University – Language Technologies Institute
Can Current Agents Close the Discovery-to-Application Gap? A Case Study in Minecraft
Apr 27, 2026Zhou Ziheng, Huacong Tang, Jinyuan Zhang +9AI Agents for Scientific Discovery Circuit Discovery
University of California, Los Angeles · Peking University · Amazon
Prune, Interpret, Evaluate: A Cross-Layer Transcoder-Native Framework for Efficient Circuit Discovery via Feature Attribution
Apr 18, 2026Qinhao Chen, Linyang He, Nima MesgaraniLLM Pruning Feature Attribution
Columbia University
Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformers
Apr 15, 2026Nina Żukowska, Wolfgang Stammer, Bernt Schiele +1Transformer Interpretability Mechanistic Interpretability
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany
CIRCUS: Circuit Consensus under Uncertainty via Stability Ensembles
Feb 28, 2026Swapnil ParekhCircuit Discovery Circuits in Language Models
Intuit
Certified Circuits: Stability Guarantees for Mechanistic Circuits
Feb 26, 2026Alaa Anani, Tobias Lorenz, Bernt Schiele +2Circuit Discovery Mechanistic Interpretability
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany
IBCircuit: Towards Holistic Circuit Discovery with Information Bottleneck
Feb 26, 2026Tian Bian, Yifan Niu, Chaohao Yuan +7Circuit Discovery Circuits in Language Models
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Hong Kong, China · DAMO Academy, Alibaba Group, Hangzhou, China · Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China +4
Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guarantees
Feb 18, 2026Itamar Hadad, Guy Katz, Shahaf BassanCircuit Discovery Mechanistic Interpretability
School of Computer Science and Engineering, The Hebrew University of Jerusalem
Beyond Transfer Accuracy: Mechanism-Guided Controlled Adaptation for Low-Resource Languages
Jan 13, 2026Khumaisa Nur'aini, Ayu Purwarianti, Alham Fikri Aji +1Selective Fine-Tuning Circuit Discovery
Monash University Indonesia · MBZUAI · Institute Teknologi Bandung +1

May 8, 2026J/K move · Enter open · S save

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

Michael Li, Nishant Subramani

Northeastern University · Carnegie Mellon University – Language Technologies Institute

Circuit Discovery Mechanistic Interpretability Circuits in Language Models Causal Interventions in Language Models

Open paper AskPDF arXiv

The circuits framework in mechanistic interpretability aims to identify sparse subgraphs of model components that are causally responsible for a behavior, typically evaluated by measuring necessity and sufficiency. But these criteria say little about whether a circuit consistently captures how a model performs a task, or if it is specific to that task. We study these two properties, consistency and specificity, across six tasks and five models, extracting circuits at the component level (attention heads and MLP blocks) and at the level of individual MLP neurons. We find that component-level circuits are highly consistent and causally important for most tasks, but they are not specific: for a given task, ablating its own circuit drops accuracy by about as much as ablating another task's circuit. Neuron-level circuits, on the other hand, exhibit higher specificity across tasks, but far less consistency within tasks. This is explained by circuit overlap: component-level circuits share most of their components across tasks, related or not, while neuron-level circuits overlap only between closely related tasks. In a case study of the components shared by the task circuits of Llama-3.2-3B, we show that they consist mostly of MLP blocks, while the few attention heads within turn out to have general-purpose roles. Overall, our findings raise questions about the degree to which circuits can support targeted understanding of, and intervention on, model behavior.