CIRCUS: Circuit Consensus under Uncertainty via Stability Ensembles

Swapnil Parekh

cs.CLFeb 28, 2026

CIRCUS: Circuit Consensus under Uncertainty via Stability Ensembles

Authors: Swapnil Parekh

Organizations: Intuit

Abstract

Every mechanistic circuit carries an invisible asterisk: it reflects not just the model's computation, but the analyst's choice of pruning threshold. Change that choice and the circuit changes, yet current practice treats a single pruned subgraph as ground truth with no way to distinguish robust structure from threshold artifacts. We introduce CIRCUS, which reframes circuit discovery as a problem of uncertainty over explanations. CIRCUS prunes one attribution graph under B configurations, assigns each edge an empirical inclusion frequency s(e) in [0,1] measuring how robustly it survives across the configuration family, and extracts a consensus circuit of edges present in every view. This yields a principled core/contingent/noise decomposition (analogous to posterior model-inclusion indicators in Bayesian variable selection) that separates robust structure from threshold-sensitive artifacts, with negligible overhead. On Gemma-2-2B and Llama-3.2-1B, consensus circuits are 40x smaller than the union of all configurations while retaining comparable influence-flow explanatory power, consistently outperform influence-ranked and random baselines, and are confirmed causally relevant by activation patching.

Figures & tables

Figure 1: CIRCUS pipeline. A single attribution graph is pruned under $B$ configurations to yield multiple views; edges receive stability scores $s(e)$ . Strict consensus $C_{\tau=1}$ keeps only edges present in all views (solid lines); dashed edges are contingent alternatives. The core/contingent/noise taxonomy is described in Section 3 .

Model	Configs	IR(C 1 )	W-UP	W-Rand	Match
Gemma	nested $B{=}3$	$.817{\pm}.045$	50/50	37/50	50/50
	non-nest. $B{=}5$	$.822{\pm}.044$	30/50	20/50	3/50
	non-nest. $B{=}9$	$.808{\pm}.051$	47/50	37/50	8/50
Llama	nested $B{=}3$	$.886{\pm}.044$	20/20	20/20	20/20
	non-nest. $B{=}5$	$.883{\pm}.044$	16/20	20/20	0/20
	non-nest. $B{=}9$	$.892{\pm}.052$	19/20	20/20	0/20

Table 1: Wide-scale results: mean IR and win rates across prompts (Gemma-2-2B, 50 prompts; Llama-3.2-1B, 20 prompts). Non-nested configs cross node/edge thresholds. All baselines use

|C_{1}|

edges. “Match” = consensus equals a single config (Match

=B/B

indicates the consensus is trivially the tightest view, a nesting artifact; lower Match values confirm that consensus is a genuinely new circuit).

Figure 2: Left: Per-prompt KL divergence (log scale); consensus (blue) clusters near

10^{-3}

while union-pruned (orange) collapses on outliers. Right: Activation patching recovery; consensus at

57\%

of oracle (

p{=}0.0004

) vs.

20\%

for random.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Circuit	Edges	Nodes	IR
$C_{1}$ (core)	625	61	0.78
$C_{2}$ (residual)	35,725	1,183	—
$C_{1}\cup C_{2}$ (full)	36,350	1,184	0.96
$C_{1}\cup C_{2}$ compact	9,546	942	0.91
Post-pruned (95%)	18,690	1,045	0.94

Appendix

Table 2: Tiered boosting: core, residual, full, and variants (

B{=}3

Figure 3: Stability vs. influence. Mean influence per edge (log scale) rises monotonically with stability score; edges surviving all 25 configurations carry ${\sim}70\times$ higher mean influence than single-configuration edges, within the chosen $5{\times}5$ grid family.

Figure 4: Stability–coverage tradeoff (

B{=}25

configs). As the stability threshold

\tau

increases, circuit size shrinks (left axis) and IR decreases gracefully (right axis). Error bars at

\tau{=}1

are bootstrap 95% CIs. This curve is specific to the chosen configuration family; the tradeoff shifts with different families.

Figure 5: Leave-one-out consistency of

s(e)

(

B{=}3

). Top panel: histogram of edge counts per stability bin. Bottom panel: stability score

s(e)

vs. observed retention fraction in the held-out configuration; all three points lie on the diagonal, showing that

s(e)

is a consistent empirical frequency within this configuration family.

Figure 6: Bootstrap distribution of strict-consensus size and IR (

B{=}25

). Histograms show 100 bootstrap samples (config resampling with replacement) for

|C_{\tau=1}|

(left) and

\mathrm{IR}(C_{\tau=1})

(right), with 95% CIs. The tight distributions confirm that the consensus circuit is robust to which specific

B

configs are sampled.

Analysis	Summary
Rejection	2.5% stability $=1$ (625 edges); 73% stability $<0.5$ .
Alternatives	6.2k contingent edges carrying 45% of total influence.
Approx. intervention	Spearman 0.29 (stability, ablation effect).
Ablation ( $n{=}20$ )	16/20 vs. matched; diff 6.0 [2.7, 9.2], $p{=}0.012$ .
Patching ( $n{=}20$ )	17/20 vs. matched; diff 17.8 [10.8, 23.1], $p{=}0.0026$ .
Prompt-level	Spearman(IR, prob) $=0.69$ [0.28, 0.89].

Appendix

Table 3: Full uncertainty analyses summary (

B{=}3

n{=}20

prompts).

Circuit Discovery

Circuit discovery identifies compact groups of neural-network components that causally implement specific computations or behaviors.

See all 39 papers

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits
May 8, 2026Michael Li, Nishant SubramaniCircuit Discovery Mechanistic Interpretability
Northeastern University · Carnegie Mellon University – Language Technologies Institute
Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit
Aug 27, 2026Sai Adith Senthil KumarMechanistic Interpretability Circuits in Language Models
George Mason University
Circuit Claims Depend on What Is Extracted and How It Is Compared
Jul 21, 2026Yang Sheng, Jie FuTransformer Interpretability Circuit Discovery
Fudan University · Shanghai Innovation Institute · IQuest Research

May 8, 2026J/K move · Enter open · S save

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

Michael Li, Nishant Subramani

Northeastern University · Carnegie Mellon University – Language Technologies Institute

Circuit Discovery Mechanistic Interpretability Circuits in Language Models Causal Interventions in Language Models

Open paper AskPDF arXiv

The circuits framework in mechanistic interpretability aims to identify sparse subgraphs of model components that are causally responsible for a behavior, typically evaluated by measuring necessity and sufficiency. But these criteria say little about whether a circuit consistently captures how a model performs a task, or if it is specific to that task. We study these two properties, consistency and specificity, across six tasks and five models, extracting circuits at the component level (attention heads and MLP blocks) and at the level of individual MLP neurons. We find that component-level circuits are highly consistent and causally important for most tasks, but they are not specific: for a given task, ablating its own circuit drops accuracy by about as much as ablating another task's circuit. Neuron-level circuits, on the other hand, exhibit higher specificity across tasks, but far less consistency within tasks. This is explained by circuit overlap: component-level circuits share most of their components across tasks, related or not, while neuron-level circuits overlap only between closely related tasks. In a case study of the components shared by the task circuits of Llama-3.2-3B, we show that they consist mostly of MLP blocks, while the few attention heads within turn out to have general-purpose roles. Overall, our findings raise questions about the degree to which circuits can support targeted understanding of, and intervention on, model behavior.

Abstract

Circuit Discovery

Explore similar work

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit

Circuit Claims Depend on What Is Extracted and How It Is Compared