Mechanistic Circuit Identification for Controllable Data Generation

Nakyung Lee; Sangwoo Hong; Jungwoo Lee

cs.LGAug 25, 2026

Mechanistic Circuit Identification for Controllable Data Generation

Authors: Nakyung Lee, Sangwoo Hong, Jungwoo Lee

Organizations: Department of Electrical and Computer Engineering, Seoul National University · Department of Computer Science and Engineering, Konkuk University

Abstract

While recent advances in data synthesis aim to curate high-quality datasets, most generation pipelines still rely on heuristic prompt-based control. This black-box paradigm provides limited insight into how individual samples interact with a model's underlying learning dynamics. To bridge this gap, we propose a circuit-grounded framework that connects training-dynamics-based data valuation with mechanistic interpretability (MI). Specifically, we conceptualize data quality along three complementary utility axes, learnability, challenge, and alignment. First, we uncover specialized model-internal circuits that causally govern these utility signals. Then, moving beyond heuristic prompting toward mechanistic control, we leverage these circuits as controllable interfaces, actively steering generation to produce utility-targeted data. Building on this capability, we introduce SAMS (Stage-Aware Mechanistic Scheduling), which schedules circuit-steered data according to the model's evolving optimization needs. Experiments on multiple-choice QA tasks demonstrate that our approach yields precisely controlled data with greater diversity than prompt-based baselines, consistently improving downstream performance and calibration. Ultimately, this work establishes a principled white-box paradigm for interpretable data generation, pioneering the use of MI not just as an analytical tool, but as a practical, controllable interface.

Figures & tables

Figure 1: Overview of the proposed framework. We first discover and causally validate internal circuits tied to specific training-utility metrics (Section 3 ). We then leverage these circuits to steer the generation process, curating distinct data pools with targeted optimization profiles (Section 4 ). Finally, our SAMS framework dynamically schedules these utility-targeted datasets according to the model’s evolving learning phases during downstream fine-tuning (Section 5 ).

Figure 2: Absolute Circuit Performance Ratio (Abs-CPR) of discovered circuits versus random baselines. The x-axis denotes the number of extracted top- $K$ edges, defining the subgraph size. Higher Abs-CPR indicates that the extracted circuit successfully governs the model’s original behavior.

Metric	Circuit	$S_{\mathrm{drop}}$	Corr.	GapRed $\uparrow$
AUM	High	7.042	-0.007	99.98%
	Low	7.347	-0.173	98.05%
	Random	1.171	0.940	26.58%
EL2N	High	0.637	0.024	99.53%
	Low	0.636	0.003	99.58%
	Random	0.080	0.898	15.95%

Table 1: Causal intervention of top-250 metric-correlated circuits. Corr. denotes the Pearson correlation between

q_{\mathrm{full}}

and

q_{\mathrm{abl}}

, and GapRed measures the collapse of the original high/low score separation after ablation.

Criteria	AUM	EL2N	GradAlign
HighC	85.8	86.7	86.6
LowC	84.8	87.0	86.4
ContC	86.3	85.8	87.7
Rand	84.0	85.2	85.8
OriS	82.3	85.1	66.2
Full (100%)	87.4	87.6	87.4

Table 2: Downstream SciQ accuracy on 30% filtered data subsets. Rand denotes random filtering, and OriS denotes filtering based on the original scalar utility score.

Figure 3: Shared backbone and functional divergence of discovered utility circuits. (A) Layer-wise normalized incoming mass for each circuit. The black dashed line denotes the cross-circuit mean, and the gray band shows $\pm 1$ standard deviation. Narrow variance indicates shared backbone usage, while broader deviations indicate utility-specific routing. (B) Utility-level enrichment of each circuit, reported as cross-circuit $z$ -scores. Positive values indicate that a circuit allocates relatively more mass to that module than the cross-circuit average, while negative values indicate relative suppression.

Category	Config	S&S	Sch.	Accuracy ( $\uparrow$ )			ECE ( $\downarrow$ )			NLL ( $\downarrow$ )
Category	Config	S&S	Sch.	50%	60%	70%	50%	60%	70%	50%	60%	70%
In-Domain: SciQ
Baseline	Source Only (Full)	✗	✗	83.4			0.157			1.408
Prompt Gen.	Uniform-Mix	✗	✗	82.9	83.7	82.8	0.070	0.078	0.093	0.541	0.551	0.588
Prompt Gen.	Stage-Aware Sch.	✗	✓	83.4	83.1	83.2	0.074	0.079	0.090	0.525	0.568	0.594
	Uniform-Mix	✓	✗	83.9	84.5	84.9	0.060	0.071	0.069	0.513	0.544	0.540

Table 3: Downstream fine-tuning results across source-data ratios. 50%, 60%, and 70% indicate the ratio of original source data to generated data under a fixed total training budget. S&S indicates circuit-based steering and selection. Sch. applies the exact scheduling ratios from Algorithm 1 identically to both prompt-generated and circuit-steered pools. We report Accuracy (

\uparrow

), Expected Calibration Error (ECE,

\downarrow

), and Negative Log-Likelihood (NLL,

\downarrow

Config (uniform)	SciQ	ECE $\downarrow$	NII $\downarrow$
Unsteered + Random	$82.80\pm 0.46$	$0.077\pm 0.057$	$0.487\pm 0.050$
Unsteered + Filter	$82.27\pm 0.97$	$0.057\pm 0.035$	$0.507\pm 0.065$
Steered + Random	$83.10\pm 0.17$	$0.057\pm 0.015$	$0.480\pm 0.020$
Steered + Diversity	$83.23\pm 0.31$	0.050 $\pm$ 0.020	0.477 $\pm$ 0.015
Steered + Filter	84.77 $\pm$ 0.38	$0.063\pm 0.006$	$0.510\pm 0.026$

Table 4: Isolation of steering and selection under uniform scheduling.

Config	SciQ ACC $\uparrow$	SciQ ECE $\downarrow$	SciQ NII $\downarrow$	ARC (OOD) $\uparrow$
Bengio (logit-margin as the difficulty proxy)	$82.63\pm 0.23$	$0.060\pm 0.026$	$0.487\pm 0.051$	$75.38\pm 0.34$
Uniform (33:33:34)	$84.77\pm 0.38$	$0.063\pm 0.006$	$0.510\pm 0.026$	$76.09\pm 0.25$
Linear interpolation (per-batch)	$84.43\pm 0.91$	$0.063\pm 0.015$	$0.497\pm 0.042$	$74.55\pm 0.99$
Reversed SAMS (reverse phase order)	$84.83\pm 0.38$	$0.057\pm 0.025$	0.483 $\pm$ 0.076	$75.56\pm 0.68$
SAMS	85.20 $\pm$ 0.56	0.057 $\pm$ 0.023	$0.490\pm 0.053$	76.15 $\pm$ 0.30

Table 5: Sensitivity to phase boundaries and scheduling strategies over the same pools.

Model	Config	SciQ ACC $\uparrow$	SciQ ECE $\downarrow$	SciQ NII $\downarrow$	ARC (OOD) $\uparrow$
Gemma	Source-only	$84.70\pm 0.00$	$0.125\pm 0.007$	$0.835\pm 0.064$	75.74 $\pm$ 0.27
	Prompt-Uniform	$84.45\pm 0.07$	$0.050\pm 0.014$	$0.460\pm 0.000$	$74.83\pm 0.30$
	Prompt-Sched	$84.35\pm 0.64$	$0.050\pm 0.028$	$0.460\pm 0.000$	$73.92\pm 0.80$
	Circuit-Uniform	$85.10\pm 0.57$	$0.055\pm 0.007$	$0.480\pm 0.028$	$75.55\pm 0.24$
	SAMS	85.75 $\pm$ 0.21	0.045 $\pm$ 0.007	0.425 $\pm$ 0.007	$75.21\pm 0.65$
Llama	Source-only	$87.90\pm 0.00$	$0.110\pm 0.000$	$0.835\pm 0.035$	79.20 $\pm$ 0.42

Table 6: Cross-family generalization of the generated data.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Selected Edges (Top- $K$ )	Proportion of Total Edges (%)
100	0.12%
250	0.30%
500	0.59%
1,000	1.18%
2,000	2.36%
5,000	5.90%

Appendix

Table 7: Sparsity of extracted circuits at varying Top-

K

edge thresholds. The full computational graph for the Qwen2.5-1.5B model (28 layers, 12 attention heads, 2 GQA groups, and

d_{\text{head}}=128

) yields a total of 365 nodes and 84,715 potential valid edges.

Ablated Circuit	$\mathrm{mean\_abs}\Delta$	$E_{\Delta}(D_{H})$	$E_{\Delta}(D_{L})$	Pearson(full, abl)
AUM-High	$7.042$	$10.684$	$3.399$	$-0.007$
AUM-Low	$7.347$	$11.065$	$3.628$	$-0.173$
AUM-Random	$1.171$	$1.240$	$1.102$	$0.940$
EL2N-High	$0.637$	$0.404$	$0.870$	$0.024$
EL2N-Low	$0.636$	$0.404$	$0.869$	$0.003$
EL2N-Random	$0.080$	$0.157$	$0.003$	$0.898$

Appendix

Table 8: Effect of circuit ablation on utility scores.

\mathrm{mean\_abs}\Delta

is the mean absolute change in sample scores after ablation.

E_{\Delta}(D_{H})

and

E_{\Delta}(D_{L})

report this quantity within the high- and low-score buckets, respectively. Pearson(full, abl) is the correlation between scores before and after ablation.

	AUM-hi	AUM-lo	EL2N-hi	EL2N-lo	GA-hi	GA-lo
AUM-hi	1.000	0.355	0.374	0.553	0.319	0.458
AUM-lo	0.355	1.000	0.323	0.370	0.282	0.441
EL2N-hi	0.374	0.323	1.000	0.374	0.359	0.362
EL2N-lo	0.553	0.370	0.374	1.000	0.292	0.393
GA-hi	0.319	0.282	0.359	0.292	1.000	0.377
GA-lo	0.458	0.441	0.362	0.393	0.377	1.000

Appendix

Table 9: Jaccard overlap between the top-250-edge circuits.

Circuit $\backslash$ Utility Bucket	AUM	EL2N	GradAlign
AUM-high	$0.648$ ( $d=+0.46$ )	$0.354$ ( $d=-0.44$ )	$0.457$ ( $d=-0.13$ )
EL2N-high	$0.327$ ( $d=-0.51$ )	$0.673$ ( $d=+0.52$ )	$0.551$ ( $d=+0.15$ )
GA-high	$0.384$ ( $d=-0.32$ )	$0.606$ ( $d=+0.24$ )	$0.569$ ( $d=+0.17$ )

Appendix

Table 10: Metric-conditioned functional selectivity of the discovered circuits. Each cell reports ROC-AUC and Cohen’s

d

Circuit	Key Pathways
AUM-high	input $\rightarrow$ a0.h6--11 $\langle$ v $\rangle$ ; m1,m2 ; m1,m2 $\rightarrow$ a6.h6--11 $\langle$ k $\rangle$ ; a14 ; late L12/13/17/18/19; m23 $\rightarrow$ logits
AUM-low	m1 $\rightarrow$ a2 $\langle$ q $\rangle$ ; m1,m2 $\rightarrow$ a5 $\langle$ k $\rangle$ ; m1,m2 $\rightarrow$ m3 (suppr.); m27 $\rightarrow$ logits (suppr.)
EL2N-low	input $\rightarrow$ a0.h6--11 $\langle$ v $\rangle$ ; m1 $\rightarrow$ m2 $\rightarrow$ m3 ; m1,m2 $\rightarrow$ a6.h6--11 $\langle$ k $\rangle$ ; m23 $\rightarrow$ logits
EL2N-high	m1 $\rightarrow$ a2 $\langle$ k $\rangle$ (suppr.); m1,m2 $\rightarrow$ a5 $\langle$ k $\rangle$ ; m4,m6,m13 ; m2 $\rightarrow$ a6,a14 (suppr.)
GradAlign-high	m1,m2 $\rightarrow$ m13 ; m1,m2 $\rightarrow$ a14 $\langle$ k $\rangle$ (suppr.); m1,m2 $\rightarrow$ a4 $\langle$ k $\rangle$ (suppr.); L13–L20
GradAlign-low	input $\rightarrow$ a0.h6--11 $\langle$ v $\rangle$ ; m1,m2 ; m1,m2 $\rightarrow$ m3 (suppr.); m23 $\rightarrow$ logits ; m27 $\rightarrow$ logits (suppr.)

Appendix

Table 11: Representative location of the six discovered training-utility circuits. “Key Pathways” lists the dominant or most discriminative nodes and edges. “suppr.” denotes edges or nodes with suppressive(negative) attribution.

Figure 7: Attention-interface decomposition of SciQ utility circuits. Left: Q/K/V share of attention-interface attribution mass across the six discovered circuits. Right: signed attention-layer profile, computed by aggregating attention-node attribution scores within each layer.

Figure 8: Comparison of mean difficulty proxy between raw and selected datasets.

Hyperparameter	Value
Base model	Qwen2.5-1.5B-Instruct
Seed examples	5,000 SciQ training examples
Candidates per source example	4
Selected candidates per source example	2
Sampling temperature	0.8
Top- $p$	0.95

Appendix

Table 12: Common generation settings used across all utility profiles for both circuit-steered and prompt-based baseline generations.

Setting	Prompt Template
Shared system instruction	Generate science multiple-choice training data. Keep the sample factually correct, avoid trivia leakage, and return strict JSON. Each choice must be plain text without letter or number prefixes. The answer field must be exactly one uppercase letter among A , B , C , and D .
Circuit-steered fixed prompt	Source topic: {topic} Source item: {source question} Source answer: {optional source answer} Create one new science MCQA sample in a similar domain. Keep it self-contained, factually correct, and naturally written. Do not explicitly mention a target difficulty band in the output. Return only JSON with fields question , choices , answer , rationale , difficulty , and helpful .
Prompt baseline: learnable-style	Create one new MCQA sample that is easier than the source. Use a single canonical science fact or one direct concept application. Distractors should be plausible but clearly eliminable by the decisive concept. Return only JSON with the same schema.
Prompt baseline: challenging-style	Create one new MCQA sample that is harder than the source. Require at least one extra inference step, concept disambiguation, or stronger distractor discrimination. Return only JSON with the same schema.
Prompt baseline: aligned-style	Create one new MCQA sample optimized to be instructional and helpful. The rationale must state the decisive evidence and why at least one distractor is wrong. Return only JSON with the same schema.

Appendix

Table 13: Prompt templates for generation. Circuit-steered generation uses the same target-independent fixed prompt for all profiles, whereas prompt-based baselines use explicit profile-specific instructions.

Circuit Preset	Functional Role	Steering Targets
AUM-High	Canonical anchor	$\mathrm{a0.h[6\!-\!11]}\langle V\rangle^{\star}$ ; $m_{1}$ ; $m_{2}$ ; $\mathrm{a6.h[6\!-\!11]}\langle K\rangle^{\star}$ ; $\mathrm{a14.h2}\langle K\rangle^{\star}$ ; $m_{23}$
AUM-Low	Boundary branch	$\mathrm{a2}\langle K\rangle$ ; $m_{3}$ ; $\mathrm{a5}\langle K\rangle$ ; $m_{27}$
EL2N-Low	Low-pressure anchor	$\mathrm{a0.h[6\!-\!11]}\langle V\rangle^{\star}$ ; $m_{1}$ ; $m_{2}$ ; $m_{3}$ ; $\mathrm{a6.h[6\!-\!11]}\langle K\rangle^{\star}$ ; $\mathrm{a7}\langle K\rangle$ ; $m_{23}$
EL2N-High	Informative rerouting	$\mathrm{a2}\langle K\rangle$ ; $\mathrm{a4}\langle K\rangle$ ; $\mathrm{a5}\langle K\rangle$ ; $m_{4}$ ; $m_{6}$ ; $m_{13}$
GradAlign-High	Target-aligned control	$m_{13}$ ; $\mathrm{a14.h2}\langle K\rangle^{\star}$ ; $\mathrm{a17}$ – $\mathrm{a20}$ late-control attention targets
GradAlign-Low	Shortcut evidence	$\mathrm{a0.h[6\!-\!11]}\langle V\rangle^{\star}$ ; $m_{1}$ ; $m_{2}$ ; $m_{3}$ ; $m_{23}$

Appendix

Table 14: Default circuit steering presets used for Qwen2.5-1.5B-Instruct on SciQ. All listed locations are used as activation-addition targets when constructing utility-specific steering directions. Targets marked with

\star

additionally receive targeted attention-logit steering during decoding.

	Activation Scale (Attention Scale)
Profile	AUM-H	AUM-L	EL2N-H	EL2N-L	GradAlign-H	GradAlign-L
$\mathcal{D}_{\text{learnable}}$	$+0.40$ ( $+0.28$ )	$-0.25$ ( $0.00$ )	$-0.35$ ( $-0.14$ )	$+0.47$ ( $+0.31$ )	$+0.10$ ( $+0.02$ )	$+0.12$ ( $+0.12$ )
$\mathcal{D}_{\text{challenging}}$	$-0.37$ ( $-0.18$ )	$0.00$ ( $0.00$ )	$+0.40$ ( $+0.15$ )	$-0.40$ ( $-0.21$ )	$+0.36$ ( $+0.15$ )	$-0.30$ ( $-0.18$ )
$\mathcal{D}_{\text{aligned}}$	$+0.22$ ( $+0.10$ )	$-0.14$ ( $0.00$ )	$-0.10$ ( $-0.06$ )	$+0.20$ ( $+0.10$ )	$+0.45$ ( $+0.17$ )	$-0.28$ ( $-0.15$ )

Appendix

Table 15: Per-profile activation and attention steering scales applied to each circuit preset. Values are presented as Activation Scale (Attention Scale) . Positive values amplify the corresponding circuit’s forward-pass contribution, while negative values suppress it. GradAlign-H inherently targets

\text{a14}\langle K\rangle

with an inverted weight, thus, a positive recipe scale yields suppressive steering on that specific node.

Profile	Raw	Selected	JSON	Q. Len.	Rat. Len.	BLEU-1	Jaccard
Circuit-learnable	20,000	10,000	98.2	9.9	20.0	0.2752	0.1507
Circuit-challenging	20,000	10,000	99.5	10.8	22.6	0.2495	0.1395
Circuit-aligned	20,000	10,000	99.2	10.1	20.5	0.2680	0.1468
Prompt-learnable	20,000	10,000	99.5	9.5	20.7	0.3150	0.1725
Prompt-challenging	20,000	10,000	98.7	12.9	36.6	0.2473	0.1500
Prompt-aligned	20,000	10,000	98.5	10.3	40.6	0.4039	0.2497

Appendix

Table 16: Basic statistics of generated datasets after selection. JSON denotes the percentage of candidates successfully parsed into the required MCQA JSON format. BLEU-1 and Jaccard are computed between the source question and the generated question. Lower values indicate less lexical copying from the seed sample.

Figure 9: Internal proxy alignment across the generated datasets. The distributions of support scores and the difficulty proxy show that each dataset successfully aligns with its targeted circuit profile.

Hyperparameter	Value
Training epochs	6
Per-device batch size	12
Gradient accumulation steps	4
Effective batch size	48
Evaluation batch size	8
Learning rate	$2\times 10^{-5}$

Appendix

Table 17: Fine-tuning hyperparameters for downstream validation experiments. The same configuration is used for all methods, schedules, and source ratios.

Config	SciQ ACC $\uparrow$	SciQ ECE $\downarrow$	SciQ NII $\downarrow$	ARC (OOD) $\uparrow$
Prompt-Uniform	$83.87\pm 0.38$	$0.077\pm 0.006$	$0.543\pm 0.015$	$73.79\pm 0.21$
Prompt + Diversity	$83.30\pm 0.28$	$0.085\pm 0.021$	$0.555\pm 0.064$	$72.16\pm 0.45$
Prompt + Reject-AUM	$84.10\pm 0.79$	$0.053\pm 0.040$	$0.490\pm 0.095$	$76.01\pm 0.99$
Prompt + Reject-EL2N	$82.20\pm 0.99$	$0.080\pm 0.028$	$0.555\pm 0.078$	$64.98\pm 6.85$
Prompt + Reject-GA	$83.43\pm 0.60$	$0.043\pm 0.012$	$0.477\pm 0.006$	$73.71\pm 0.28$
Random-Circuit steering	$83.93\pm 0.50$	$0.040\pm 0.010$	$0.460\pm 0.017$	$74.26\pm 0.84$

Appendix

Table 18: Comparison with stronger prompt and circuit baselines.

Synthetic Data Generation for Language Models

Synthetic data generation for language models creates artificial text, tasks, and interaction traces to train or evaluate models.

See all 293 papers

Reasoning-Driven Synthetic Data Generation and Evaluation
Mar 31, 2026Tim R. Davidson, Benoit Seguin, Enrico Bacis +2Synthetic Data Generation Synthetic Data Evaluation
EPFL · Google · Google Deepmind
QAQ: Bidirectional Semantic Coherence for Selecting High-Quality Synthetic Code Instructions
Mar 12, 2026Jiayin Lei, Ming Ma, Yunxi Duan +2Instruction-Tuning Data Selection Code Generation
Beijing University of Technology, Beijing, 100124, China · Institute of Neuroscience, State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, 200031, China · University of Chinese Academy of Sciences, Beijing, 100049, China +1
AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions
May 13, 2026Ishika Agarwal, Sofia Stoica, Emre Can Acikgoz +4Synthetic Data Generation Synthetic Data Generation for Language Models
University of Illinois Urbana-Champaign1 · Amazon2

Mar 31, 2026J/K move · Enter open · S save

Reasoning-Driven Synthetic Data Generation and Evaluation

Tim R. Davidson, Benoit Seguin, Enrico Bacis, Cesar Ilharco+1

EPFL · Google · Google Deepmind

Synthetic Data Generation Synthetic Data Evaluation

Open paper AskPDF arXiv

Although many AI applications of interest require specialized multi-modal models, relevant data to train such models is inherently scarce or inaccessible. Filling these gaps with human annotators is prohibitively expensive, error-prone, and time-consuming, leading model builders to increasingly consider synthetic data as a scalable alternative. However, existing synthetic data generation methods often rely on manual prompts, evolutionary algorithms, or extensive seed data from the target distribution - limiting their scalability, explainability, and control. In this paper, we introduce Simula: a novel reasoning-driven framework for data generation and evaluation. It employs a seedless, agentic approach to generate synthetic datasets at scale, allowing users to define desired dataset characteristics through an explainable and controllable process that enables fine-grained resource allocation. We show the efficacy of our approach on a variety of datasets, rigorously testing both intrinsic and downstream properties. Our work (1) offers guidelines for synthetic data mechanism design, (2) provides insights into generating and evaluating synthetic data at scale, and (3) unlocks new opportunities for developing and deploying AI in domains where data scarcity or privacy concerns are paramount.

Abstract

Synthetic Data Generation for Language Models

Explore similar work

Reasoning-Driven Synthetic Data Generation and Evaluation

QAQ: Bidirectional Semantic Coherence for Selecting High-Quality Synthetic Code Instructions

AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions