How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution

Vedant Palit; Florent Draye; Terry Jingchen Zhang; Bernhard Schölkopf; Zhijing Jin

cs.CLAug 27, 2026

How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution

Authors: Vedant Palit, Florent Draye, Terry Jingchen Zhang, Bernhard Schölkopf, Zhijing Jin

Organizations: Jinesis Lab, University of Toronto & Vector Institute · EuroSafeAI · Max Planck Institute for Intelligent Systems, Tübingen, Germany · Hector Foundation ·

Abstract

Transcoder attribution graphs are usually trained to explain why a model assigns high probability to a particular next token. We introduce Concept-Targeted Attribution (CTA), which instead trains attribution graphs with respect to a linear probe direction. CTA therefore yields probe-specific circuits that explain why an internal concept representation arises in a prompt, independently of whether it is expressed in the generated token. Using Cross-Layer Transcoders, we show that these probe-targeted graphs contain predictive structure: graph-level features predict probe accuracy across four widely studied concept categories ( $ρ= 0.91$ , $R^2 = 0.84$ ), while local features identify the sparse components driving per-prompt classification. This connects probe performance to interpretable circuit structure, allowing us to ask not only whether a probe works, but which internal computations make it work. Causal ablations further show that probe-targeted and logit-targeted graphs capture functionally distinct mechanisms. Removing probe-relevant features reduces internal concept scores while largely preserving generated tokens, whereas removing logit-relevant features changes the generated token in 92% to 100% of cases with near-zero effect on probe scores. CTA provides a framework for moving from behavioral probe accuracy to mechanistic explanations of probe performance, enabling more detailed audits of internal concept representations, including safety-critical ones. Our code is available at https://github.com/vedantpalit/concept-targeted-attribution

Figures & tables

Figure 1: The CTA pipeline: CLT replaces MLP modules for interpretability; attribution graphs capture causal feature flow toward the probe; graph-structural features predict probe accuracy; CLT feature activations predict the probe margin.

Figure 2: Probe score trajectories across layers for all four concepts. Shaded bands mark the steepest single-step rise, indicating the belief point

L_{b}

where the residual stream most sharply aligns with the concept direction.

	Gemma-2-2B						Llama-3.2-1B
	Toxicity (L22)			Truthfulness (L14)			Toxicity (L14)			Truthfulness (L1)
Subspace	Post- $S$	$\boldsymbol{\Delta S}$	Flip	Post- $S$	$\boldsymbol{\Delta S}$	Flip	Post- $S$	$\boldsymbol{\Delta S}$	Flip	Post- $S$	$\boldsymbol{\Delta S}$	Flip
Clean	102.52	—	0.0%	41.87	—	0.0%	0.43	—	0.0%	0.11	—	0.0%
$G_{\text{probe}}\setminus G_{\text{logit}}$	41.28	$-$ 61.24	0.0%	29.84	$-$ 12.03	25.0%	$-$ 0.10	$-$ 0.53	0.0%	0.11	$\approx$ 0	0.0%
$G_{\text{logit}}\setminus G_{\text{probe}}$	101.82	0.70	100.0%	41.84	0.03	100.0%	0.43	$\approx$ 0	100.0%	0.09	$-$ 0.02	100.0%
$G_{\text{probe}}\cap G_{\text{logit}}$	$+$ 36.50	$-$ 66.02	38.0%	$+$ 2.93	$-$ 38.94	67.0%	$+$ 0.67	$+$ 0.24	78.0%	$+$ 0.08	$-$ 0.03	17.0%

Table 1: Causal dissociation for Toxicity and Truthfulness depicted at their respective

L_{b}

, implemented across Gemma-2-2B and Llama-3.2-1B. Ablating probe-exclusive nodes collapses

\Delta S

with near-zero behavioural flip while ablating logit-exclusive nodes flips generation in 100% of cases with negligible impact on

S

\Delta S

magnitudes differ between models due to scale, however the dissociation effects remain consistent across both. Extended results in Appendices 7 and D .

Figure 3: Distributions of five selected graph-structural features for high- versus low-accuracy probe layers, shown for Toxicity. The separation is statistically significant for each feature (

p<0.05

, independent

t

-test) and corresponds to medium-to-large effect sizes, with Cohen’s

d

ranging from

+0.88

+1.68

. These results indicate that CTA graphs contain a separable structural signal of a layer’s concept-encoding quality.

Model	Spearman $\rho$	$R^{2}$
Layer baseline	$0.08\pm 0.15$	$-0.19\pm 0.29$
Ridge	$0.65\pm 0.11$	$0.33\pm 0.27$
GB	$\mathbf{0.91}\pm 0.05$	$\mathbf{0.84}\pm 0.14$

Table 2: Predicting probe accuracy from graph-structural features using 5-fold group-aware cross-validation. Bootstrap 95% confidence intervals for GB are

\rho\in[0.724,0.964]

, with

p<0.005

under group-aware permutation.

Figure 4: Top activated examples for L6/F4508. This feature is present in CTAs of more than 50% of positive prompts from the SST-2 sentiment dataset, specifically activating in positive superlatives.

Figure 5: Top predictive features appear 6–20

\times

more frequently than a random feature at

L_{b}

, revealing a stable mechanistic core despite prompt-level circuit variation.

Figure 6: Consecutive-layer Jaccard

J(L,L-1)

for a representative prompt per concept, showing rapid circuit evolution in early layers followed by stabilization at

J\approx 0.9

by intermediate layers.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Probe validation accuracy across layers for Gemma-2-2B (left) and Llama-3.2-1B (centre). Toxicity is linearly decodable from early layers in both models; Sentiment and Truthfulness show weaker trajectories in Llama-3.2-1B, consistent with limited concept encoding at the 1B scale. Right: normalised probe score trajectories for Llama-3.2-1B, with shaded bands marking the belief layer

L_{b}

per concept.

	Gemma-2-2B		Llama-3.2-1B
Concept	$L_{b}$	Val acc.	$L_{b}$	Val acc.
Toxicity	22	90.0%	14	90.9%
Sentiment	21	80.1%	15	63.0%
Reasoning	15	87.6%	9	76.5%
Truthfulness	14	88.2%	1	55.2%

Appendix

Table 3: Probe accuracy at each concept’s belief layer

L_{b}

for both models. Sentiment and Truthfulness show lower ceilings in Llama-3.2-1B, consistent with weaker encoding of affective and factual concepts at the 1B scale.

	Toxicity (L22)				Sentiment (L21)				Reasoning (L15)				Truthfulness (L14)
Subspace	N	$S^{\prime}$	$\boldsymbol{\Delta S}$	Flip	N	$S^{\prime}$	$\boldsymbol{\Delta S}$	Flip	N	$S^{\prime}$	$\boldsymbol{\Delta S}$	Flip	N	$S^{\prime}$	$\boldsymbol{\Delta S}$	Flip
Control	—	$+$ 102.52	—	0%	—	$+$ 28.31	—	0%	—	$-$ 21.44	—	0%	—	$+$ 41.87	—	0%
$G_{\text{probe}}\setminus G_{\text{logit}}$	${\sim}$ 190	$+$ 41.28	$-$ 61.24	0%	${\sim}$ 203	$-$ 12.11	$-$ 40.42	0%	${\sim}$ 130	$-$ 16.39	$+$ 5.05	0%	${\sim}$ 168	$+$ 29.84	$-$ 12.03	25%
$G_{\text{logit}}\setminus G_{\text{probe}}$	${\sim}$ 347	$+$ 101.82	$-$ 0.70	100%	${\sim}$ 394	$+$ 28.70	$+$ 0.39	100%	${\sim}$ 539	$-$ 20.91	$+$ 0.53	92%	${\sim}$ 297	$+$ 41.84	$-$ 0.03	100%
$G_{\text{probe}}\cap G_{\text{logit}}$	${\sim}$ 166	$+$ 36.50	$-$ 66.02	38%	${\sim}$ 174	$+$ 0.57	$-$ 27.74	30%	${\sim}$ 195	$-$ 11.24	$+$ 10.20	17%	${\sim}$ 190	$+$ 2.93	$-$ 38.94	67%

Appendix

Table 4: Full causal ablation results across all four concepts on Gemma-2-2B (see Section 4.2 for the main study).

S^{\prime}

= post-ablation probe score;

\Delta S

= change from control. Reasoning’s

\Delta S

signs are inverted because the L15 probe is anti-aligned with all test prompts (baseline

S=-21.44

); ablation reduces magnitude rather than score, but the dissociation pattern holds: probe-exclusive ablation yields

0\%

flip, logit-exclusive yields

92\%

flip with

\Delta S\approx 0

. The logit-exclusive subgraph is

1.77

–

4.15\times

larger than the probe-exclusive subgraph across concepts (Reasoning’s

4.15\times

ratio reflects sparse probe encoding at the anti-aligned token position).

Model	Raw		De-meaned
	$\rho$	$R^{2}$	$\rho$	$R^{2}$
Layer baseline	$+0.083$	$-0.197$	$+0.587$	$+0.240$
Ridge	$+0.655$	$+0.325$	$+0.819$	$+0.648$
GB	$+0.905$	$+0.835$	$+0.925$	$+0.850$

Appendix

Table 5: 5-fold group-aware CV results for Channel 2 (Gemma-2-2B).

	Raw	De-meaned
Observed $\rho$	$+0.890$	$+0.908$
95% CI on $\rho$	$[+0.724,\,+0.964]$	$[+0.817,\,+0.962]$
Null mean $\rho$	$-0.005\pm 0.128$	$-0.001\pm 0.122$
$p(\rho)$	$<0.005$	$<0.005$

Appendix

Table 6: Bootstrap 95% CI (500 resamples) and permutation test (200 shuffles) for Channel 2 GB model (Gemma-2-2B).

Figure 8: Spearman

\rho

between predicted and actual probe score margins across layers for all four concepts. Toxicity maintains consistently high predictability across all layers; Truthfulness peaks at mid-depth before decaying; Sentiment and Reasoning show weak and noisy signals throughout, reflecting distributed or geometrically encoded concept representations that individual CLT features cannot recover.

Figure 9: Top predictive features appear 8–21

\times

more frequently than a random feature at highest validation accuracy layer, revealing a stable mechanistic core despite prompt-level circuit variation.

Figure 10: Mean per-prompt Jaccard similarity $J(T{@}L_{T},\,R{@}L_{R})$ between Truthfulness and Reasoning attribution graphs across all layer-pair combinations. Cross-concept overlap is non-negligible only when both layers are in the same depth regime.

Panel of Figure 11 — Figure 11: Representative CLT features identified by the probe attribution pipeline, visualized via Neuronpedia. Each panel shows the feature’s layer, index, mean activation, and top/bottom predicted tokens, with activating contexts highlighted. The features encode semantically coherent concepts: L9/F3223 fires on content keywords in structured titles; L9/F2578 and L9/F14198 specialize in geographic place names and city tokens respectively; L1/F3007 captures French proper names and French-language text; and L9/F15693 activates on common English function words across diverse domains. The semantic breadth of these features illustrates that probe-predictive circuits draw on general linguistic primitives alongside concept-specific representations.

Toxicity (L14)

Sentiment (L15)

Reasoning (L9)

Truthfulness (L1)

Subspace

S^{\prime}

\boldsymbol{\Delta S}

Flip

S^{\prime}

\boldsymbol{\Delta S}

Flip

S^{\prime}

\boldsymbol{\Delta S}

Flip

S^{\prime}

\boldsymbol{\Delta S}

Flip

Control

—

+0.43

—

-0.47

—

+0.60

—

+0.11

—

G_{\text{probe}}\setminus G_{\text{logit}}

{\sim}

244

-0.10

\mathbf{-0.53}

{\sim}

168

+2.40

\mathbf{+2.87}

{\sim}

115

+0.39

\mathbf{-0.21}

{\sim}

+0.11

\mathbf{-0.001}

G_{\text{logit}}\setminus G_{\text{probe}}

{\sim}

203

+0.43

+0.002

100%

{\sim}

289

-0.48

-0.011

100%

{\sim}

372

+0.57

-0.028

100%

{\sim}

359

+0.09

-0.020

100%

G_{\text{probe}}\cap G_{\text{logit}}

{\sim}

169

+0.67

+0.24

78%

{\sim}

112

+0.45

+0.92

25%

{\sim}

206

-0.33

-0.93

58%

{\sim}

+0.08

-0.026

17%

Appendix

Table 7: Full causal ablation results across all four concepts on Llama-3.2-1B.

S^{\prime}

= post-ablation probe score;

\Delta S

= change from control. Sentiment baseline is negative as the probe encodes positive sentiment and all prompts are negative-sentiment text; ablating probe-exclusive features correctly reduces the magnitude. Truthfulness at

L_{1}

shows a near-zero internal effect due to the weak probe circuit (

\sim

11 nodes,

55.2\%

accuracy) at that layer; the logit dissociation holds cleanly (

100\%

flip).

Model	$\rho$	$R^{2}$
Layer baseline	$-0.437$	$-0.660$
Ridge	$+0.423$	$+0.069$
GB	$+0.808$	$+0.850$

Appendix

Table 8: 5-fold group-aware CV results for Channel 2 (Llama-3.2-1B).

	Raw
Observed $\rho$	$+0.780$
95% CI on $\rho$	$[+0.368,\,+0.955]$
Null mean $\rho$	$-0.011\pm 0.179$
$p(\rho)$	$<0.005$

Appendix

Table 9: Bootstrap 95% CI (500 resamples) and permutation test (200 shuffles) for Channel 2 GB model (Llama-3.2-1B).

Panel of Figure 12 — Figure 12: Gemma-2-2B : Distributions of five selected graph-structural features for high- versus low-accuracy probe layers, shown for sentiment. The separation is statistically significant for each feature ( $p<0.05$ , independent $t$ -test) and corresponds to medium-to-large effect sizes, with Cohen’s $d$ ranging from $+0.93$ to $+2.59$ . These results indicate that CTA graphs contain a separable structural signal of a layer’s concept-encoding quality.

Panel of Figure 13 — Figure 13: Gemma-2-2B : Distributions of five selected graph-structural features for high- versus low-accuracy probe layers, shown for reasoning. The separation is statistically significant for each feature ( $p<0.05$ , independent $t$ -test) except the influence concentration and corresponds to medium to large effect sizes, with Cohen’s $d$ ranging from $+0.69$ to $+2.00$ . These results indicate that CTA graphs contain a separable structural signal of a layer’s concept-encoding quality.

Panel of Figure 14 — Figure 14: Gemma-2-2B : Distributions of five selected graph-structural features for high- versus low-accuracy probe layers, shown for truthfulness. The separation is not purely significant for features ( $p<0.05$ , independent $t$ -test) except graph density and corresponds to low to medium effect sizes, with Cohen’s $d$ ranging from $-0.84$ to $+0.59$ . These results indicate that CTA graphs contain a separable structural signal of a layer’s concept-encoding quality.

Panel of Figure 15 — Figure 15: Llama-3.2-1B : Distributions of five selected graph-structural features for high- versus low-accuracy probe layers, shown for Sentiment. No feature reaches significance at $p<0.05$ , and effect sizes are small, with Cohen’s $d$ ranging from $-0.98$ to $+0.58$ , consistent with the weak within-concept structural separation observed for Sentiment in Llama-3.2-1B.

Panel of Figure 16 — Figure 16: Llama-3.2-1B : Distributions of five selected graph-structural features for high- versus low-accuracy probe layers, shown for Sentiment. No feature reaches significance at $p<0.05$ , and effect sizes are small, with Cohen’s $d$ ranging from $-0.98$ to $+0.58$ , consistent with the weak within-concept structural separation observed for Sentiment in Llama-3.2-1B.

Panel of Figure 17 — Figure 17: Llama-3.2-1B : Distributions of five selected graph-structural features for high- versus low-accuracy probe layers, shown for Reasoning. Maximum logit edge and graph density reach significance ( $p<0.005$ ), with large effect sizes ( $d=-2.00$ and $d=-1.73$ respectively); the remaining features do not. Cohen’s $d$ ranges from $-2.00$ to $+0.84$ . Negative $d$ values indicate that lower-accuracy layers exhibit higher feature values, a pattern inverted relative to Gemma-2-2B.

Panel of Figure 18 — Figure 18: Llama-3.2-1B : Distributions of five selected graph-structural features for high- versus low-accuracy probe layers, shown for Truthfulness. No feature reaches significance at $p<0.05$ , and effect sizes are small, with Cohen’s $d$ ranging from $-0.40$ to $+0.71$ , reflecting the near-flat val_acc trajectory and weak concept encoding of Truthfulness at the 1B scale.

Mechanistic Interpretability

Mechanistic interpretability studies how neural networks implement behaviors through internal features, circuits, and causal information flow.

See all 391 papers

Attribution Graphs and Causal Probing for Mechanistic Discovery and Bias Repair in Multimodal Generative Learning
Oct 14, 2025Noor Islam S. Mohammad, Uluğ BayazıtGradient-Based Attribution Causal Representation Learning
1,2Istanbul Technical University
ADAG: Automatically Describing Attribution Graphs
Apr 8, 2026Aryaman Arora, Zhengxuan Wu, Jacob Steinhardt +1Mechanistic Interpretability Circuits in Language Models
Stanford University · Transluce
Matryoshka attribution: Learning to attribute language model outputs to representations and weights
Sep 22, 2026Aryaman Arora, Kirill Acharya, Nathan Hu +4LLM Interpretability Circuits in Language Models
Stanford University

Oct 14, 2025J/K move · Enter open · S save

Attribution Graphs and Causal Probing for Mechanistic Discovery and Bias Repair in Multimodal Generative Learning

Noor Islam S. Mohammad, Uluğ Bayazıt

1,2Istanbul Technical University

Gradient-Based Attribution Causal Representation Learning Algorithmic Fairness Causal Attribution Mechanistic Interpretability Algorithmic Bias+2

Open paper AskPDF arXiv

We treat the internals of generative models as mechanistic objects rather than black boxes. We introduce Attribution Graphs (AGs), which extend GradCAM++ to circuit-level representations, and Causal Probing, a do-calculus intervention method for identifying causal latent structures, enabling detection and correction of spurious correlations, demographic biases, and misaligned decision circuits during training. We further propose the Cognitive Alignment Score (CAS), quantifying agreement between model-internal representations and human concepts, a saliency-first privacy mechanism sharing only thresholded attribution nodes, a bias-aware regularizer aligning subgroup statistics, and a Reveal-to-Revise loop integrating attribution signals into parameter updates without separate fine-tuning. Evaluated on CelebA, FairFace, Jigsaw, and HateXplain, our method achieves 94.1% accuracy, 92.3% macro F1, 79.4% IoU-XAI, and 12.7 FID at 72--76% adversarial robustness, while reducing subgroup disparity

Δ_{\mathrm{bias}}

by 41%, demonstrating that mechanistic interpretability, fairness, and generative performance can be jointly optimized.

Abstract

Mechanistic Interpretability

Explore similar work

Attribution Graphs and Causal Probing for Mechanistic Discovery and Bias Repair in Multimodal Generative Learning

ADAG: Automatically Describing Attribution Graphs

Matryoshka attribution: Learning to attribute language model outputs to representations and weights