Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair

Areeb Ahmad; Pratinav Seth; Vinay Kumar Sankarapu

cs.LGOct 1, 2026

Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair

Authors: Areeb Ahmad, Pratinav Seth, Vinay Kumar Sankarapu

Organizations: Lexsi Labs

Abstract

Ablate a component of a language model, and other components often appear to adjust and compensate. This phenomenon, termed self-repair, has been observed repeatedly, but its mechanism remains unclear. The most systematic study to date concluded that self-repair is noisy and unlikely to have a single explanation. We argue that it has one: a gain already present before any ablation. Any intervention on a causally important component can be viewed as a point on a coordinate axis $λ$ , the signed strength of a counterfactual contrast. Hence, conventional ablation methods are uncalibrated points on this axis. We show that the causal repair response for a fine-grained unit $r$ is governed by an affine law, $E_r(λ)=\mathrm{own}_r+γ_rλ$ . The slope $γ_r$ is a fixed coefficient that consistently influences the model, with or without ablation, and its sign determines whether the unit counteracts or reinforces the removed signal. On a factual-verdict task across four models from distinct families (Gemma, Qwen, LLaMA, and Mistral), we identify components including MLP neurons, OV neurons, and singular directions that follow this affine law, 68 of 81 downstream directions in all. Moreover, we can anticipate the magnitude of $γ_r$ from the fixed weights. On the IOI circuit of GPT-2 Small, seven of the ten heads the intervention can reach follow the law, and all seven are counterweights. From this perspective, what may appear as self-repair is a counterweight performing its usual operation when the contrastive signal emerges at the core.

Figures & tables

Figure 1: (a) Any intervention on the core L28.n2046 (Gemma-2-9B-it) is a dose

\lambda

in units of the pair’s class contrast, with

+1

clean,

0

class signal removed, and

-1

inverted. Zero and dataset-mean ablation land at per-prompt doses set by the data (true prompts shown), while resample is predicted to land at the donor’s dose. (b) Its counterweight L38.n7088 (

\gamma_{r}<0

) has a contribution affine in

\lambda

, fitted on three doses (filled), predicting held-out doses (hollow).

Panel of Figure 2 — Figure 2: The nested counterfactual intervention . Open switches cut a read from the residual stream, and dashed arrows inject the replacement value. (a) Clean world, $b=\varnothing$ . The downstream direction $r$ is patched with $a_{r}(x^{\prime};\varnothing)$ , its value on the partner prompt $x^{\prime}$ in the unmodified model. (b) Dosed world, $b=\pi_{\lambda}$ . The core $c$ is first clamped to dose $\lambda$ (Eq. 4 ), and $r$ is then patched with $a_{r}(x^{\prime};\pi_{\lambda})$ , computed on $x^{\prime}$ through this same mirror-dosed graph.

Model	Core(s)	$E^{0}$	$E^{\pi}$	core share
Llama-3-8B-Instruct	L18.n11065	$7.78$ $(0.11)$	$3.67$ $(0.05)$	$52.7\%$
Gemma-2-9B-it	L28.n2046 $+$ L33.n4294	$9.50$ $(0.11)$	$4.38$ $(0.05)$	$53.9\%$
Qwen2.5-7B-Instruct	L22.n13149 $+$ L24.n14758	$18.76$ $(0.17)$	$8.46$ $(0.07)$	$54.9\%$
Mistral-7B-Instruct-v0.3	L20.n14286 $+$ L19.n8228	$11.59$ $(0.15)$	$5.03$ $(0.06)$	$56.6\%$

Table 1: The core carries roughly half the total effect. Total effect

E=E_{T}+E_{F}

of the whole model on the verdict, where

E_{T}

is the True

-

False logit margin on the true statement and

E_{F}

the False

-

True margin on the false statement, at the answer token.

E^{0}

is the clean model, and

E^{\pi}

is the same with the core swapped to its full counterfactual (

k=1

\lambda=-1

). The last column is the share of the clean total effect the core is responsible for,

(E^{0}-E^{\pi})/E^{0}

. Standard errors in parentheses.

Panel of Figure 3 — Figure 3: The featured counterweight in each family moves against the core. Net contribution $E^{\pi}_{r}=E_{T}+E_{F}$ of one direction per family against the dose $\lambda$ on that family’s certified core(s). The shaded band is the on-manifold window $|\lambda|\leq 1$ , where $+1$ is the clean model and $-1$ the counterfactual. Each opposes the verdict at rest and crosses zero on both classes at a sufficient dose. Points are per-dose means, and standard errors are smaller than the markers (paired $t$ , App. F.1 )

Figure 4: The Affine Law , four model families. For one exemplar direction per model, the erased margin

E_{r}^{\pi}

against the dose

\lambda

. Lines are fit on the three filled rungs, with the hollow rungs run afterward and held out. The band is the refit on all five.

Figure 5: Coupling strength is legible in the weights . Weight-space alignment

z

between the core’s write and each direction’s read, computed from the checkpoint alone, against the intervention-measured coupling

|{}\gamma_{r}|{}

, pooled on a within-model rank scale (left,

\rho=+0.73

n=81

) and per model (right). Hollow markers are upstream components with no causal path to the core.

Figure 6: Deletion on the dose axis. For each certified core, the distribution over pairs of the dose

\lambda(v)=2(m-v)/\Delta

administered by a zero clamp (

v=0

, blue) and a dataset-mean clamp (

v=\bar{a}

, orange), with medians marked. The vertical line is

\lambda=0

, the shaded band is the on-manifold window, the broken axis is the off-manifold tail.

Appendix figures & tables61 assets

Supplementary material from the paper’s appendix.

Appendix

Main-text claim	Supporting appendix section
Contribution 1 (compensation is standing)	§ F (recruitment), § G (other primaries), § H (base checkpoints), § I (not a LayerNorm artefact)
Contribution 2 (a dose axis and a nested intervention)	§ E , § N
Contribution 3 (the coupling has an affine form)	§ J and § K , with IOI in § L
Contribution 4 (named counterweights and relays, coupling legible in the weights)	§ D (catalogues, clean census), § M

Appendix

Table 2: Claim-to-evidence map. Each row names the section that carries the primary burden of proof. Supporting figures/tables for a claim may also appear incidentally elsewhere.

Symbol	Meaning
$x$	The prompt (SCM exogenous variable).
$x^{\prime}$	The matched counterfactual partner of $x$ (opposite class, identical phrasing).
$b$	World index: $b=\varnothing$ (clean), $b=\pi_{k}$ (core dosed to $k$ ; the main text writes $\pi_{\lambda}$ for the same world), $b=\delta$ (learned perturbation).
$r$	A weight-space direction (an mlp_neuron , ov_neuron , ov_svd , or mlp_svd unit; § B ).
$\omega_{r}$	$r$ ’s fixed write vector.
$a_{r}(x;b)$	$r$ ’s activation coefficient on prompt $x$ in world $b$ .

Appendix

Table 3: Notation used throughout the appendix. The dose is always

\lambda

, and

x

always denotes a prompt.

Gated MLP, neuron $j$ of layer $\ell$ : $g_{j}=\mathbf{w}^{\mathrm{gate}}_{j}\!\cdot x$ $u_{j}=\mathbf{w}^{\mathrm{up}}_{j}\!\cdot x$ $\mathrm{MLP}(x)=\sum_{j}a_{j}(x)\,W_{\mathrm{down}}[:,j]$
Model	MLP input $x$	Neuron scalar $a_{j}(x)$	Residual write
Llama-3-8B-Instruct	$\mathrm{RMSNorm}_{\mathrm{post\text{-}attn}}(h)$	$\mathrm{SiLU}(g_{j})\,u_{j}$	$W_{\mathrm{down}}[:,j]$
Gemma-2-9b-it	$\mathrm{RMSNorm}_{\mathrm{pre\text{-}ffw}}(h)$	$\mathrm{GELU}_{\tanh}(g_{j})\,u_{j}$	$\bm{\gamma}_{\mathrm{post}}\odot W_{\mathrm{down}}[:,j]$
Qwen2.5-7B-Instruct	$\mathrm{RMSNorm}_{\mathrm{post\text{-}attn}}(h)$	$\mathrm{SiLU}(g_{j})\,u_{j}$	$W_{\mathrm{down}}[:,j]$
Mistral-7B-Instruct-v0.3	$\mathrm{RMSNorm}_{\mathrm{post\text{-}attn}}(h)$	$\mathrm{SiLU}(g_{j})\,u_{j}$	$W_{\mathrm{down}}[:,j]$

Appendix

Table 4: The mlp_neuron scalar, per model. Every intervention in this paper acts on

a_{j}

, which is coordinate

j

of the down_proj input and the scalar multiplying that neuron’s fixed write vector. All four models use a gated MLP, and in all four, the up branch is linear . Only the gate passes through the nonlinearity, so the scalar is

\sigma(g_{j})\,u_{j}

and never

\sigma(g_{j})\,\sigma(u_{j})

Experiment	$n$	Section
Llama single-core swap ( L18.n11065 )	532	§ F
Gemma joint-core swap ( L28.n2046+L33.n4294 )	583	§ F
Qwen joint-core swap ( L22.n13149+L24.n14758 )	478	§ F
Mistral joint-core swap ( L20.n14286+L19.n8228 )	598	§ F
Gemma / Qwen dose-resolved single-core comparisons	100	§ F
Learned perturbation ( perfect_on_pairs , all models)	– (full pool)	§ G

Appendix

Table 5: Sample sizes referenced in this appendix, consolidated from individual table/figure captions.

Axis	Promoted (top-4)	Suppressed (bottom-4)
L23.n8972	iller, rello, LC, etz	lie , lies , lying , Lie
L19.n2738	ledged, Alo, ADDE , くだ (Jp.)	illusion , illusions , false , illusion
L23.n9811	truth , Truth , truth , Truth	oby, imore, Tell, tell
L19.n3754	Edition, onte , フ (Jp.) , rale	sure , yes , yes , Sure
L22.h1.svd117	miss, missing, chances, misses	true , true , Important , TRUE
L24.n4543	yes , true , True , Yes	OrNil, uelle, illard, izh (Cyr.)

Appendix

Table 6: Vocabulary poles of the recruited directions in Llama-3-8B-Instruct (

\omega_{r}^{\!\top}W_{U}

, weight-fixed, intervention-independent, with the top-

4

promoted and the bottom-

4

suppressed). Green marks truth-coding tokens, red falsity-coding tokens, gray non-interpretable subword fragments.

Axis	Promoted (top-4)	Suppressed (bottom-4)
L38.n7088	wrong , wrong , Wrong , Wrong	__). , BarStyle , preferably , AutoSize
L39.n10085	fake , Fake , pseudo , pseud	argout , cyd , MessageOf , MigrationBuilder
L38.n854	ModelExpression , __": , ituary , orghini	true , true , True , True
L41.n8771	SOUNDBITE , estekak , protoimpl , ChrTalk	True , true , True , true
L39.n10210	incorrect , incorrect , Incorrect , WRONG	IBOutlet , setopt , openzeppelin , Geos
L41.n3789	haikusbot , writeFieldEnd , complexType , getDescription	real , skuto , truly , genuine

Appendix

Table 7: Logit-lens poles of the recruited Gemma-2-9b-it directions and the two cores (

\omega_{r}\!\cdot\!W_{U}

, weight-fixed, intervention-independent, with the top-4 promoted and the bottom-4 suppressed). Green

=

truth-coding, red

=

falsity-coding, gray

=

non-interpretable subword. Most recruited anti-directions carry a clean English truth/falsity pole on one side against subword noise on the other. The core L28.n2046 carries the cross-lingual 否 (no) beside English yes/true .

Axis	Promoted (top-4)	Suppressed (bottom-4)
L27.n4228	$\varnothing$ , -Headers , ómo , ’}}>	truly , 真正 (genuine) , true , 真正的 (genuine)
L22.n13149	true , True , _true , true	false , False , false , False
L20.n2073	False , false , False , false	true , True , true , True
L22.n4538	false , false , False , False	true , True , True , true
L27.n13033	fa , ase , л (Cyr.) , fa	false , False , False , false
L24.n3278	true , true , 真实 (real) , True	ulfill , .Gray , naz , 泄露 (Zh.)

Appendix

Table 8: A catalogue of interpretable truth/falsity directions in Qwen2.5-7B-Instruct (

\omega_{r}\!\cdot\!W_{U}

, weight-fixed, intervention-independent, with the top-4 promoted and the bottom-4 suppressed). Green

=

truth-coding, red

=

falsity-coding, gray

=

non-interpretable subword. As in Gemma-2-9b and Llama-3-8b, Qwen carries a large population of directions whose unembedding poles encode the True/False verdict for this task (

6{,}310

marked across the four component families). A representative sample spanning MLP neurons (layers L20 to L27) and attention (OV) axes is listed by

|\!\cos(\omega_{r},w_{TF})|

. Coding is markedly cross-lingual, pairing Chinese 真 (true)/错 (wrong)/假 (fake)/虚假 (false) with English true/false/fake . Whether each contributes to, opposes, or is dormant for the realized verdict is measured separately ( § F ).

Axis	Promoted (top-4)	Suppressed (bottom-4)
L15.n11835	fake , invalid , false , False	riel , lib , trends , HAL
L16.n1170	ensures , ensuring , confirmed , ingo	STO , FALSE , false , $\varnothing$
L16.n5090	ugno , iten , Graf , rolog	pret , pretend , fake , nomin
L16.n9991	false , fals , False , false	$\varnothing$ , amon , bert , bis
L17.n11073	integrity , respons , genuine , proper	artificial , stere , unsafe , false
L17.n13852	true , real , True , TRUE	asp , ape , dom , Mend

Appendix

Table 9: A catalogue of interpretable truth/falsity directions in Mistral-7B-Instruct-v0.3 (

\omega_{r}\!\cdot\!W_{U}

, weight-fixed, intervention-independent, with the top-4 promoted and the bottom-4 suppressed). Green

=

truth-coding, red

=

falsity-coding, gray

=

non-interpretable subword. All 35 bench directions are listed, ordered by layer. Of these, 14 carry a clean bipolar verdict axis and the rest carry a clean pole on one side against subword noise on the other.

\dagger

marks the core(s) used as the primary intervention.

secondary	family	detects	$T$	$F$	$T$	$F$	$T$	$F$	$T$	$F$	$T$	$F$	flip@
			$E^{0}$ (clean)		$E^{\pi}\times 1$		$E^{\pi}\times 2$		$E^{\pi}\times 3$		$E^{\pi}\times 4$
L23.n8972	mlp neuron	lie	$-0.04$	$-0.06$	$+0.02^{\scriptscriptstyle 13}$	$+0.02^{\scriptscriptstyle 13}$	$+0.06^{\scriptscriptstyle 34}$	$+0.04^{\scriptscriptstyle 32}$	$+0.11^{\scriptscriptstyle 47}$	$+0.06^{\scriptscriptstyle 41}$	$+0.16^{\scriptscriptstyle 56}$	$+0.09^{\scriptscriptstyle 52}$	$\times 1$
L23.n9811	mlp neuron	truth	$-0.02$	$-0.02$	$+0.00^{\scriptscriptstyle 3}$	$+0.00^{\scriptscriptstyle 4}$	$+0.05^{\scriptscriptstyle 25}$	$+0.03^{\scriptscriptstyle 22}$	$+0.10^{\scriptscriptstyle 37}$	$+0.05^{\scriptscriptstyle 34}$	$+0.11^{\scriptscriptstyle 48}$	$+0.02^{\scriptscriptstyle 8}$	$\times 1$
L19.n3754	mlp neuron	sure	$-0.01$	$-0.02$	$+0.00^{\scriptscriptstyle 3}$	$+0.00^{\scriptscriptstyle 4}$	$+0.01^{\scriptscriptstyle 11}$	$+0.02^{\scriptscriptstyle 15}$	$+0.02^{\scriptscriptstyle 18}$	$+0.02^{\scriptscriptstyle 13}$	$+0.02^{\scriptscriptstyle 14}$	$+0.01^{\scriptscriptstyle 9}$	$\times 1$
L21.n4049	mlp neuron	wrong	$-0.01$	$-0.01$	$+0.01^{\scriptscriptstyle 8}$	$+0.01^{\scriptscriptstyle 10}$	$+0.03^{\scriptscriptstyle 21}$	$+0.03^{\scriptscriptstyle 28}$	$+0.04^{\scriptscriptstyle 29}$	$+0.05^{\scriptscriptstyle 37}$	$+0.03^{\scriptscriptstyle 19}$	$+0.04^{\scriptscriptstyle 30}$	$\times 1$
L19.n2738	mlp neuron	illusion	$-0.01$	$-0.01$	$+0.00$	$+0.00^{\scriptscriptstyle 2}$	$+0.01^{\scriptscriptstyle 5}$	$+0.02^{\scriptscriptstyle 13}$	$+0.01^{\scriptscriptstyle 6}$	$+0.04^{\scriptscriptstyle 30}$	$+0.01^{\scriptscriptstyle 4}$	$+0.06^{\scriptscriptstyle 42}$	$\times 2$

Appendix

Table 10: Backup recruitment across dose under ablation of the Llama-3-8B-Instruct verdict core ( L18.n11065 , contrastive swap, sec_ref=regime ,

n{=}532

). Per side, the clean erased margin

E^{0}_{T}=D_{TF}(T)-D_{TF}(T,\mathrm{sec})

and

E^{0}_{F}=D_{TF}(F,\mathrm{sec})-D_{TF}(F)

(

+

supports the verdict, with colour giving the clean class as anti / contributor / null ), then

E^{\pi}_{T}

and

E^{\pi}_{F}

under the ablation at each swap dose. Green

=

reliably

>0

(now a contributor on that side), red

=

reliably

<0

. The superscript is the paired

t=|\mathrm{mean}|/\mathrm{sem}

. Uncoloured cells are below threshold (

t<2

). Row shade

=

clean class, and ‘flip@’

=

first dose a clean null/anti axis is reliably

>0

on both sides. Across the coloured entries the median

t\approx 8

(up to

t\approx 56

). The pair-averaged verdict gap runs

-7.8

(clean)

\to-3.7,+1.4,+6.8,+12.3

across

\times 1..\times 4

, so the verdict gives way between

\times 1

and

\times 2

secondary	family	detects	$T$	$F$	$T$	$F$	$T$	$F$	$T$	$F$	$T$	$F$	flip@
			$E^{0}$ (clean)		$E^{\pi}\times 1$		$E^{\pi}\times 2$		$E^{\pi}\times 3$		$E^{\pi}\times 4$
L27.n13033	mlp neuron	false	$-2.86$	$-2.88$	$-0.50^{\scriptscriptstyle 15}$	$-0.51^{\scriptscriptstyle 15}$	$+2.93^{\scriptscriptstyle 116}$	$+3.22^{\scriptscriptstyle 109}$	$+3.95^{\scriptscriptstyle 189}$	$+4.45^{\scriptscriptstyle 178}$	$+4.08^{\scriptscriptstyle 199}$	$+4.65^{\scriptscriptstyle 201}$	$\times 2$
L27.n4228	mlp neuron	truly	$-0.84$	$-0.87$	$+0.03^{\scriptscriptstyle 4}$	$+0.03^{\scriptscriptstyle 4}$	$+0.37^{\scriptscriptstyle 29}$	$+0.41^{\scriptscriptstyle 29}$	$+1.43^{\scriptscriptstyle 47}$	$+1.58^{\scriptscriptstyle 47}$	$+2.62^{\scriptscriptstyle 63}$	$+2.96^{\scriptscriptstyle 63}$	$\times 1$
L25.n4929	mlp neuron	false	$-0.25$	$-0.07$	$-0.01^{\scriptscriptstyle 3}$	$-0.00^{\scriptscriptstyle 2}$	$+0.02^{\scriptscriptstyle 12}$	$+0.05^{\scriptscriptstyle 17}$	$+0.05^{\scriptscriptstyle 21}$	$+0.21^{\scriptscriptstyle 35}$	$+0.05^{\scriptscriptstyle 21}$	$+0.41^{\scriptscriptstyle 45}$	$\times 2$
L23.n8341	mlp neuron	illusion	$-0.18$	$-0.11$	$+0.00^{\scriptscriptstyle 2}$	$+0.00$	$+0.04^{\scriptscriptstyle 18}$	$+0.15^{\scriptscriptstyle 29}$	$+0.08^{\scriptscriptstyle 28}$	$+0.49^{\scriptscriptstyle 52}$	$+0.12^{\scriptscriptstyle 37}$	$+0.84^{\scriptscriptstyle 66}$	$\times 2$
L24.n12848	mlp neuron	false	$-0.21$	$-0.04$	$+0.01$	$+0.00$	$+0.11^{\scriptscriptstyle 25}$	$+0.29^{\scriptscriptstyle 33}$	$+0.04^{\scriptscriptstyle 15}$	$+0.32^{\scriptscriptstyle 41}$	$-0.01^{\scriptscriptstyle 3}$	$+0.26^{\scriptscriptstyle 35}$	$\times 2$

Appendix

Table 11: Backup recruitment across dose under JOINT ablation of both Qwen2.5-7B-Instruct verdict cores ( L22.n13149

+

L24.n14758 , contrastive swap, sec_ref=regime ,

n{=}478

). Columns as in Table 10 . Across the coloured entries the median

t\approx 21

(up to

t\approx 201

\kappa_{\mathrm{base}}=-18.8

, and the joint

\kappa_{\mathrm{primary}}

\times 1..\times 4=-8.5,+4.7,+15.8,+23.4

crosses

0

\times 2

, where the verdict flips. Null-class rows stay

\sim\!0

across all doses (specificity).

secondary	family	detects	$T$	$F$	$T$	$F$	$T$	$F$	$T$	$F$	$T$	$F$	flip@
			$E^{0}$ (clean)		$E^{\pi}\times 1$		$E^{\pi}\times 2$		$E^{\pi}\times 3$		$E^{\pi}\times 4$
L38.n7088	mlp neuron	wrong	$-1.420$	$-0.715$	$+0.474^{\scriptscriptstyle 72}$	$+0.613^{\scriptscriptstyle 54}$	$+1.060^{\scriptscriptstyle 194}$	$+2.202^{\scriptscriptstyle 122}$	$+1.178^{\scriptscriptstyle 228}$	$+2.587^{\scriptscriptstyle 163}$	$+1.215^{\scriptscriptstyle 252}$	$+2.727^{\scriptscriptstyle 197}$	$\times 1$
L39.n10085	mlp neuron	fake	$-0.504$	$-0.418$	$-0.364^{\scriptscriptstyle 88}$	$-0.342^{\scriptscriptstyle 94}$	$-0.179^{\scriptscriptstyle 44}$	$-0.181^{\scriptscriptstyle 43}$	$-0.017^{\scriptscriptstyle 4}$	$-0.022^{\scriptscriptstyle 4}$	$+0.071^{\scriptscriptstyle 15}$	$+0.077^{\scriptscriptstyle 15}$	$\times 4$
L38.n854	mlp neuron	true	$-0.179$	$-0.281$	$-0.044^{\scriptscriptstyle 26}$	$-0.055^{\scriptscriptstyle 26}$	$+0.075^{\scriptscriptstyle 35}$	$+0.072^{\scriptscriptstyle 40}$	$+0.177^{\scriptscriptstyle 55}$	$+0.146^{\scriptscriptstyle 66}$	$+0.247^{\scriptscriptstyle 69}$	$+0.192^{\scriptscriptstyle 78}$	$\times 2$
L41.n8771	mlp neuron	True	$-0.107$	$-0.202$	$+0.047^{\scriptscriptstyle 16}$	$+0.057^{\scriptscriptstyle 16}$	$+0.261^{\scriptscriptstyle 69}$	$+0.221^{\scriptscriptstyle 66}$	$+0.339^{\scriptscriptstyle 80}$	$+0.236^{\scriptscriptstyle 80}$	$+0.354^{\scriptscriptstyle 90}$	$+0.232^{\scriptscriptstyle 86}$	$\times 1$
L39.n10210	mlp neuron	incorrect	$-0.167$	$-0.127$	$-0.019^{\scriptscriptstyle 13}$	$-0.018^{\scriptscriptstyle 13}$	$+0.046^{\scriptscriptstyle 30}$	$+0.048^{\scriptscriptstyle 31}$	$+0.019^{\scriptscriptstyle 15}$	$+0.021^{\scriptscriptstyle 17}$	$+0.007^{\scriptscriptstyle 5}$	$+0.006^{\scriptscriptstyle 5}$	$\times 2$

Appendix

Table 12: Backup recruitment across dose under JOINT ablation of both Gemma-2-9b-it verdict cores ( L28.n2046

+

L33.n4294 , contrastive swap, sec_ref=regime ,

n{=}583

). Columns as in Table 10 . Across the coloured entries the median

t\approx 20

(up to

t\approx 252

\kappa_{\mathrm{base}}=-9.50

, and joint

\kappa_{\mathrm{primary}}

\times 1..\times 4=-4.38,+2.30,+5.59,+7.14

(crosses

0

\times 2

). The one null-class row, L36.n1119 , turns reliably negative from

\times 2

(

E^{\pi}

between

-0.10

and

-0.22

) and never supports the verdict.

secondary	family	detects	$T$	$F$	$T$	$F$	$T$	$F$	$T$	$F$	$T$	$F$	flip@
			$E^{0}$ (clean)		$E^{\pi}\times 1$		$E^{\pi}\times 2$		$E^{\pi}\times 3$		$E^{\pi}\times 4$
L31.n13669	mlp neuron	true	$-1.929$	$-2.083$	$-1.470^{\scriptscriptstyle 88}$	$-1.624^{\scriptscriptstyle 79}$	$+0.227^{\scriptscriptstyle 7}$	$+0.209^{\scriptscriptstyle 6}$	$+0.736^{\scriptscriptstyle 43}$	$+0.793^{\scriptscriptstyle 44}$	$+0.522^{\scriptscriptstyle 40}$	$+0.583^{\scriptscriptstyle 40}$	$\times 2$
L31.n8773	mlp neuron	false	$-0.771$	$-0.748$	$+0.024$	$+0.050^{\scriptscriptstyle 2}$	$+1.088^{\scriptscriptstyle 61}$	$+1.240^{\scriptscriptstyle 56}$	$+1.489^{\scriptscriptstyle 78}$	$+1.754^{\scriptscriptstyle 72}$	$+1.438^{\scriptscriptstyle 58}$	$+1.732^{\scriptscriptstyle 56}$	$\times 2$
L28.n4079	mlp neuron	true	$-0.254$	$-0.757$	$-0.062^{\scriptscriptstyle 33}$	$-0.149^{\scriptscriptstyle 38}$	$+0.415^{\scriptscriptstyle 37}$	$+0.325^{\scriptscriptstyle 56}$	$+0.958^{\scriptscriptstyle 55}$	$+0.511^{\scriptscriptstyle 95}$	$+1.153^{\scriptscriptstyle 88}$	$+0.569^{\scriptscriptstyle 129}$	$\times 2$
L22.h23.svd2	ov svd	wrong	$-0.092$	$-0.132$	$-0.049^{\scriptscriptstyle 29}$	$-0.092^{\scriptscriptstyle 37}$	$-0.018^{\scriptscriptstyle 12}$	$-0.017^{\scriptscriptstyle 10}$	$+0.010^{\scriptscriptstyle 8}$	$+0.018^{\scriptscriptstyle 11}$	$+0.032^{\scriptscriptstyle 25}$	$+0.053^{\scriptscriptstyle 36}$	$\times 3$
L22.n8735	mlp neuron	false	$-0.123$	$-0.083$	$-0.045^{\scriptscriptstyle 16}$	$-0.096^{\scriptscriptstyle 23}$	$+0.018^{\scriptscriptstyle 5}$	$-0.034^{\scriptscriptstyle 6}$	$+0.025^{\scriptscriptstyle 11}$	$+0.058^{\scriptscriptstyle 8}$	$+0.008^{\scriptscriptstyle 5}$	$+0.112^{\scriptscriptstyle 18}$	$\times 3$

Appendix

Table 13: Backup recruitment across dose under JOINT ablation of both Mistral-7B-Instruct-v0.3 verdict cores ( L20.n14286

+

L19.n8228 , contrastive swap, sec_ref=regime ,

n{=}598

). Rows are the clean anti/null axes that become verdict contributors under the ablation (the recruitment set). Clean contributors and never-flipping null controls are omitted (they are in the total-effect scan). Columns as in Table 10 . Across the coloured entries the median

t\approx 20

(up to

t\approx 129

\kappa_{\mathrm{base}}=-11.6

, and joint

\kappa_{\mathrm{primary}}

\times 1..\times 4=-5.0,+3.9,+10.5,+15.9

(crosses

0

\times 2

, where the verdict flips).

Figure 7: Interpretable truth detectors that are dormant or opposing in the clean model become a verdict contributor when the core neuron is ablated, in proportion to dose and on both prompt classes. For each named secondary

r

we exchange its activation with the value it takes on the contrastive partner within the ablated regime (

\mathrm{sec}_{r}

in Eq. 5 ), the SCM-faithful counterfactual, and read off

E^{\pi}_{r}

(Eq. 6 ), the faithful True

-

False margin the swap erases in the ablated world

\pi

(

E>0

supports the verdict,

E<0

opposes it). The horizontal axis sweeps the dose from the clean baseline

E^{0}

(leftmost) through swap scales

k\in\{1,2,3,4\}

, driving the primary neuron L18.n11065 progressively past its contrastive value. The left panel shows true statements, the right fake statements. Each detector starts in the dormant/opposing band (

E\leq 0

) and rises into the contributor band (

E>0

) as the ablation deepens, monotonically for the core detectors, led by the lie-writer L23.n8972 . A contributor must be positive on both panels, and the both-classes gate also exposes the lone one-sided case, L24.n4543 ( yes / true ), which rises on true statements but reverses on fake statements at high dose. Junk-poled directions (grey dashed) stay pinned at zero. Llama-3-8B-Instruct .

Figure 8: Dose-resolved recruitment under joint ablation of both Gemma-2-9b-it verdict cores ( L28.n2046

+

L33.n4294 ). Per-side erased margin

E^{\pi}

vs. swap dose ( clean

=E^{0}

, then

\times 1..\times 4

), with true statements on the left and fake statements on the right. Each coloured curve is one recruited anti-detector, and the dotted grey curves are the null controls. As the joint swap strengthens, the opposing detectors are pulled from

E^{\pi}<0

up through zero into positive (verdict-supporting) territory, while the null controls stay flat, so recruitment is specific to truth axes, including opposing ones.

Figure 9: Dose-resolved recruitment under joint ablation of both Qwen2.5-7B-Instruct verdict cores ( L22.n13149

+

L24.n14758 , contrastive swap, sec_ref=regime ,

n{=}100

). Per-detector erased margin

E^{\pi}

on true statements (left) and fake statements (right) as the joint swap dose increases from clean to

\times 4

, where

E^{\pi}>0

means the detector now supports the verdict. The four identified detectors (colour) start as clean opponents (

E^{0}<0

, shaded clean band) and are progressively recruited. L27.n13033 ( false ) rockets from

-2

+3.7/+4.2

nats, flipping at

\times 2

, and L27.n4228 (suppresses true ) flips at

\times 1

. The remaining anti detectors (slate) follow the same upward drift at smaller magnitude, while the null controls (dotted) stay pinned at zero across all doses. The joint ablation flips the verdict at

\times 2

(

\kappa_{\mathrm{primary}}

crosses

0

), so beyond that the panels report reconstitution of the erased fact/fake signal by the backup detectors.

Figure 10: Dose-resolved recruitment under joint ablation of both Mistral-7B-Instruct-v0.3 verdict cores ( L20.n14286

+

L19.n8228 , contrastive swap, sec_ref=regime ,

n{=}598

). Per-side erased margin

E^{\pi}

vs. swap dose ( clean

=E^{0}

, then

\times 1..\times 4

), with true statements on the left and fake statements on the right, following the layout of Figs. 8 to 9 . The eight recruited anti-detectors of Table 13 are pulled from

E^{\pi}<0

through zero as the joint swap strengthens. L31.n8773 and L31.n13669 show the largest swings (clean

E^{0}\approx-0.8

and

-1.9

-2.1

, rising to

E^{\pi}\approx+1.4

1.8

and

+0.5

0.8

\times 3

\times 4

), and L28.n4079 reaches

+1.15

on the true side by

\times 4

, while L22.h23.svd2 , L22.n8735 , L22.n1070 and L20.n4786 recruit later and more weakly, consistent with the flip@ column of Table 13 .

Figure 11: Removing the verdict as a whole recruits more backup than removing either core separately (Mistral-7B-Instruct-v0.3,

n{=}598

). Over the

9

axes that oppose the verdict in the clean model, the panels show mean net recruitment

R

(left) and the number crossing into net contribution (right), with the cores L20.n14286 and L19.n8228 ablated jointly (solid) or each alone (dashed, dotted). Joint ablation dominates at every dose (

R{=}1.52

8/9

recruited at

\times 4

, vs.

1.36

4

and

0.61

5

for the single cores) and is super-additive at low dose, exceeding the sum of the two single-core effects at

\times 1

and

\times 2

before saturating.

Figure 12: Single-core vs. joint-core recruitment across dose (Gemma-2-9b-it, contrastive swap, sec_ref=regime ,

n{=}100

). Recruitment of the

12

clean anti detectors when we ablate each verdict core alone ( L28.n2046 dashed, L33.n4294 dotted) versus jointly (both cores, solid), as a function of the swap dose

\times 1..\times 4

. The left panel shows mean net recruitment

R=R_{T}+R_{F}

averaged over the

12

detectors. The joint ablation dominates either core alone at every dose (

0.43/0.74/0.85/0.89

vs. L33

0.34/0.57/0.65/0.69

vs. L28

0.13/0.37/0.53/0.60

). The right panel shows the number of the

12

that cross into net contribution (

E^{\pi}>0

on both sides). The joint ablation recruits

10/12

\times 2

and

11/12

\times 4

. L33 leads at

\times 1

(

1

vs.

0

), the two single cores tie at

\times 2

(

3

each), and L28 then overtakes,

8

vs.

5

\times 3

and

11

vs.

6

\times 4

, where it matches the joint count. No single core reproduces the joint mean recruitment.

Figure 13: Single-core vs. joint-core recruitment across dose (Qwen2.5-7B-Instruct, contrastive swap, sec_ref=regime ,

n{=}100

). Recruitment of the

10

clean anti detectors when we ablate the dominant truth core L22.n13149 alone (dashed), the correctness co-core L24.n14758 alone (dotted), or both jointly (solid), vs. swap dose

\times 1..\times 4

. The left panel shows mean net recruitment

R=R_{T}+R_{F}

over the anti detectors. The joint ablation dominates either core alone at every dose, but L22.n13149 alone, which carries

\sim\!35\%

of the clean fact/fake margin, recovers most of it, while the correctness co-core L24.n14758 recruits far less. The right panel shows the number crossing into net contribution (

E^{\pi}>0

both sides). The joint ablation peaks at

8/10

\times 3

and falls to

6/10

\times 4

, while L22.n13149 alone reaches

7/10

and L24.n14758 alone

4/10

. Unlike Gemma-2-9b, whose two co-equal cores each recruit only partially, Qwen’s backup recruitment is carried predominantly by the single dominant core.

Figure 14: Backup recruitment is driven by the cores jointly rather than by either core alone (Mistral-7B-Instruct-v0.3, contrastive swap, sec_ref=regime , perfect-on-pairs partition,

n{=}598

). One panel per recruited detector, showing its net erased margin

E^{\pi}=E^{\pi}_{T}+E^{\pi}_{F}

against swap dose when the two verdict cores L20.n14286 and L19.n8228 are ablated jointly (solid) or each alone (dashed, dotted).

E^{\pi}>0

means the detector now supports the verdict, and the shaded band is the gain from joint ablation beyond whichever single core does better. Every panel is recruited by the joint ablation at

\times 2

. L31.n8773 moves

-1.52\!\rightarrow\!+3.17

, L28.n4079

-1.01\!\rightarrow\!+1.72

, L31.n13669

-4.01\!\rightarrow\!+1.10

(clean

\rightarrow

joint at

\times 4

). Single-core ablation is strictly weaker and asymmetric. L20.n14286 alone reaches

+2.71

+1.27

and

+1.21

and recruits

4

of the

9

clean-anti axes in the bench, L19.n8228 alone reaches only

+0.38

+0.09

and

-2.62

(it never recruits L31.n13669 at any dose), and both together as a joint ablation recruit

8

9

Figure 15: Backup detectors are recruited into verdict contributors, and more strongly under joint than single-core ablation (Gemma-2-9b-it). Each panel follows one of the four identified secondary detectors, and the net erased margin

E^{\pi}=E^{\pi}_{T}+E^{\pi}_{F}

is plotted at the clean state (

E^{0}

) and at contrastive-swap doses

\times 1

\times 4

, under joint ablation of both verdict cores ( L28.n2046

+

L33.n4294 , solid) versus each core alone ( L28.n2046 dashed, L33.n4294 dotted). All four are anti directions at rest (

E^{0}<0

, with values

-2.14,-0.31,-0.46,-0.92

for L38.n7088 , L41.n8771 , L38.n854 , L39.n10085 ) yet cross zero under ablation. Joint ablation flips L38.n7088 and L41.n8771 already at

\times 1

, L38.n854 at

\times 2

, and even the initially strongly opposing L39.n10085 by

\times 4

n{=}583

perfect-on-pairs prompts.

Figure 16: Backup detectors are recruited into verdict contributors, and more strongly under joint than single-core ablation (Qwen2.5-7B-Instruct). Each panel follows one of the four identified secondary detectors and plots the net erased margin

E^{\pi}=E^{\pi}_{T}+E^{\pi}_{F}

at the clean state (

E^{0}

) and at contrastive-swap doses

\times 1

\times 4

, under joint ablation of both verdict cores ( L22.n13149

+

L24.n14758 , solid) versus each core alone (truth core L22.n13149 dashed, correctness co-core L24.n14758 dotted). All four are anti at rest (

E^{0}<0

, with values

-5.74,-1.71,-0.29,-0.32

for L27.n13033 , L27.n4228 , L23.n8341 , L25.n4929 ). Joint ablation flips L27.n4228 at

\times 1

and the remaining three by

\times 2

. Several directions are recruited by only one of the two cores unless both are removed together.

n{=}478

perfect-on-pairs prompts.

Label Assignment Rule ( $\pi$ )	Class $+$ Query ( $q^{+}$ )	Class $-$ Query ( $q^{-}$ )
$xy$ (Class $+$ maps to $x$ )	Prompt: $\mathrm{ICL}^{i}_{xy}(q^{+})$	Prompt: $\mathrm{ICL}^{i}_{xy}(q^{-})$
	Expected Output: $x$	Expected Output: $y$
$yx$ (Class $+$ maps to $y$ )	Prompt: $\mathrm{ICL}^{i}_{yx}(q^{+})$	Prompt: $\mathrm{ICL}^{i}_{yx}(q^{-})$
	Expected Output: $y$	Expected Output: $x$

Appendix

Table 30

model	clean $\to$ sup	clean $\to$ fab	flip?	recruited
	true $D_{TF}$	fake $D_{TF}$	strict	interp. backups
Llama-3-8B	$+3.60\!\to\!$ $+2.40$	$-4.17\!\to\!$ $+0.08$	$\times$	10
Gemma-2-9B	$+4.28\!\to\!$ $+0.41$	$-5.22\!\to\!$ $+1.67$	$\times$	5
Qwen2.5-7B	$+6.69\!\to\!$ $-0.99$	$-12.07\!\to\!$ $+2.28$	✓	6
Mistral-7B-v0.3	$+4.60\!\to\!$ $+4.23$	$-6.99\!\to\!$ $-0.65$	$\times$	3

Appendix

Table 14: The trained receptor-space perturbation as the primary intervention (

\delta=\mathrm{scale}\cdot\eta_{\text{learned}}\cdot v

, scale

=2

, no dose sweep, epoch 31). Per-class verdict

D_{TF}=\mathrm{mean}(\mathrm{logit}[\text{True}])-\mathrm{mean}(\mathrm{logit}[\text{False}])

at the answer token, on perfect_on_pairs . A strict verdict flip requires BOTH sides to cross zero, so the true statement must read False (suppress,

D_{TF}\!<\!0

) and the fake statement must read True (fabricate,

D_{TF}\!>\!0

). Only Qwen2.5-7B inverts both sides, Llama-3-8B and Gemma-2-9B flip the fabricate side only, and Mistral-7B-v0.3 crosses neither side. “recruited” counts interpretable null/anti axes turned into positive contributors (Table 15 ).

Llama-3-8B (clean $D_{TF}$ : T $+3.60$ /F $-4.17$ $\to$ prim T $+2.40$ /F $+0.08$ )
			$E^{0}$ (clean)		$E^{\pi}$ (perturb)		$R=E^{\pi}\!-\!E^{0}$
axis	family	detects	$E^{0}_{T}$	$E^{0}_{F}$	$E^{\pi}_{T}$	$E^{\pi}_{F}$	$R_{T}$	$R_{F}$
L22.h1.svd117	ov svd	true	$-0.004$	$-0.006$	$+0.111^{\scriptscriptstyle 88}$	$+0.177^{\scriptscriptstyle 117}$	$+0.115$	$+0.183$
L19.n3754	mlp neuron	sure	$-0.008$	$-0.022$	$+0.018^{\scriptscriptstyle 16}$	$+0.034^{\scriptscriptstyle 29}$	$+0.027$	$+0.056$
L22.h1.c60	ov neuron	valid	$+0.000$	$-0.001$	$+0.035^{\scriptscriptstyle 30}$	$+0.030^{\scriptscriptstyle 27}$	$+0.035$	$+0.031$
L21.n4049	mlp neuron	wrong	$-0.011$	$-0.011$	$+0.018^{\scriptscriptstyle 16}$	$+0.018^{\scriptscriptstyle 16}$	$+0.029$	$+0.029$

Appendix

Table 15: Interpretable backup detectors recruited from opposing/inert to contributing by the trained receptor-space perturbation (scale

=2

, epoch 31, perfect_on_pairs , with 10 / 5 / 6 / 3 axes for Llama-3-8B/Gemma-2-9B/Qwen2.5-7B/Mistral-7B-v0.3, ranked by recruitment

R

). Each axis is null/anti in the clean model and turns a positive contributor to

D_{TF}

under the primary intervention, shown separately on the two sides as

E_{T}

(suppress side) and

E_{F}

(fabricate side).

E^{0}

is the same quantity with no primary, and

R=E^{\pi}-E^{0}

is the recruitment. Row shade

=

clean class ( anti , null ). Superscripts are paired

t

values, which are inflated because a fixed intervention makes each per-prompt shift near-deterministic, so recruitment is read from the

E^{0}\!\to\!E^{\pi}

sign flip and

R

rather than from

t

		clean	Swap		Perturb		Swap $+$ Perturb
axis	detects	$E^{0}$	T	F	T	F	T	F	all
Llama-3-8B
L19.n3754	sure	anti	✓	✓	✓	✓	✓	✓	$\bigstar$
L21.n4049	wrong	anti	✓	✓	✓	✓	✓	✓	$\bigstar$
L22.h1.c60	valid	null	✓	✓	✓	✓	✓	✓	$\bigstar$
L22.h1.svd116	correct	null	✓	✓	✓	✓	✓	✓	$\bigstar$

Appendix

Table 16: Recruitment is consistent across three distinct interventions. Each row is an interpretable truth/falsity axis that is null/anti in the clean model (

E^{0}\!\leq\!0

). The verdict sides are scored separately, T (true/suppress,

E_{T}

) and F (false/fabricate,

E_{F}

), and ✓ marks that side becoming a contributor (

E^{\pi}\!>\!0

|t|\!\geq\!2

). Swap is an in-distribution contrastive fact

\leftrightarrow

fake exchange, dose-swept

\times 1\!-\!\times 4

( ✓

=

rose monotonically and reliable by

\times 4

). Perturb is an out-of-distribution trained steering vector applied at every layer (scale

2

), and Swap

+

Perturb stacks both in one pass. The clean class is anti (

E^{0}\!<\!0

) or null (

E^{0}\!\approx\!0

), and

\bigstar

marks recruitment on both sides under all three.

Figure 17: The same flagship backups are recruited in the pretrained checkpoint (Llama-3-8B base vs. Llama-3-8B-Instruct, contrastive swap of the single verdict core L18.n11065 , sec_ref=regime ,

n{=}240

). Each panel shows one flagship detector identified as a recruit in the instruct model, plotting net erased margin

E^{\pi}=E^{\pi}_{T}+E^{\pi}_{F}

against swap dose for instruct (solid) and base (dashed). The ring marks the first dose at which that checkpoint’s mean margin turns positive. Every one of the four is recruited in the base checkpoint as well as in instruct , monotonically in dose and with the same ordering. L23.n8972 (‘lie’) reaches

+0.09

in base against

+0.28

in instruct, L23.n9811 (‘truth’)

+0.08

against

+0.13

, and the two weaker axes L19.n2738 and L21.n4049

+0.02

against

+0.08

and

+0.07

. The paired

t

reaches

44

(instruct) and

24

(base) on L23.n8972 , while base’s whole verdict is smaller to begin with (

\kappa_{\mathrm{base}}{=}-2.0

against

-8.4

). The clean-model classification does differ for L19.n2738 and L21.n4049 , which oppose the verdict outright in instruct but are only weakly polarised in base, so instruction tuning sharpened them into opponents without changing whether the ablation recruits them.

Figure 18: Backup recruitment is already present in the pretrained checkpoint, in the same components (Gemma-2-9B base vs. Gemma-2-9b-it, joint contrastive swap of the verdict cores L28.n2046

+

L33.n4294 , sec_ref=regime ,

n{=}535

). Each panel shows one flagship recruited detector, plotting net erased margin

E^{\pi}=E^{\pi}_{T}+E^{\pi}_{F}

against swap dose for instruct (solid) and base (dashed). All four oppose the verdict in both clean models and are recruited in both , at the same dose in three of four panels. L38.n7088 (‘wrong’) moves

-1.02\!\rightarrow\!+3.79

in base against

-2.20\!\rightarrow\!+3.97

in instruct (the curves nearly coincide at

\times 4

), and on L38.n854 and L41.n3789 the base curve finishes above instruct. The paired

t

reaches

120

in base and

262

in instruct. Across the full bench, all

11

detectors that oppose the verdict in both clean models are recruited in both checkpoints by

\times 4

. Base magnitudes are comparable despite its verdict being far smaller (

\kappa_{\mathrm{base}}{=}-2.83

against

-9.81

). Normalised by the margin the ablation destroys, the base backups restore

11

13\%

of the erased verdict against instruct’s

6

9\%

Figure 19: Backup recruitment is already present in the pretrained checkpoint, in the same components (Qwen2.5-7B base vs. Qwen2.5-7B-Instruct, joint contrastive swap of the verdict cores L22.n13149

+

L24.n14758 , sec_ref=regime ,

n{=}470

). One panel per flagship recruited detector, showing its net erased margin

E^{\pi}=E^{\pi}_{T}+E^{\pi}_{F}

against swap dose for the instruct (solid) and base (dashed) checkpoint. Every detector opposes the verdict in both clean models and is recruited in both under ablation, at the same dose in three of four panels. L27.n13033 moves

-4.91\!\rightarrow\!+7.15

in base against

-5.76\!\rightarrow\!+8.74

in instruct, and L27.n4228

-2.95\!\rightarrow\!+5.65

against

-1.71\!\rightarrow\!+5.61

. In that panel the base curve finishes above instruct. The recruitment ordering is identical in the two checkpoints ( L27.n13033

>

L27.n4228

>

L25.n8444

>

L23.n8341 ), including the multilingual L25.n8444 (top suppressed token 假 (fake)). Absolute magnitudes are smaller in base only because its whole verdict is smaller (

\kappa_{\mathrm{base}}{=}-6.16

against

-18.87

). Normalised by the margin the ablation destroys, the base backups restore

22

27\%

of the erased verdict against instruct’s

9

10\%

Figure 20: Backup recruitment is already present in the pretrained checkpoint, in the same components (Mistral-7B-v0.3 base vs. Mistral-7B-Instruct-v0.3, joint contrastive swap of the verdict cores L20.n14286

+

L19.n8228 , sec_ref=regime ,

n{=}423

). All four detectors oppose the verdict in both clean models and are recruited in both . L31.n8773 reaches

+1.65

in base against

+3.00

in instruct, L28.n4079

+0.30

against

+1.78

, L31.n13669

+0.28

against

+1.03

, and the attention-head axis L22.h23.svd2

+0.02

against

+0.08

, which extends the same recruitment one family beyond MLP neurons. Base magnitudes are smaller only because its whole verdict is smaller (

\kappa_{\mathrm{base}}{=}-1.80

against

-12.40

, a

7\times

scale gap). Normalised by the margin the ablation destroys, the base backups restore a near-constant

24

25\%

of the erased verdict at every dose, against instruct’s

8

11\%

. By

\times 4

all six shared clean-anti detectors are recruited in both checkpoints.

Model	Class	$\rho_{\varnothing}$	$g_{\times 1}$	$g_{\times 2}$	$g_{\times 3}$	$g_{\times 4}$	profile in $k$
Llama-3-8B-Instruct	T	0.990	1.023	1.060	1.105	1.157	increasing
	F	0.930	0.992	1.000	1.010	1.032	flat/slight
Gemma-2-9b-it	T	28.99	0.997	0.957	0.934	0.926	decreasing
	F	31.07	1.080	1.103	1.089	1.086	flat
Qwen2.5-7B-Instruct	T	6.09	0.996	0.978	0.941	0.890	decreasing
	F	5.92	1.024	1.033	1.017	0.985	flat

Appendix

Table 17: The final-norm gain across dose.

\rho_{\varnothing}

is the clean answer-position RMS of the pre-unembedding stream, and

g_{k}=\rho_{\varnothing}/\rho_{\pi_{k}}

is the factor the final normalization applies to every fixed write’s direct path in the dosed world. Values above unity amplify, values below unity attenuate. The artifact account requires

g_{k}>1

and increasing in

k

. In the data, the gain decreases with dose on true statements in both joint-core families, is elevated but dose-flat on Gemma’s fake side, and at the on-manifold dose

\times 1

exceeds

1.03

in no other family-class cell.

			$\times 1$		$\times 2$		$\times 3$		$\times 4$
Model	Cls	$D_{\varnothing}$	$D_{\pi}$	$D_{\pi,\mathrm{sec}}$	$D_{\pi}$	$D_{\pi,\mathrm{sec}}$	$D_{\pi}$	$D_{\pi,\mathrm{sec}}$	$D_{\pi}$	$D_{\pi,\mathrm{sec}}$
Llama-3-8B	T	$+3.51$	$+1.44$	$+1.42$	$-0.83$	$-0.89$	$-3.31$	$-3.41$	$-6.00$	$-6.17$
	F	$-4.13$	$-2.08$	$-2.07$	$+0.70$	$+0.74$	$+3.71$	$+3.77$	$+6.60$	$+6.69$
Gemma-2-9b	T	$+4.40$	$+1.55$	$+1.08$	$-0.83$	$-1.90$	$-1.92$	$-3.12$	$-2.46$	$-3.68$
	F	$-5.21$	$-2.80$	$-2.19$	$+1.73$	$+3.95$	$+3.93$	$+6.53$	$+4.90$	$+7.65$
Qwen2.5-7B	T	$+6.39$	$+1.12$	$+1.50$	$-3.95$	$-6.89$	$-7.51$	$-11.50$	$-9.68$	$-13.82$

Appendix

Table 18: Class-resolved verdict margins across dose (

n{=}50

pairs per family, same runs as Table 17 ).

D

is the class-fixed margin

D_{TF}

of main text § 2 , with correct sign

+

on true statements and

-

on fakes.

D_{\pi}

is the margin in the dosed world

\pi_{k}

, and

D_{\pi,\mathrm{sec}}

additionally overrides the emblematic anti-detector with its regime reference (Llama L23.n8972 , Gemma L38.n7088 , Qwen L27.n13033 ), so each column pair regenerates

E^{\pi}

via

E_{T}=D_{\pi}-D_{\pi,\mathrm{sec}}

and

E_{F}=D_{\pi,\mathrm{sec}}-D_{\pi}

Unit (detects)	Cls	$E^{0}$	dose	$R$	$R_{\mathrm{gain}}$	share of $R$
L23.n8972 ( lie )	T	$-0.04$	$\times 1$	$+0.06$	$-0.001$	wrong sign
Llama-3-8B	T	$-0.04$	$\times 4$	$+0.21$	$-0.006$	wrong sign
	F	$-0.06$	$\times 1$	$+0.08$	$+0.000$	$0.6\%$
	F	$-0.06$	$\times 4$	$+0.16$	$-0.002$	wrong sign
L38.n7088 ( wrong )	T	$-1.42$	$\times 1$	$+1.89$	$+0.004$	$0.2\%$
Gemma-2-9b	T	$-1.42$	$\times 4$	$+2.64$	$+0.105$	$4.0\%$

Appendix

Table 19: The maximum recruitment attributable to renormalization. For each emblematic recruit,

E^{0}

is the clean contribution,

R=E^{\pi}-E^{0}

the observed recruitment, and

R_{\mathrm{gain}}=(g_{k}-1)E^{0}

the recruitment predicted by the strict gain null of Eq. 11 . “Wrong sign” marks cells in which the gain predicts recruitment opposite in direction to what is observed.

Intervention	Clamp $v$	Dose $\lambda(v)$	In expectation	Used by
Counterfactual patch	$a_{x^{\prime}}$ (partner)	$-1$	$-1$	Vig et al. [20] , Geiger et al. [45]
Neutral clamp	$m$	$0$	$0$	this paper
Dosed swap	$m-\tfrac{\lambda}{2}\Delta$	chosen	$\lambda$	this paper
Zero ablation	$0$	$2m/\Delta$	unit-set; $0$ iff $m=0$	Michel et al. [46] ; Gong et al. [32]
Mean ablation	$\bar{a}$	$2(m-\bar{a})/\Delta$	$2q-1$ ; $0$ if balanced	Wang et al. [1]
Resample ablation	$a_{c}(x^{\prime})$ , $x^{\prime}\!\sim\!D$	$2(m-a_{c}(x^{\prime}))/\Delta$	$\lambda(\mu_{D})$ , sd $\rho_{D}$	Chan et al. [21] , McGrath et al. [5] , Rushing and Nanda [6]

Appendix

Table 20: Every clamp is a dose. For any

v

, Equation 3 gives its position on the axis and Equation 8 its downstream effect.

q

is the fraction of the presented class in the reference set.

\mu_{D}

and

\sigma_{D}

are the donor mean and standard deviation, and

\rho_{D}=2\sigma_{D}/|\Delta|

. Measured placements are in Figure 6 , and derivations and per-core detail are in Appendix N .

model	ds.	cert.	$\gamma<0$	$\gamma>0$	mlp	ov/svd	inh.	in-window	beyond
			coupling sign		median $R^{2}$			flip err. (doses)
Llama-3-8B-Instruct	20	18	18	0	0.84	0.74	0.67 (6)	0.02 (13)	—
Gemma-2-9b-it	22	17	14	3	0.78	0.72	0.38 (15)	0.05 (2)	0.12 (12)
Qwen2.5-7B-Instruct	16	14	9	5	0.79	0.24	0.40 (13)	0.02 (3)	0.22 (9)
Mistral-7B-Instruct-v0.3	23	19	11	8	0.58	0.45	0.23 (13)	0.03 (3)	0.22 (11)
pooled	81	68	52	16	0.76	0.68	0.39 (47)	0.02 (21)	0.19 (32)

Appendix

Table 21: The coupling law across families. ds. counts bench directions downstream of the certified core(s), and cert. those whose coupling is reliably non-zero (BH-FDR within family,

q{=}0.05

R^{2}

medians are given separately for MLP-neuron and attention-derived (OV channel, singular mode) directions. inh. is the median share of the clean contribution inherited from the core, over the certified directions whose clean value is resolved (

n

in parentheses). flip err. is the median discrepancy between the predicted and the interpolated crossing dose, split by whether the crossing falls inside the on-manifold segment or beyond full inversion.

direction	$\mathrm{own}_{r}$	$\gamma_{r}$	$R^{2}$	inh.	$k^{\star}$	$k_{\mathrm{obs}}$	lin.	$z$
Llama-3-8B-Instruct
L23.n8972	-0.015	-0.032	0.95	0.68	0.74	0.73	$\times 1$	+4.7
L23.n9811	-0.009	-0.012	0.95	0.59	0.85	0.85	$\times 1$	+5.1
L21.n4049	-0.000	-0.011	0.99	0.97	0.52	0.51	$\times 2$	+0.5
L19.n3754	-0.005	-0.009	0.76	0.66	0.76	0.76	$\times 1$	+7.0
L22.h1.svd117	+0.003	-0.007	0.95	–	0.31	0.29	$\times 1$	-0.8

Appendix

Table 22: The coupling law, per direction. For every downstream bench direction with a coupling reliably different from zero (BH-FDR within family,

q{=}0.05

), the table gives the on-manifold fit of Eq. 8 (pooled intercept, as certified in App. K.2 , while the exemplar tables split it), with intercept

\mathrm{own}_{r}

, slope

\gamma_{r}

(negative

=

counterweight , positive

=

relay ), and fit quality

R^{2}

. inh.

=\gamma_{r}/(\mathrm{own}_{r}{+}\gamma_{r})

k^{\star}

is the crossing dose predicted by the fit,

k_{\mathrm{obs}}

the crossing located in the ladder by interpolation, and † marks crossings beyond full inversion. lin. is the largest dose at which the residual stays on the on-manifold line.

z

is the weight-only wiring alignment of § M . Rows shaded grey are hybrids, whose clean class and coupling sign disagree. Directions upstream of or level with the cores are omitted because they cannot read the core write. They are listed in Table 34 .

direction	$\mathrm{own}_{r}$	$\gamma_{r}$	$R^{2}$	inh.	$k^{\star}$	$k_{\mathrm{obs}}$	lin.	$z$
Mistral-7B-Instruct-v0.3
L31.n8773	-0.393	-0.398	0.98	0.50	0.99	0.96	$\times 1$	+18.0
L31.n13669	-1.850	-0.208	0.58	0.10	4.96	1.88 †	$\times 1$	+14.3
L28.n4079	-0.239	-0.158	0.55	0.40	1.25	1.22 †	$\times 1$	+13.4
L24.n4907	+0.095	-0.093	0.90	–	–	–	$\times 1$	+13.7
L22.h20.svd0	+0.302	-0.083	0.44	–	–	–	$\times 1$	+9.1

Appendix

Table 22: The coupling law, per direction (continued).

Figure 21: Affine fits for the six strongest-coupling downstream directions of Llama-3-8B-Instruct. Each panel plots the measured erased margin

E

against the dose regressor

\lambda=1-2k

, separately for the true-prompt side (blue) and the fake-prompt side (orange). Markers are the archived per-dose means with their standard errors, so the six markers per colour correspond to

k=0,\tfrac{1}{2},1,2,3,4

. The shaded band marks the on-manifold window

\lambda\in[-1,+1]

(equivalently

k\leq 1

) that the headline fit uses. Each side’s straight line is fitted on that window and extrapolated across the full range. The two lines have their own slopes, so the panels show per-class fits rather than the shared-slope fit of Eq. 8 , and drawing them separately makes a sign disagreement between the sides visible. The axis label

x=1-2k

\lambda

Figure 22: Affine fits for the six strongest-coupling downstream directions of Gemma-2-9B-it. Axes, colours, shading and fitted line as in Figure 21 (

\lambda=1-2k

). The vertical scale differs by roughly an order of magnitude between panels, since the coupling strengths

|\gamma_{r}|

vary widely within a single model. This spread is why the wiring correlations in the main text are reported on ranks rather than raw values.

Figure 23: Affine fits for the six strongest-coupling downstream directions of Qwen2.5-7B-Instruct. Axes, colours, shading and fitted line as in Figure 21 (

\lambda=1-2k

Figure 24: Affine fits for the six strongest-coupling downstream directions of Mistral-7B-Instruct-v0.3. Axes, colours, shading and fitted line as in Figure 21 (

\lambda=1-2k

Model	Direction	$\mathrm{own}^{T}$	$\mathrm{own}^{F}$	$\Delta\mathrm{own}$	$\gamma_{r}$ (SE)	$R^{2}$	$\gamma_{r}^{\,5\mathrm{pt}}$	max $\|z\|$	% span	track
		intercepts			shared slope			half rungs
Llama-3-8B	L23.n8972	$-0.0111$	$-0.0190$	$+0.0080$	$-0.0319\;(0.0007)$	$0.969$	$-0.0328$	$6.1$	$1.4\%$	$3.3\%$
Gemma-2-9B	L38.n7088	$-0.3318$	$-0.1034$	$-0.2285$	$-0.7272\;(0.0051)$	$0.944$	$-0.7284$	$17.9$	$7.2\%$	$8.7\%$
Qwen2.5-7B	L27.n13033	$-1.7789$	$-1.8004$	$+0.0215$	$-1.1619\;(0.0153)$	$0.973$	$-1.1542$	$5.6$	$2.4\%$	$3.7\%$
Mistral-7B-v0.3	L31.n8773	$-0.4038$	$-0.3815$	$-0.0223$	$-0.3986\;(0.0119)$	$0.980$	$-0.3962$	$2.1$	$2.1\%$	$2.9\%$

Appendix

Table 23: The exemplar fits of main text Figure 4 , and where the post-fit rungs landed. For each exemplar direction the table gives the two per-class intercepts, their difference

\Delta\mathrm{own}_{r}

, the shared slope with its standard error, the weighted

R^{2}

, and the slope refitted with the half rungs included. track is how far the two parallel lines sit from their own class’s fitted points, as a fraction of the exemplar’s full data span. The two half-rung columns give the out-of-sample landing as the largest deviation of the four half-rung means from the line of their own class, in SEs and as a fraction of span. That line was fitted before those rungs existed.

Model	$n$	$r$	$\rho_{\|\gamma\|}$	sign	med. $\|\Delta\gamma\|/\mathrm{SE}$	med. $\|\Delta\gamma\|/\|\gamma\|$	misfit rem.	$R^{2}$	OOS $R^{2}$
		$\gamma$ : 5 rungs vs 3			shift	$+$ quadratic
Llama-3-8B	20	$0.99957$	$0.982$	$1.00$	$0.33$	$0.6\%$	$5\%$	$0.981$	$0.905$
Gemma-2-9B	22	$0.99998$	$0.998$	$1.00$	$0.39$	$0.9\%$	$4\%$	$0.974$	$0.803$
Qwen2.5-7B	16	$0.99999$	$0.997$	$1.00$	$0.51$	$2.5\%$	$9\%$	$0.965$	$0.904$
Mistral-7B-v0.3	23	$0.99973$	$0.999$	$1.00$	$0.26$	$1.3\%$	$3\%$	$0.913$	$0.653$

Appendix

Table 24: The affine window survives doubling its interior sampling. With the two half rungs

\lambda=\pm\tfrac{1}{2}

added (

478

598

pairs each, same population as the ladder), the columns give agreement between

\gamma_{r}

fitted on five rungs and on the original three, the median slope shift, and two Occam columns (median

|\gamma|

change and misfit removed when a quadratic term is added).

R^{2}

is the per-class variance in

E

explained by the affine law over the five rungs, pooled over directions. The last column is a holdout test in which the law’s parameters, fitted on the original three rungs alone, are scored against the two half-rung means the fit never saw (

324

held-back means). The same test run on the pooled single-intercept form the split intercepts replaced gives

0.86

0.59

0.87

and

-0.41

, so out of sample the pooled form is worse than predicting each direction-class’s own mean in Mistral.

Model	Downstream directions	Correction family	Prompt pairs
Llama-3-8B-Instruct	20	20	532
Gemma-2-9B-it	22	22	583
Qwen2.5-7B-Instruct	16	22	478
Mistral-7B-Instruct-v0.3	23	33	598
Total	81	97

Appendix

Table 25: The fitted population. Directions downstream of a core enter the fits, and the correction family is the full bench, which in Qwen and Mistral includes 6 and 10 upstream placement controls. The same prompt pairs are measured at every dose.

Status	Count	$p$ (pooled fit)
Slope indistinguishable from zero	7	$0.37$ – $0.99$
Excluded at the margin of the correction	6	$0.031$ – $0.077$

Appendix

Table 26: The 13 uncertified directions.

	in-sample median $R^{2}$		held-out $R^{2}$
Model	split (reported)	single	split (reported)	single
Llama	0.874	0.755	0.905	0.858
Gemma	0.911	0.599	0.803	0.586
Qwen	0.821	0.665	0.904	0.867
Mistral	0.903	0.499	0.653	$-0.406$

Appendix

Table 27: The split intercept, in sample and out of sample. The left columns give the median per-direction

R^{2}

of the affine law on

\lambda\in\{+1,0,-1\}

under the two intercept forms. The right columns give the variance-weighted

R^{2}

of the frozen three-rung line on the 324 held-back half-rung class means, which the line never saw.

		intercepts		shared slope		half rungs
Model	Direction	$\mathrm{own}^{T}_{r}$	$\mathrm{own}^{F}_{r}$	$\gamma_{r}$ (SE)	$R^{2}$	$\gamma^{\,5\text{pt}}_{r}$	% span
Llama-3-8B	L23.n8972	$-0.011$	$-0.019$	$-0.032\ (0.0007)$	$0.97$	$-0.033$	$1.4$
Gemma-2-9B	L38.n7088	$-0.332$	$-0.103$	$-0.727\ (0.005)$	$0.94$	$-0.728$	$7.2$
Qwen2.5-7B	L27.n13033	$-1.779$	$-1.800$	$-1.162\ (0.015)$	$0.97$	$-1.154$	$2.4$
Mistral-7B-v0.3	L31.n8773	$-0.404$	$-0.382$	$-0.399\ (0.012)$	$0.98$	$-0.396$	$2.1$

Appendix

Table 28: The exemplar fits, and where the held-out rungs landed. Per-class intercepts and shared slope from the three-rung fit with its weighted

R^{2}

, the slope refitted with the half rungs included, and the largest deviation of the four half-rung means from their class’s line, as a fraction of the direction’s data span.

Withheld	points	affine line $\chi^{2}$	bracket interpolation $\chi^{2}$	quadratic $\chi^{2}$
half rungs $\lambda=\pm\tfrac{1}{2}$	324	7501	1929	1188
neutral rung $\lambda=0$	162	3221	530	390

Appendix

Table 29: Three predictors on the same held-out points. The affine line is the frozen fit (three rungs for the half rungs, the four outer rungs for the neutral rung). Bracket interpolation joins the two measured rungs on either side of the withheld one and assumes no law. The quadratic passes exactly through the training rungs.

\chi^{2}

is in units of the per-rung standard error, and lower is better.

Withheld rung	median $\|z\|$	single-intercept $\|z\|$	median % of span	frac. $\|z\|>2$
$+1$	2.97	4.14	17.9	0.62	endpoint
$+\tfrac{1}{2}$	1.39	2.36	9.1	0.36	interior
$0$ (neutral)	1.36	2.19	9.0	0.36	interior
$-\tfrac{1}{2}$	1.64	1.94	8.4	0.41	interior
$-1$	3.04	2.73	16.5	0.59	endpoint

Appendix

Table 30: Leave-one-rung-out. Each row refits on the other four rungs and scores the prediction at the withheld rung, in standard errors of the withheld measurement and as a fraction of the direction’s data span.

Figure 25: Standardised residuals from the on-manifold fit, by dose, for all downstream directions in each model. Each box aggregates the residuals

(E-\hat{E})/\mathrm{SE}

over every downstream direction and both verdict sides at that dose. Whiskers are the usual

1.5\times

IQR range and outliers are suppressed. The vertical axis is symmetric-logarithmic, linear within

\pm 2

SE and logarithmic beyond, so that the tightly clustered residuals at

k\leq 1

and the heavy tails at the stress doses are both resolvable in one frame. Residuals at

k=0,\tfrac{1}{2},1

are flat and centred on zero by construction (those are the three points the fit uses), so the informative content is the abrupt fanning at

k=2,3,4

, which is the same saturation seen per direction in Figures 21 to 24 and is why the headline

\gamma_{r}

is estimated on the

k\leq 1

window rather than the full ladder.

Figure 26: Which heads require a next-order term. The panels show only reachable recruits, the heads above the core’s layer. Adding

\beta_{r}\lambda^{2}

and taking

|\beta_{r}/\gamma_{r}|

divides them cleanly. Nine of ten sit at or below

0.572

and follow the line (a) . One, L11.h10 at

3.430

, sits well above it, a sixfold gap with nothing inside (c) . Its parabola turns inside the window (b) , and its

\gamma_{r}

does not certify, so the quadratic is free to dominate a ratio whose denominator is unresolved. Built from the clean-reference ladder.

Head	Role	$\gamma_{r}$	(SE)	cert.
L10.h7	negative name mover	$-0.8864$	$0.0211$	yes
L11.h2	backup name mover	$-0.1269$	$0.0095$	yes
L10.h2	backup name mover	$-0.0841$	$0.0055$	yes
L11.h6	backup name mover	$-0.0584$	$0.0048$	yes
L10.h0	name mover	$-0.0545$	$0.0120$	yes
L11.h10	negative name mover	$-0.0348$	$0.0253$	–

Appendix

Table 31: The coupling law on the ten reachable IOI heads. Shared slope

\gamma_{r}

with its standard error, fitted on the on-manifold window. Roles are Wang et al.’s. Every certified head is a counterweight, with

\gamma_{r}<0

Figure 27: Coupling of the ten reachable IOI heads.

\gamma_{r}

with

95\%

intervals from Table 31 . Filled markers are significant after Benjamini-Hochberg correction at

q=0.05

. Every point estimate is negative, and L10.h7 , the copy-suppression head, dominates.

Figure 28: The coupling law transfers to GPT-2 small on IOI. (a) Dose response of four heads, solid inside the window

|\lambda|\leq 1

where the law is fitted, dashed outside. (b) The 40 held-out means at

\lambda=\pm\tfrac{1}{2}

against the frozen three-rung line, with the two beyond

2

SE ringed. (c)

|\gamma_{r}|

for heads the core swap can reach (above layer 9) and cannot (at or below it).

Figure 29: L11.h10 ’s exception is a missing core term. Left, under the two-head core the on-manifold response falls and then rises, with

R^{2}=0.413

|\beta_{r}/\gamma_{r}|=2.608

and vertex at

\lambda^{*}=+0.19

. Right, with L10.h7 clamped into the core the response is a line, with

R^{2}=0.972

|\beta_{r}/\gamma_{r}|=0.292

, vertex at

+1.71

and held-out points no longer rejected (

p=0.837

and

0.459

, against

0.070

and

0.041

). The coefficient nearly quadruples, from

-0.0701

-0.2695

. Filled markers are the three fitted doses, and open markers at

\lambda=\pm\tfrac{1}{2}

are held out. The clean effects

E^{0}

are identical between the two ladders.

Figure 30: The coupling is legible in the query weights alone. Alignment between the core’s write direction and each reachable head’s read rows, computed from the checkpoint with no prompts and no forward pass, against the measured

|\gamma_{r}|

. The query channel carries the association (

\rho=+0.818

p=0.0056

, rising to

+0.917

p=0.0012

, with L11.h10 excluded as mediated), while the key and value channels are null (

-0.249

-0.261

). Family-wise

p

across the three channels is

0.0170

Figure 31: Mean ablation of the core, placed on the dose axis. Each prompt pair puts the clamp at its own dose

\lambda(v)=2(m-v)/\Delta

. Both core name movers centre on the neutral dose, with median realised doses of

+0.03

( L9.h6 ) and

-0.00

( L9.h9 ). L9.h9 places

8\%

of pairs outside the on-manifold window (shaded) and L9.h6

40\%

. The realised dose is a ratio, so the median rather than the mean is the reliable summary.

Model	$n$	$\rho(z_{r},\|\gamma_{r}\|)$	one-sided perm. $p$
Llama-3-8B	20	$0.555$	$0.006$
Gemma-2-9B	22	$0.823$	$5.0\times 10^{-5}$ (floor)
Qwen2.5-7B	16	$0.815$	$2.5\times 10^{-4}$
Mistral-7B-v0.3	23	$0.729$	$2.5\times 10^{-4}$

Appendix

Table 32: Per-model single-hop test. Spearman

\rho

between the weights-only alignment

z_{r}

and

|\gamma_{r}|

over downstream directions, with a one-sided within-model permutation

p

(

H_{1}\colon\rho>0

20{,}000

random re-pairings,

p=(k+1)/(N+1)

, floor

5.0\times 10^{-5}

). These four values enter the Fisher combination

\chi^{2}_{8}=63.3

p\approx 1\times 10^{-10}

, an upper bound because Gemma sits at the floor.

		count
Family		bench	down	cert.	$\widetilde{R}^{2}$	$\widetilde{\|\gamma\|}$	cw / relay	$\widetilde{\|\Delta\gamma\|}/\mathrm{SE}$ (sign)
mlp_neuron	MLP neuron	67	54	49	$0.758$	$0.0286$	35 / 14	$0.39$ (49/49)
ov_neuron	OV channel	16	16	10	$0.621$	$0.0023$	10 / 0	$0.29$ (10/10)
ov_svd	OV SVD	11	10	8	$0.698$	$0.0091$	6 / 2	$0.30$ (8/8)
mlp_svd	MLP SVD	3	1	1	$0.680$	$0.0016$	1 / 0	$0.07$ (1/1)
All		97	81	68	sign preserved on all 68 certified directions

Appendix

Table 33: The coupling law resolved by component family. The

68

certified downstream directions span four families beyond MLP neurons. For each family the columns give the number benched, downstream and certified, the median weighted

R^{2}

of the three-rung fit, the median

|\gamma_{r}|

, and the split into counterweights (

\gamma_{r}<0

) and relays (

\gamma_{r}>0

). The last column is the family-agnostic robustness check, giving the median slope shift when the two half rungs are added and the number of certified directions that keep their sign.

direction	family	layer	clean	$z$	$\Delta a$ rel.	$\|R\|$
Qwen2.5-7B-Instruct cores at L22, L24
L22.n4538	mlp neuron	22	anti	$+14.06$	$0.0000$	$0.0063$
L22.n9609	mlp neuron	22	contribute	$+6.55$	$0.0000$	$0.2087$
L20.h1.svd6	ov svd	20	contribute	$+4.19$	$0.0000$	$0.1968$
L22.n13025	mlp neuron	22	anti	$+2.54$	$0.0000$	$0.0381$
L11.svd690	mlp svd	11	null	$-0.10$	$0.0000$	$0.0047$

Appendix

Table 34: Alignment without causal reach. Bench directions that sit at or above the layer of every certified core, so that no core writes strictly before them. That their coefficients do not move under the clamp (

\Delta a

rel.

=0

exactly) is layerwise causality rather than a finding, and verifies the pipeline. The control lies in the alignment column. Two of these directions are among the most strongly wired in the study ( L22.n4538 at

z={+}14.1

and L19.n11056 at

z={+}10.1

, above all but a handful of the downstream directions of Table 22 ), and neither responds. Llama and Gemma contribute no rows, because their benches lie below the cores.

Model	Direction	$z_{r}$	$\|\gamma_{r}\|$	fold	$\|\gamma_{r}\|$	fold	$\Delta a_{r}/\|a_{r}\|$
			three-point fit		full ladder
Qwen2.5-7B	L22.n4538	$+14.06$	$0.00335$	$67.7$	$0.00066$	$285.0$	$0.0000$
Qwen2.5-7B	L22.n9609	$+6.55$	$0.05401$	$0.26$	$0.01732$	$0.73$	$0.0000$
Qwen2.5-7B	L20.h1.svd6	$+4.19$	$0.00397$	$1.5$	$0.01404$	$0.39$	$0.0000$
Qwen2.5-7B	L22.n13025	$+2.54$	$0.00526$	$0.62$	$0.00270$	$1.1$	$0.0000$
Qwen2.5-7B	L11.svd690	$-0.10$	$0.00129$	$0.95$	$0.00017$	$7.0$	$0.0000$

Appendix

Table 35: The placement control, scored against the fitted trend. The same

16

directions as Table 34 , placed against its own model’s downstream trend

\log_{10}|\gamma_{r}|=a+b\,z_{r}

. Fold is how far below that line it falls, so

>\!1

means less coupling than its alignment alone would buy. 13 of 16 fall below the line on the three-point fit the coupling law uses (sign test

p=0.011

) and 13 of 16 on the full-ladder fit. Qwen L22.n9609 is the only direction that clears its trend under both fits. It is a strong verdict unit that sits beside the core rather than after it.

Explore similar work

Jul 2, 2026cs.LG

Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits

Mechanistic interpretability seeks to explain transformer behavior through circuits: sets of internal components that causally support a behavior. However, self-repair creates a blind spot: ablating a primary component can activate a dormant backup, so a circuit that explains behavior in the intact model can become incomplete under the intervention used to test it. We formulate this gap as conditional circuit completion: given a primary set, identify components that become causally important after its removal. We introduce conditional co-ablation (CoAx), which ranks candidates by growth in ablation effect after primary-set removal. We show that a perfectly dormant backup can be indistinguishable from an irrelevant component to per-unit intact-state scores, whereas its conditional effect change exactly aggregates all interaction orders linking it to the removed set. On GPT-2-small's Indirect Object Identification (IOI) circuit, CoAx recovers the documented backup heads at 0.941 ROC-AUC, versus 0.815 for the strongest intact-state attribution baseline and 0.758 for the matched conditional-energy control. Recovery drops to 0.40 +/- 0.13 AUC for alternative component sets matched in behavioral effect, output displacement, and depth, showing that recovery is specific to the removed circuit. Beyond recovery, the CoAx-selected heads are causally load-bearing: freezing them after primary removal sharply reduces the IOI margin, while adding them to the incomplete circuit reduces incompleteness from 0.75 to 0.21. More broadly, conditional growth aligns with intervention-derived repair in 11/12 held-out instances across 4 mechanism clusters, and CoAx completions outperform matched random completions on all 8 non-GPT-2 models spanning 6 architecture families. Together, causal explanations of self-repairing transformers must account for backup circuitry when primary components fail.

Zhiren Gong, He Lu, Tiantong Wang +6

College of Computing and Data Science, Nanyang Technological University, Singapore · Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore

Ablation Transformer Architectures Mechanistic Interpretability Circuits

Ask PDF

Aug 4, 2026cs.CL

Sensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its Scaling

When a language model fails on surface-perturbed input (typos, OCR noise, homophones), "which layer is responsible" has three natural operationalizations: where representations diverge most (sensitivity), where restoring clean activations recovers the prediction (causality), and where a small adapter can repair the damage (compensatory capacity) - and we show these three layer maps dissociate. Across a five-model panel we identify two propagation regimes - spike-and-suppress (Phi-3.5, Gemma-2-9B) and late-accumulation (Llama-3, Mistral, Qwen2.5-7B) - and on the two models meeting an 80% identity-patch gate, sensitivity and causality are anti-correlated (rho = -0.72 to -0.88). Within-family scaling on Qwen2.5 (1.5B to 14B) shows the late-accumulation signature strengthening monotonically with scale, corroborated on a second family. We propose cascade disruption as the mechanism behind the dissociation: adapters placed at causally implicated early layers break intact downstream computation, making diagnostic-flagged sites the worst adapter placements. A fixed-harness layer sweep across four models (3.8-8B) confirms the core prediction on chain-of-thought GSM8K - the flagged sites are the most damaging adapter windows on every adjudicable model - and is sign-consistent but strongly attenuated on a multiple-choice control, consistent with damage that compounds with generation length. The sweep yields practical guidance: a training-free LRD pre-screen and a default-deepest placement rule, though absolute gains over no-adapter baselines remain small. Finally, apparent gains from a representation-stability loss reverse under an adequate generation budget - truncated chain-of-thought had been scored as empty - a methodological warning for any intervention evaluated on chain-of-thought tasks.

Nathan Labiosa, David Buff, Ena Nayak +1

University of Southern California

Large Language Models Fail Layer-Wise Chain-of-Thought Reasoning Dissociation Causal Reasoning Distributional Signatures

Ask PDF

Aug 11, 2026cs.CL

What Iterated Self-Feeding Probes of Language Models Measure, and a test that separates the construction from the model

A growing class of methods probes a language model by feeding it its own output: self-consistency, iterated refinement, agentic loops. We ask what such a probe measures, in a construction chosen to make the question sharp: a ring of token cells resampled in place by the model's own windowed conditional p_r(x_i | x_{i+-r}). The substrate is Glauber dynamics on token sequences and is not new; what we change is the coupling. Advancing two rings that differ in one token under common random numbers makes undamaged copies diverge by exactly zero, so damage spreading becomes measurable where a maximal coupling gives mixing times instead. The answer is that it measures two different things at once, in readings that look alike. Some quantities are fixed by the construction: the damage light cone is kinematic, and the radius scaling of the token-space Lyapunov exponent lambda_ca(r) is model-invariant across 19 models and two scale ladders spanning 70x. Others genuinely track the model: lambda_ca crosses zero at a reproducible point in training, and the attractor share ranks models consistently however the lattice is built. Left undistinguished, the first kind is readily mistaken for the second -- we did so ourselves for four months, and report a phase transition we measured to three decimal places that belongs to the probe rather than to any language model. We give the test that separates them: hold the construction fixed and vary the model, or hold the model fixed and vary the construction, and see which readings move. We validate the instrument by reproduction first, recovering a Domany-Kinzel damage field bit-exactly against an independent prediction, and we report the estimator failures that this discipline caught -- four retracted verdicts, each on a quantity that looked like a measurement. The methodology ships as a package.

Nicolás Vera Zúñiga

Independent Researcher, Chile

Agent Loop Lyapunov Function

Ask PDF