Transformer MLP Gate Thresholds Are Couplings to a Carried Reference Direction

Olli Tuomi

cs.CLSep 25, 2026

Transformer MLP Gate Thresholds Are Couplings to a Carried Reference Direction

Authors: Olli Tuomi

Organizations: Evident Solutions Oy

Abstract

The corpus-mean direction of a transformer's residual stream is a component shared across all inputs, and is commonly removed by mean-centering before representational analysis. We present evidence that it is a functional component: the reference against which the MLP gate population sets its operating point. In Phi-2, an exact decomposition of resting gate pre-activations shows that at mid-stack layers the resting inhibition of >99.9% of gates is carried by the coupling $w \cdot b_L$ to the carried mean direction, at 48-56 $\times$ the explicit bias parameter, and the coupling is direction-specific: a random direction at matched norm orders the population's firing rates at Spearman $ρ\leq 0.09$ where the reference reaches 0.95. (That 0.95 is near-tautological on its own; the paper derives its null.) Causally, removing the stream's projection on the reference multiplies above-threshold firing by about 9 $\times$ , dose-monotonically, at 23-59 $\times$ a norm-matched control; a random-initialised twin is flat, and replacement tests show that the direction carries the function and the magnitude does not. A direction-matched control makes the same point: noise injected along the reference costs 8-42 $\times$ the same energy along a random direction. The decomposition replicates on four further families spanning both gate types (GELU with an explicit gate bias, bias-free SwiGLU), and the dose-response on two of them. Across an eight-model scan the mechanism is present in every GELU and SiLU family and absent only in OPT, where an opposing LayerNorm bias cancels the carried reference. Gate thresholds are implemented as couplings to a constant the network builds for the distribution it is reading, with the bias parameters contributing little; how much of that constant is carried in the stream and how much in parameters depends on the architecture.

Figures & tables

$\alpha$	$\Delta$ frac(act $>$ 0.5)			KL vs. clean
	L6	L10	L14	L6	L10	L14
0.0 (removed)	$+0.103$	$+0.086$	$+0.075$	1.427	0.519	0.274
0.5	$+0.023$	$+0.023$	$+0.021$	0.047	0.040	0.035
1.0 (identity)	0.000	0.000	0.000	0.000	0.000	0.000
1.5	$-0.006$	$-0.007$	$-0.007$	0.014	0.018	0.020
2.0 (doubled)	$-0.008$	$-0.009$	$-0.009$	0.055	0.060	0.110

Table 1: Dose-response of gate firing and output damage to scaling the stream’s projection on

\hat{m}_{L}

at the manipulated layer (Phi-2, trained). The twin is flat at every

\alpha

(

|\Delta\text{frac}|\leq 0.003

, no ordering). Point estimates are document medians over 39 documents; 95% document-bootstrap CIs (2000 resamples) on the removal row:

\Delta

frac

[+0.096,+0.112]/[+0.079,+0.098]/[+0.068,+0.088]

and KL

[1.14,2.03]/[0.36,0.75]/[0.15,0.40]

(L6/L10/L14), each disjoint from the norm-matched control’s KL CI (

\leq[0.010,0.032]

). The document-sampling spread on KL dwarfs the run-to-run fp16 jitter (

\sim

1%), so only the leading two digits are informative.

replacement $\hat{c}$	$\cos(\hat{c},\hat{m}_{10})$	$\Delta$ frac(act $>$ 0.5)	KL
none (pure removal)	—	$+0.0845$	0.516
coordinate-shuffled $\hat{m}_{10}$	$-0.003$	$+0.0840$	0.950
random direction	$+0.011$	$+0.0778$	0.484
function-word $W_{U}$ sum	$+0.096$	$+0.0740$	0.395
layer 18’s $\hat{m}_{18}$	$+0.940$	$+0.0027$	0.004
identity	1.000	0.000	0.000

Table 2: Replacement panel at L10 (Phi-2, 23 documents). Preservation of the baseline firing level tracks

\cos(\hat{c},\hat{m}_{10})

monotonically.

layer	$w\cdot b_{L}$ (median)	$b_{n}$ (median)	resting	ref-dominant	$\rho$ (resting, duty)	$\rho$ (random dir)
6	$-1.535$	$-0.027$	$-1.568$	99.98%	0.943	0.058
10	$-1.226$	$-0.022$	$-1.250$	99.99%	0.959	0.072
14	$-1.091$	$-0.021$	$-1.115$	99.96%	0.960	0.085
22	$-1.130$	$-0.018$	$-1.149$	99.94%	0.954	0.073

Table 3: Exact resting decomposition of gate pre-activations (Phi-2, 19 documents). “Ref-dominant” is the fraction of gates whose resting inhibition is carried mostly by

w\cdot b_{L}

(

|w\cdot b_{L}|>|b_{n}|

). The last column repeats the rank correlation with a random direction substituted for

b_{L}

at matched norm (median of 50 draws,

p_{95}\leq 0.192

, maximum

0.223

; from a 39-document run, which reproduces the

\rho

(resting, duty) column at

0.946

–

0.964

). The

\rho

(resting, duty) column is near-tautological read alone; it is the contrast between the last two columns that carries direction specificity. 95% document-bootstrap CIs (2000 resamples): ref-dominant stays

\geq 99.89\%

at every layer (widest

[99.89,99.98]

at L22), and

\rho

[0.936,0.945]/[0.951,0.959]/[0.953,0.959]/[0.945,0.952]

(L6/L10/L14/L22; percentile intervals, and resampling biases

\rho

slightly low, so a point estimate can sit just above its interval). The

48

–

56\times

quoted in the text is the ratio of median magnitudes,

|w\cdot b_{L}|

against

|b_{n}|

(the latter

0.027/0.023/0.023/0.021

), which differ slightly from the signed medians shown. The resting column is a per-gate median and does not equal the sum of the

w\cdot b_{L}

and

b_{n}

medians, since a median does not distribute over addition across a heterogeneous gate population.

model	gate (bias?)	resting med.	ref-dom.	$\rho$ (rest, duty)
GPT-2-medium	c_fc (yes)	$-1.52$ to $-0.99$	$99.4$ – $99.7\%$	$0.91$ – $0.97$
Pythia-1.4B	h_to_4h (yes)	$-1.08$ to $-0.74$	$>99.9\%$	$0.96$ – $0.98$
Qwen2.5-1.5B	gate_proj (no)	$-4.14$ to $-0.37$	—	$0.62$ – $0.99$
TinyLlama-1.1B	gate_proj (no)	$-0.21$ to $-0.07$	—	$0.97$ – $0.99$

Table 4: Cross-model resting decomposition (30 documents, four mid-stack layers per model at depth fractions

0.2

–

0.65

; ranges span those layers). “Ref-dom.” is the fraction of gates whose resting inhibition is carried mostly by

w\cdot b_{L}

rather than the bias parameter (vacuous for the bias-free SwiGLU gates; for these the informative statistic is the negative resting coupling and its duty-prediction). Where a bias exists,

|w\cdot b_{L}|/|b_{n}|

15

–

18\times

(GPT-2) and

38

–

43\times

(Pythia). Every model’s random-init twin is floored:

|\text{resting median}|\leq 0.01

, duty

\approx 0.5

Transformer Interpretability

Transformer interpretability studies how attention, feed-forward layers, and hidden representations support model predictions and behavior.

See all 339 papers

Jun 17, 2026

Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale

Pretrained transformers sit near singular minima of the loss, where the Fisher information metric degenerates along dead directions: directions in parameter space along which the directional Fisher vanishes. Locating such a direction normally needs a forward pass and an eigendecomposition of activations, or a sampling-based complexity estimate; none returns a direction computable from the network's parameters alone. We give one, for LayerNorm transformers. The inverse-scale direction

γ^{-1}/\|γ^{-1}\|

of the LayerNorm affine is an exact algebraic kernel of the post-final-norm centred activation covariance, for any input distribution, and induces a corresponding dead direction in parameter space. It is read from the LN scale parameter alone, with no forward or backward pass and no eigensolve: the cheapest dead-direction read, specific to LayerNorm. We test it on

14

pretrained transformers (

9

LayerNorm,

5

RMSNorm;

160

M-

35

B; language and vision objectives). At random initialisation the predicted direction matches the measured bottom singular direction (one forward pass, direct SVD) to four decimal places on

9/9

LayerNorm models, and is correctly absent on

5/5

RMSNorm models, which lack the mean-subtraction projector that creates it. On the trained checkpoint the covariance eigenvalue along this direction deepens by

{\sim}10^3\times

and further dead directions open; the random-init-to-trained gap is a one-forward-pass, per-checkpoint readout of singular structure along the predicted coordinate. Two consequences follow in closed form: the residual stream's smallest singular value is preserved block-to-block on

13/14

transformers measured on their own input distribution, the one exception (Gemma

4

31

B) a genuine dead direction the same read pinpoints; and the kernel direction's presence classifies a transformer's normalisation from the parameters alone.

Tejas Pradeep Shirodkar, P. J. Narayanan

IIIT, Hyderabad

Information Geometry Transformer Neural Network Optimization Layer Normalization

Ask PDF

Mar 3, 2026

Half the Nonlinearity Is Wasted: Measuring and Reallocating the Transformer's MLP Budget

We investigate when transformer MLP nonlinearity is actually necessary. A gate with

d+1

parameters decides when to replace the full MLP with a linear surrogate. Through systematic investigation across six models (162M-2.8B parameters), two architectures, and three corpora, we establish that nonlinearity need cannot be predicted from token identity: cross-corpus correlation is zero (

r < 0.05

). The routing decision is fully contextual. Despite weak per-instance predictability, the gate exploits a heavily skewed distribution where most MLP computations are near-linear, achieving 25-56% linear routing at <1% perplexity cost in GPT-2. In GPT-2 Large, 11 of 36 layers beat baseline with gating and no layer exceeds 3.7% all-linear cost. This success is architecture-dependent: Pythia models show higher costs, though Pythia-2.8B's full 32-layer sweep reveals one layer that narrowly beats baseline. As a proof of concept, we progressively replace middle-layer MLPs with frozen linear matrices: 5 of 24 layers linearize at zero cost. With a full training budget, 4 linearized layers yield a 10.2% perplexity improvement -- and a two-phase gated approach pushes this to 17.3%, beating a vanilla fine-tuning control and confirming that the nonlinear MLPs at these layers were actively harmful.

Peter Balogh

Transformer FFNs Conditional Computation

Ask PDF

Oct 21, 2026

Reader--Writer Coupling in Transformer MLPs

A transformer MLP gives each hidden channel two roles: a reader that decides when the channel responds, and a writer that decides what it adds back to the residual stream. Read as a key--value memory, the layer can be measured one factor set at a time: the pairing of readers to writers is carried only by their shared index, invisible to any quantity computed from one factor set alone. Yet training couples the two factors through their joint update, so we ask what reading them together shows. The block supplies the joint instrument: the MLP writes back into the very stream it reads, so the exact Jacobian of its update is square, and its agreement with its own transpose resolves into reader--writer terms whose inner products give each channel, read as a couple, a signed pairing coordinate. Ranked by this coordinate, each block we measure has a small core of strongly coupled channels; where initialization gives either sign equally, the core is nearly all opposed: each writer points against the direction its reader responds to. This opposed core recurs from the smallest models trained from scratch to released checkpoints; its sign is in place within a few percent of training and holds while the coupling grows. Read as couples, most channels return their contribution against the reader that fired it, and the readers of later blocks take it up; few show in the logits. Training holds the couples in place: flipping the opposed writers moves the output toward common tokens, and a writer permuted onto another reader regrows to fit that reader while the reader stays.

Giyeong Oh, Sunghee Ahn, Youngjae Yu

Seoul National University

Transformer FFNs Transformer Interpretability Transformer Training Dynamics

Ask PDF

Abstract

Transformer Interpretability

Explore similar work

Algebraic Dead Directions in LayerNorm Transformers: A Forward-Pass-Only Diagnostic at LLM Scale

Half the Nonlinearity Is Wasted: Measuring and Reallocating the Transformer's MLP Budget

Reader--Writer Coupling in Transformer MLPs