Closed Gates Leak: Decomposing Adjacent-Layer Cancellation in Gated Language Models

Harvey Dam; Ganesh Gopalakrishnan

cs.LGOct 28, 2026

Closed Gates Leak: Decomposing Adjacent-Layer Cancellation in Gated Language Models

Authors: Harvey Dam, Ganesh Gopalakrishnan

Organizations: Kahlert School of Computing, University of Utah, Salt Lake City, USA.

Abstract

In transformer language models with gated multilayer perceptrons (MLPs), adjacent layers write partly against each other into the residual stream. Previous work interpreted this cancellation as layers proposing features and the next layer selectively rejecting the inappropriate ones. We decompose the cancellation by removing the previous layer's write from the input of the next MLP: what the next layer writes along that write splits into what it writes anyway and how much less its MLP writes because the write is present, which we call the damping, and we trace the damping to the neurons that carry it. In gated MLPs the damping is carried by neurons whose gates are closed. SiLU and GELU are slightly negative below zero, so a closed gate still passes a small negative multiple of what the neuron reads, and a neuron whose output weight points along its input weight then subtracts a fraction of the direction it read. Summed over such neurons, closed gates contract the average direction a gated MLP reads, and the previous layer's write is not singled out: the same layer's write at another token is damped as much or more, and the eigenvalue signature from which that work concluded that the correction is selective appears for another token's write as well. Over training the damping grows while what the next layer writes anyway stays flat, and cancellation begins when the damping overtakes it; annealing the negative side of the gate activation away in continued pretraining removes the damping and the cancellation. Adjacent-layer cancellation is therefore weak evidence about which features a model rejects: it is produced by a contraction that gated MLPs apply, on average over directions, to whatever they read.

Figures & tables

Panel of Figure 1 — Figure 1: Adjacent layers cancel when the next MLP’s damping of the previous layer’s write exceeds what the next layer writes along that write anyway. (a) Adjacent layers in sixteen models (window means, in units of the previous write’s norm): what the next layer writes along the previous write anyway, $A+C$ (gray), and the damping $D$ (blue); the gap is the next layer’s projection on the previous write, $A+C-D$ , and the pair cancels when the damping is the larger (blue segment). Values with 95% intervals in Table 4 . (b) Over the released checkpoints of OLMo-1B, $A+C$ stays flat while the damping grows past it; in the shaded interval the cosine between adjacent writes, and then $A+C-D$ , turns negative. (c) In the same run the average damping $\bar{D}$ comes first from open neurons whose output weights oppose their input weights and later from closed strengthening neurons (Figure 2 ).

Panel of Figure 2 — Figure 2: Where the damping comes from. (a) The average damping $\bar{D}$ of a random direction (tick) and two of its sources: closed strengthening neurons through their linear input (blue, the lobe term) and open neurons whose output weight opposes their input weight (orange); in models without a gate, active neurons with negative slope and aligned weights take the blue role. (b) The damping $D$ of the previous write split by neuron group (each neuron’s group fixed at the model’s actual input); the parts sum to the total (tick). (c) The local damping along the previous write, along the same layer’s write at another token, and along a random direction, on the same tokens (95% half-widths at most 0.005). Values in Tables 5 and 6 .

Panel of Figure 3 — Figure 3: Continued pretraining of Qwen2.5-0.5B for 249M tokens with the closed side of every gate activation scaled by $\lambda$ , annealed from $1$ to $0$ (solid; $\lambda$ in panel c, right axis) or left at $1$ (dashed), from the same weights and batches; window means on the common documents, means over three seeds with bands spanning the seeds (most are narrower than the line). (a) The damping $D$ of the previous write, $A+C$ , and the cosine between adjacent contributions. (b) $\bar{D}$ and its lobe term. (c) Loss on the common documents. Values in Table 14 ; Llama-3.2-1B in Figure 11 .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: The adjacent-layer cosine on the window, on our approximation of the protocol of Patrawala et al. (2025) (40 WikiText-103 test articles of 2,048 tokens) and on ours (sixteen FineWeb-Edu documents of 192 tokens): (a) raw, with the raw cosine averaged over the middle half of the layers instead of the window; (b) controlled. Models in the order of Figure 1 .

Panel of Figure 5 — Figure 5: The terms of Equation ( 1 ) for adjacent layers over training (window, controlled): (a) OLMo-1B, a SwiGLU model, over its released checkpoints from 2B to 3T tokens; (b) Pythia-160m, which has no gate; (c) TinyLlama-1.1B, a SwiGLU model; (d) OLMoE-1B-7B, the mixture of experts, over its released checkpoints from 41B to 5.1T tokens. The dashed line is the controlled adjacent cosine. TinyLlama’s released checkpoint at 503B tokens has a higher loss on our documents (3.30 nats) than those at 105B and 1T tokens (2.72 and 2.56), and its jump between 2T and 2.5T coincides with a documented data issue (text).

Panel of Figure 6 — Figure 6: Equation ( 1 ) at lags 1–3 for every model (window, controlled, document means), one line per model, coloured by family: (a) the controlled cosine, (b) the damping $D$ , (c) attention $A$ , (d) own co-writing, what the MLP of layer $b+k$ writes along $c_{b}$ with $c_{b}$ and the intermediate layers’ components along it removed from its input.

Panel of Figure 7 — Figure 7: The spectrum of damping across directions: minus the eigenvalues of the symmetric part of the MLP’s Jacobian on the subspace orthogonal to $x^{\prime}$ (four layers of the window, one token per document). (a) The 5th to 95th percentile over directions, the mean (the average damping $\bar{D}$ ), and the mean damping of the lobe part of the Jacobian and of everything else. (b) The fraction of directions damped by more than $0.8$ (nearly removed), damped by less, and amplified (negative damping).

Panel of Figure 8 — Figure 8: Every MLP changed at once, at inference: the closed side removed, $\phi(t)\to\phi(t)\,\mathbf{1}[t>0]$ , or open entries zeroed at random with probability equal to the closed side’s share of each token’s $\sum|\phi|$ (size-matched control). (a) Window damping, (b) window controlled cosine, (c) loss on the sixteen documents. Both changes stop the models working as language models.

Panel of Figure 9 — Figure 9: Per-token test: partial Spearman correlation between the controlled projection of $c_{j}$ on $c_{j-1}$ and each term of MLP $j$ at the same token (controlling for pair, $\log\|x^{\prime}\|$ , $\log$ position, $\log\|c_{j-1}\|$ and $\log\|c_{j}\|$ , and for the lobe term also for punctuation, number class and log unigram frequency), mean over sixteen documents with 95% intervals. A negative value means tokens with more of the term cancel more.

Panel of Figure 10 — Figure 10: Training from scratch without the gate’s negative region. Each row is a 51M-parameter SwiGLU model trained for 1B tokens that differs only in the closed side of the gate activation, $\phi(t)$ for $t\leq 0$ (the open side is SiLU in every row): scaled by $\lambda$ in both the forward pass and the gradient, kept in the forward pass with no gradient through it, or removed from the forward pass with SwiGLU’s gradient (straight-through). Two seeds per row. (a) The average damping $\bar{D}$ (gray) and its lobe term (blue). (b) The fraction of gates closed. (c) The median cosine between the neurons’ input and output weights. (d) Validation loss. Without the closed side ( $\lambda=0$ ) the damping falls to 32% of SwiGLU’s at a loss the seeds cannot distinguish; none of these models cancels. Values in Table 13 .

Panel of Figure 11 — Figure 11: Continued pretraining of Llama-3.2-1B, one seed, with the closed side annealed away (solid; $\lambda$ in panel c, right axis) or left in place (dashed), as in Figure 3 . Values in Table 15 .

Transformer FFNs

Transformer feed-forward networks (FFNs) are the within-token MLP blocks whose activations, structure, efficiency, and role in model computation are studied.

See all 43 papers

LLM Layers Immediately Correct Each Other
Sep 7, 2026Arjun Patrawala, Jiahai Feng, Erik Jones +1Transformer Interpretability Representation Geometry in Language Models
University of California, Berkeley
Layer Collapse in Diffusion Language Models
May 7, 2026Alexander Conzelmann, Albert Catalan-Tatjer, Shiwei LiuLLM Compression Diffusion Language Models
Tübingen AI Center · Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen
Half the Nonlinearity Is Wasted: Measuring and Reallocating the Transformer's MLP Budget
Mar 3, 2026Peter BaloghTransformer FFNs Conditional Computation
Peter Balogh

Sep 7, 2026J/K move · Enter open · S save

LLM Layers Immediately Correct Each Other

Arjun Patrawala, Jiahai Feng, Erik Jones, Jacob Steinhardt

University of California, Berkeley

Transformer Interpretability Representation Geometry in Language Models

Open paper AskPDF arXiv

Recent methods in language model interpretability employ techniques such as sparse autoencoders to decompose residual stream contributions into linear, semantically meaningful features. Such methods are commonly interpreted as identifying features that persist in the residual stream and that subsequent layers build upon. We challenge this view by identifying the Transformer Layer Correction Mechanism (TLCM), wherein adjacent transformer layers systematically counteract portions of each other's contributions. TLCM appears in 5 out of 7 major open-source model families and activates across nearly all tokens in diverse texts. We show that TLCM emerges during pretraining, operates most strongly on contextually dependent tokens, and adaptively calibrates its correction strength based on the preceding layer's output. Using the layer Jacobian, we further show that TLCM selectively corrects specific subspaces while reinforcing others, which we interpret through a ``propose-and-reject'' framework in which layers propose candidate features and subsequent layers selectively remove inappropriate ones. This dynamic suggests that the residual stream at any layer contains transient proposals alongside persistent features, helping explain why SAE feature descriptions often have low specificity, why effective model steering requires extreme feature amplification, and why transcoders hold a theoretical advantage over SAEs.

Abstract

Transformer FFNs

Explore similar work

LLM Layers Immediately Correct Each Other

Layer Collapse in Diffusion Language Models

Half the Nonlinearity Is Wasted: Measuring and Reallocating the Transformer's MLP Budget