In transformer language models with gated multilayer perceptrons (MLPs), adjacent layers write partly against each other into the residual stream. Previous work interpreted this cancellation as layers proposing features and the next layer selectively rejecting the inappropriate ones. We decompose the cancellation by removing the previous layer's write from the input of the next MLP: what the next layer writes along that write splits into what it writes anyway and how much less its MLP writes because the write is present, which we call the damping, and we trace the damping to the neurons that carry it. In gated MLPs the damping is carried by neurons whose gates are closed. SiLU and GELU are slightly negative below zero, so a closed gate still passes a small negative multiple of what the neuron reads, and a neuron whose output weight points along its input weight then subtracts a fraction of the direction it read. Summed over such neurons, closed gates contract the average direction a gated MLP reads, and the previous layer's write is not singled out: the same layer's write at another token is damped as much or more, and the eigenvalue signature from which that work concluded that the correction is selective appears for another token's write as well. Over training the damping grows while what the next layer writes anyway stays flat, and cancellation begins when the damping overtakes it; annealing the negative side of the gate activation away in continued pretraining removes the damping and the cancellation. Adjacent-layer cancellation is therefore weak evidence about which features a model rejects: it is produced by a contraction that gated MLPs apply, on average over directions, to whatever they read.
Figures & tables
Figure 1: Adjacent layers cancel when the next MLP’s damping of the previous layer’s write exceeds what the next layer writes along that write anyway. (a) Adjacent layers in sixteen models (window means, in units of the previous write’s norm): what the next layer writes along the previous write anyway, A+C (gray), and the damping D (blue); the gap is the next layer’s projection on the previous write, A+C−D , and the pair cancels when the damping is the larger (blue segment). Values with 95% intervals in Table 4 . (b) Over the released checkpoints of OLMo-1B, A+C stays flat while the damping grows past it; in the shaded interval the cosine between adjacent writes, and then A+C−D , turns negative. (c) In the same run the average damping Dˉ comes first from open neurons whose output weights oppose their input weights and later from closed strengthening neurons (Figure 2 ).
Figure 2: Where the damping comes from. (a) The average damping Dˉ of a random direction (tick) and two of its sources: closed strengthening neurons through their linear input (blue, the lobe term) and open neurons whose output weight opposes their input weight (orange); in models without a gate, active neurons with negative slope and aligned weights take the blue role. (b) The damping D of the previous write split by neuron group (each neuron’s group fixed at the model’s actual input); the parts sum to the total (tick). (c) The local damping along the previous write, along the same layer’s write at another token, and along a random direction, on the same tokens (95% half-widths at most 0.005). Values in Tables 5 and 6 .
Figure 3: Continued pretraining of Qwen2.5-0.5B for 249M tokens with the closed side of every gate activation scaled by λ , annealed from 1 to 0 (solid; λ in panel c, right axis) or left at 1 (dashed), from the same weights and batches; window means on the common documents, means over three seeds with bands spanning the seeds (most are narrower than the line). (a) The damping D of the previous write, A+C , and the cosine between adjacent contributions. (b) Dˉ and its lobe term. (c) Loss on the common documents. Values in Table 14 ; Llama-3.2-1B in Figure 11 .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: The adjacent-layer cosine on the window, on our approximation of the protocol of Patrawala et al. (2025) (40 WikiText-103 test articles of 2,048 tokens) and on ours (sixteen FineWeb-Edu documents of 192 tokens): (a) raw, with the raw cosine averaged over the middle half of the layers instead of the window; (b) controlled. Models in the order of Figure 1 .
Figure 5: The terms of Equation ( 1 ) for adjacent layers over training (window, controlled): (a) OLMo-1B, a SwiGLU model, over its released checkpoints from 2B to 3T tokens; (b) Pythia-160m, which has no gate; (c) TinyLlama-1.1B, a SwiGLU model; (d) OLMoE-1B-7B, the mixture of experts, over its released checkpoints from 41B to 5.1T tokens. The dashed line is the controlled adjacent cosine. TinyLlama’s released checkpoint at 503B tokens has a higher loss on our documents (3.30 nats) than those at 105B and 1T tokens (2.72 and 2.56), and its jump between 2T and 2.5T coincides with a documented data issue (text).
Figure 6: Equation ( 1 ) at lags 1–3 for every model (window, controlled, document means), one line per model, coloured by family: (a) the controlled cosine, (b) the damping D , (c) attention A , (d) own co-writing, what the MLP of layer b+k writes along cb with cb and the intermediate layers’ components along it removed from its input.
Figure 7: The spectrum of damping across directions: minus the eigenvalues of the symmetric part of the MLP’s Jacobian on the subspace orthogonal to x′ (four layers of the window, one token per document). (a) The 5th to 95th percentile over directions, the mean (the average damping Dˉ ), and the mean damping of the lobe part of the Jacobian and of everything else. (b) The fraction of directions damped by more than 0.8 (nearly removed), damped by less, and amplified (negative damping).
Figure 8: Every MLP changed at once, at inference: the closed side removed, ϕ(t)→ϕ(t)1[t>0] , or open entries zeroed at random with probability equal to the closed side’s share of each token’s ∑∣ϕ∣ (size-matched control). (a) Window damping, (b) window controlled cosine, (c) loss on the sixteen documents. Both changes stop the models working as language models.
Figure 9: Per-token test: partial Spearman correlation between the controlled projection of cj on cj−1 and each term of MLP j at the same token (controlling for pair, log∥x′∥ , log position, log∥cj−1∥ and log∥cj∥ , and for the lobe term also for punctuation, number class and log unigram frequency), mean over sixteen documents with 95% intervals. A negative value means tokens with more of the term cancel more.
Figure 10: Training from scratch without the gate’s negative region. Each row is a 51M-parameter SwiGLU model trained for 1B tokens that differs only in the closed side of the gate activation, ϕ(t) for t≤0 (the open side is SiLU in every row): scaled by λ in both the forward pass and the gradient, kept in the forward pass with no gradient through it, or removed from the forward pass with SwiGLU’s gradient (straight-through). Two seeds per row. (a) The average damping Dˉ (gray) and its lobe term (blue). (b) The fraction of gates closed. (c) The median cosine between the neurons’ input and output weights. (d) Validation loss. Without the closed side ( λ=0 ) the damping falls to 32% of SwiGLU’s at a loss the seeds cannot distinguish; none of these models cancels. Values in Table 13 .
Figure 11: Continued pretraining of Llama-3.2-1B, one seed, with the closed side annealed away (solid; λ in panel c, right axis) or left in place (dashed), as in Figure 3 . Values in Table 15 .
Transformer feed-forward networks (FFNs) are the within-token MLP blocks whose activations, structure, efficiency, and role in model computation are studied.
Recent methods in language model interpretability employ techniques such as sparse autoencoders to decompose residual stream contributions into linear, semantically meaningful features. Such methods are commonly interpreted as identifying features that persist in the residual stream and that subsequent layers build upon. We challenge this view by identifying the Transformer Layer Correction Mechanism (TLCM), wherein adjacent transformer layers systematically counteract portions of each other's contributions. TLCM appears in 5 out of 7 major open-source model families and activates across nearly all tokens in diverse texts. We show that TLCM emerges during pretraining, operates most strongly on contextually dependent tokens, and adaptively calibrates its correction strength based on the preceding layer's output. Using the layer Jacobian, we further show that TLCM selectively corrects specific subspaces while reinforcing others, which we interpret through a ``propose-and-reject'' framework in which layers propose candidate features and subsequent layers selectively remove inappropriate ones. This dynamic suggests that the residual stream at any layer contains transient proposals alongside persistent features, helping explain why SAE feature descriptions often have low specificity, why effective model steering requires extreme feature amplification, and why transcoders hold a theoretical advantage over SAEs.
Diffusion language models (DLMs) have recently emerged as competitive alternatives to autoregressive (AR) language models, yet differences in their activation dynamics remain poorly understood. We characterize these dynamics in LLaDA-8B and identify a striking layer-collapse property: a few early layers exhibit highly similar, collapsed activation patterns dominated by a single large super-outlier persisting over a long token range. Despite its apparent redundancy, this outlier is critical: pruning it causes outputs to degrade into repetitive random token loops. Paradoxically, layers in LLaDA contain more redundant representations overall, with redundancy most pronounced in earlier layers -- the reverse of AR models, where deeper layers grow redundant due to undertraining. Our analysis indicates that layer collapse in DLMs is not driven by undertraining but by overtraining: a dominant outlier becomes an indispensable information carrier while remaining representations collapse into redundant structure. These findings have strong practical implications, verified through controlled pre-training experiments. DLMs are surprisingly robust to compression: LLaDA under 3-bit GPTQ quantization drops only -1.8% on GSM8K, whereas Llama-3.1-8B drops -64.7%. Optimal sparsity allocation also reverses between families: at 50% average sparsity, allocating more to early layers in LLaDA yields +8.4% over the reverse strategy, while the same allocation costs Llama -8.4%. Our findings reveal that the DLM training objective fundamentally reshapes layer dynamics relative to AR models, with direct consequences for compression and deployment. Code: github.com/Conzel/super-outlier-dlm.
Alexander Conzelmann, Albert Catalan-Tatjer, Shiwei Liu
Tübingen AI Center · Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen
We investigate when transformer MLP nonlinearity is actually necessary. A gate with d+1 parameters decides when to replace the full MLP with a linear surrogate. Through systematic investigation across six models (162M-2.8B parameters), two architectures, and three corpora, we establish that nonlinearity need cannot be predicted from token identity: cross-corpus correlation is zero (r<0.05). The routing decision is fully contextual. Despite weak per-instance predictability, the gate exploits a heavily skewed distribution where most MLP computations are near-linear, achieving 25-56% linear routing at <1% perplexity cost in GPT-2. In GPT-2 Large, 11 of 36 layers beat baseline with gating and no layer exceeds 3.7% all-linear cost. This success is architecture-dependent: Pythia models show higher costs, though Pythia-2.8B's full 32-layer sweep reveals one layer that narrowly beats baseline. As a proof of concept, we progressively replace middle-layer MLPs with frozen linear matrices: 5 of 24 layers linearize at zero cost. With a full training budget, 4 linearized layers yield a 10.2% perplexity improvement -- and a two-phase gated approach pushes this to 17.3%, beating a vanilla fine-tuning control and confirming that the nonlinear MLPs at these layers were actively harmful.