On the Geometric Structure of Layer Updates in Deep Language Models

Jun-Sik Yoo

cs.LGApr 2, 2026

On the Geometric Structure of Layer Updates in Deep Language Models

Authors: Jun-Sik Yoo

Organizations: Institute of Basic Science, Korea University, 145 Anam-ro, Seoul, South Korea 02841.

Abstract

We study the geometric structure of layer updates in deep language models. Rather than analyzing what information is encoded in intermediate representations, we ask how representations change from one layer to the next. We show that layerwise updates admit a decomposition into a dominant tokenwise component and a residual that is not captured by restricted tokenwise function classes. Across multiple architectures, including Transformers and state-space models, we find that the full layer update is almost perfectly aligned with the tokenwise component, while the residual exhibits substantially weaker alignment, larger angular deviation, and significantly lower projection onto the dominant tokenwise subspace. This indicates that the residual is not merely a small correction, but a geometrically distinct component of the transformation. This geometric separation has functional consequences: approximation error under the restricted tokenwise model is strongly associated with output perturbation, with Spearman correlations often exceeding 0.7 and reaching up to 0.95 in larger models. Together, these results suggest that most layerwise updates behave like structured reparameterizations along a dominant direction, while functionally significant computation is concentrated in a geometrically distinct residual component. Our framework provides a simple, architecture-agnostic method for probing the geometric and functional structure of layer updates in modern language models.

Figures & tables

Figure 1: Illustration of the decomposition into a local linear approximation and a residual term. The tokenwise prediction captures the dominant update direction, while the residual reflects deviations from this direction that arise from non-local structure.

Panel of Figure 2 — Figure 2: Residual-output relationship across models and layers. (a) Across architectures, residual error strongly correlates with output deviation. (b) Representative token-level scatter (Pythia-1B). (c) Layer-wise variation of residual–output alignment. (d) residual magnitude varies across layers, revealing structured regimes.

Panel of Figure 3 — Figure 3: Geometric structure of layer updates. The full update is strongly aligned with the tokenwise approximation (left), while the residual exhibits large angular deviation (right), indicating a geometrically distinct component.

Figure 4: Projection onto dominant tokenwise subspace. The full and tokenwise updates lie almost entirely within a low-dimensional subspace, while the residual exhibits significantly lower projection, confirming its geometric separation. Results shown for top-1, top-4, and top-8 singular vectors.

Model	Mean Spearman $\rho$
Pythia-70m	0.906
Pythia-410m	0.837
Pythia-1b	0.804
Pythia-1.4b	0.883
distilgpt2	0.321
Mamba-130m	0.605

Table 1: Mean alignment between approximation error and output perturbation across models.

Regime	RelErr (Lin) $\downarrow$	RelErr (MLP) $\downarrow$	KL (Lin) $\downarrow$	KL (MLP) $\downarrow$	$\rho$ (Lin) $\uparrow$	$\rho$ (MLP) $\uparrow$
Low	0.18	0.26	6.1e-4	1.30e-3	0.80	0.62
Mid	0.43	0.46	2.73e-3	1.88e-3	0.59	0.56
High	0.62	0.57	7.50e-3	2.53e-3	0.06	0.53
Overall	0.41	0.43	3.63e-3	1.91e-3	0.82	0.73

Table 2: Comparison between local linear maps and small MLP baselines across regimes.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Sensitivity to locality and rank in Pythia-70M. Left: neighborhood size sweep. Right: rank sweep.

Representation Geometry in Language Models

Representation geometry in language models studies how embeddings and hidden states organize concepts, computation, and behavior.

See all 404 papers

LLM Layers Immediately Correct Each Other
Sep 7, 2026Arjun Patrawala, Jiahai Feng, Erik Jones +1Transformer Interpretability Representation Geometry in Language Models
University of California, Berkeley
A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases
May 9, 2026Gianfranco Lombardo, Giuseppe Trimigno, Stefano CagnoniNeural Representation Geometry LLM Interpretability
Department of Engineering and Architecture University of Parma Parma, IT 43125
Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology
May 14, 2026Jesseba Fernando, Grigori GuitchountsDynamical Systems Transformer
Network Science Institute, Northeastern University · Flagship Pioneering

Sep 7, 2026J/K move · Enter open · S save

LLM Layers Immediately Correct Each Other

Arjun Patrawala, Jiahai Feng, Erik Jones, Jacob Steinhardt

University of California, Berkeley

Transformer Interpretability Representation Geometry in Language Models

Open paper AskPDF arXiv

Recent methods in language model interpretability employ techniques such as sparse autoencoders to decompose residual stream contributions into linear, semantically meaningful features. Such methods are commonly interpreted as identifying features that persist in the residual stream and that subsequent layers build upon. We challenge this view by identifying the Transformer Layer Correction Mechanism (TLCM), wherein adjacent transformer layers systematically counteract portions of each other's contributions. TLCM appears in 5 out of 7 major open-source model families and activates across nearly all tokens in diverse texts. We show that TLCM emerges during pretraining, operates most strongly on contextually dependent tokens, and adaptively calibrates its correction strength based on the preceding layer's output. Using the layer Jacobian, we further show that TLCM selectively corrects specific subspaces while reinforcing others, which we interpret through a ``propose-and-reject'' framework in which layers propose candidate features and subsequent layers selectively remove inappropriate ones. This dynamic suggests that the residual stream at any layer contains transient proposals alongside persistent features, helping explain why SAE feature descriptions often have low specificity, why effective model steering requires extreme feature amplification, and why transcoders hold a theoretical advantage over SAEs.

Abstract

Representation Geometry in Language Models

Explore similar work

LLM Layers Immediately Correct Each Other

A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases

Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology