A Mathematical Theory of the Jacobian Lens

Mitchell Ostrow; Daoyuan Qian; Toni Liu; Ila Fiete

cs.LGOct 22, 2026

A Mathematical Theory of the Jacobian Lens

Authors: Mitchell Ostrow, Daoyuan Qian, Toni Liu, Ila Fiete

Organizations: Department of Brain and Cognitive Sciences Massachusetts Institute of Technology · Department of Physics Cornell University

Abstract

Recent work from Anthropic introduced a new interpretability tool, called the Jacobian lens (J-lens, Gurnee et al., 2026). This method identified linear directions in the hidden states of frontier language models that had properties reminiscent of working memory-like representations. However, it is unclear what mechanistic properties of such features make them detectable to the J-lens. To this end, we introduce a mathematical theory of the J-lens, and show that it preferentially identifies coherent features -- directions that act like leading, spatially coherent Lyapunov modes. Small perturbations along these directions amplify across layers, remain roughly constant across token positions, and are consistent across different contexts. While we are agnostic to whether these properties are necessarily associated with a ``Global Workspace'', the analytically derived preferences of the J-lens nevertheless explain prior empirical applied to both model classes. We validate our analysis on a Bayes filter and a toy transformer, each optimally predicting next tokens emitted from a Hidden Markov Model (HMM). We show how modulating task parameters that change the coherence of features in the Bayes filter correspondingly and predictively modulates the response of the J-lens . We also introduce of a family of J-lenses built to emphasize different temporal scales of information processing in LLMs, which we call $ω$ -J-lenses.

Figures & tables

Figure 1: Low-dimensional projection (principal component analysis, PCA) of the top 100 singular values for individual (black) and mean (red) Jacobians of four different LLMs (Appendix B ). Each Jacobian is evaluated with input at an intermediate layer chosen to be halfway through the model and output at the final layer. Prompts are sampled from a range of datasets with different topics (Wikipedia, Python code, math, news, etc.). Jacobian spectra are plotted in Fig. 9 .

Figure 2: Schematic of the three coherence parameters. (left) Schematic of large

\chi>0

versus small

\chi<0

magnitude directions. A perturbation grows across layers in the red direction, but shrinks in the blue direction. (middle) Schematic of lag-coherent (black) versus lag-incoherent (gray) feature directions across token positions. (right) Context-coherent directions evolve similarly across layers in contexts, while context-incoherent directions evolve differently.

Figure 3: Graphical models of the task (a) and the Bayes filter (b) . (a) Three latents

X^{(i)}_{t}

evolve independently on their rings. The reveal variable

B_{t}

sets whether the token

Y_{t}

shows the latents, and in the context sweep the key

W_{t-1}

sets whether the shown state is flipped. (b) The belief

m_{t}

is the previous belief

m_{t-1}

propagated by

\alpha

, or, when

B_{t}=1

, the revealed state from

Y_{t}

propagated by

\alpha

Figure 4: The J-lens captures the selective modulation of magnitude

a

in the Bayes filter and transformers. (a) Cartoon of the task feature that is modulated: a state

X_{t-1}

on the ring is kept with probability

\lambda

(persistence), and sampled uniformly otherwise. (b) Learning curves for the transformer, in comparison to the optimal Bayes filter. (c) J-lens factors of each latent as a function of persistence

\lambda

: lens weight (the singular value

a\,\rho_{\tau}\,\rho_{c}

) and magnitude

a

, both relative to their largest value, then lag coherence

\rho_{\tau}

and context coherence

\rho_{c}

. Open circles: Bayes filter. Blue circles and orange squares: transformer layers 0 and 1. Large markers show the mean across seeds and small markers show single seeds. Red line: theory (Eq. 19 ).

Figure 5: The J-lens captures the selective modulation of lag coherence

\rho_{\tau}

in the Bayes filter and transformers. (a) The velocity

v

of the state along the ring, with angular velocity

\theta=2\pi v/n

, modulates lag coherence

\rho_{\tau}

. (b) Learning curves of the transformer, as in Fig. 4 . (c) J-lens factors as a function of angular velocity

\theta

, as in Fig. 4 .

Figure 6: The

\omega

-J-lens selectively upweights latent directions with lag coherences aligned with

\omega

. Solid line indicates the theoretical curves (optimally rescaled from the Bayes filter theory to fit the transformers), points indicate real sampled data. Colors (denoting latent index) are determined by alignment with latent encoding direction.

Figure 7: The J-lens captures the selective modulation of context coherence

\rho_{c}

in the Bayes filter and transformers. (a) For keyed features, the true state is revealed in the token with probability

(1+\psi)/2

. Otherwise, the state is revealed inverted, so that

\rho_{c}=\psi

. (b) Learning curves of the transformer, as in Fig. 4 . (c) J-lens factors as a function of key bias

\psi

, as in Fig. 4 .

Figure 8: Predictive theory of crossing of latents in the J-lens. (a) Empirical singular values

\sigma

(circles) and analytically derived values in the large-

H

limit (line) for the J-lens applied to a Bayes filter with two latents (L0, L1).

\lambda_{0}

is swept for L0, while

\lambda_{1}

is held fixed for L1. Vertical dotted line denotes the predicted crossing point

\lambda_{0}^{*}

. (b) Predicted crossing point

\lambda_{0}^{*}

(Eq. ( 46 )) and empirical crossing point (circles) as a function of the angular velocity

\theta_{1}

of L1. (c) Difference in singular values

\sigma_{0}-\sigma_{1}

as a function of

\lambda_{0}-\lambda_{0}^{*}

, for the transformer trained on the same HMM as in (a). Vertical solid line denotes the mean crossing point across seeds, gray shading denotes standard deviation. Each curve is averaged over

4

variations of

\theta_{1}

(circles in (b)).

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Jacobian singular value spectra of four LLMs, with estimated power laws. Individual Jacobian curve is averaged over all points in Fig. 1 , with shading denoting standard deviation.

Figure 10: Low-dimensional projection of the left and right singular vectors of Jacobians. The top 100 left and right singular vectors, ranked by singular values as in Fig. 1 , are projected into two dimensions by computing the subspace distance on Grassmannians

||P_{i}-P_{j}||_{F}=\sqrt{2\sum_{k}\sin^{2}\theta_{k}}

, where

\theta_{k}

are the principal subspace angles and

P_{i}

is the projector matrix defined by the singular vectors,

P_{i}=U_{i}U_{i}^{T}

. This pairwise distance matrix is reduced to two dimensions and visualized with PCA. (a) Left singular vectors. (b) Right singular vectors.

Figure 11: Validation of the J-lens theory for the Bayes filter. X-axis denotes the value predicted by the theory (Eq. ( 19 )), while the y-axis denotes the value measured from the lens, for magnitude $a$ , lag coherence $\rho_{\tau}$ and context coherence $\rho_{c}$ (columns) in each sweep (rows, colored by $\lambda$ , $\theta$ or $\psi$ ). There is only one visible data point for the fixed parameters in each sweep because the HMM is designed to fix these metrics in the Bayes filter.

Figure 12: Validation of the J-lens theory rank prediction for the Bayes filter and the transformer. X-axis denotes the predicted rank order of the singular vectors corresponding to each particular latent (colored by their task parameter $\lambda$ , $\theta$ or $\psi$ ), while the y-axis denotes the measured rank in the J-lens.

Figure 13: Alignment of the J-lens directions with the encoding directions in the magnitude sweep. Alignment $O_{i}$ of the J-lens singular vectors (solid: input side $\bm{v}_{i}$ , hatched: output side $\bm{u}_{i}$ ), ranked by singular value, with the encoding directions $\bm{E}$ of each latent (colored by persistence $\lambda$ ) and with other patterns of the HMM (gray). Columns show the Bayes filter belief state (left), transformer layer 0 (middle) and layer 1 (right). Dashed line: alignment of a random direction.

Figure 14: Alignment of the J-lens vectors with the decoding vectors for each latent in the magnitude sweep. (a) Test decoding accuracy (

R^{2}

) for the Bayes filter belief state

m_{t}

on each layer of the transformer, for each latent in the sweep. (b, c) Alignment

O_{i}

of the J-lens directions

\bm{v}_{i}

with each latent’s own decoding directions

\bm{D}_{i}

(colored) and with the other latents’ decoding directions (gray), for transformer layer 0 (b) and layer 1 (c).

Figure 15: Alignment of the J-lens directions with the encoding directions in the lag coherence sweep. Latents are colored by angular velocity $\theta$ . Panels as in Fig. 13 .

Figure 16: Alignment of the J-lens vectors with the decoding vectors for each latent in the lag coherence sweep. (a) Test decoding accuracy (

R^{2}

) for the Bayes filter belief state

m_{t}

on each layer of the transformer, for each latent in the sweep. (b, c) Alignment

O_{i}

of the J-lens directions

\bm{v}_{i}

with each latent’s own decoding directions

\bm{D}_{i}

(colored) and with the other latents’ decoding directions (gray), for transformer layer 0 (b) and layer 1 (c).

Figure 17: Alignment of the J-lens directions with the encoding directions in the context coherence sweep. Latents are colored by key bias $\psi$ . Panels as in Fig. 13 .

Figure 18: Alignment of the J-lens vectors with the decoding vectors for each latent in the context coherence sweep. (a) Test decoding accuracy (

R^{2}

) for the Bayes filter belief state

m_{t}

on each layer of the transformer, for each latent in the sweep. (b, c) Alignment

O_{i}

of the J-lens directions

\bm{v}_{i}

with each latent’s own decoding directions

\bm{D}_{i}

(colored) and with the other latents’ decoding directions (gray), for transformer layer 0 (b) and layer 1 (c).

Figure 19: J-lens direction alignment with other features of the HMM that are not the latent states in the context sweep. Black denotes the reveal state

B_{t}

: whether or not the latent state is revealed in the token, and the colors denote the key state

W_{t-1}

(whether the latent state is flipped) for each key bias

\psi

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 778 papers

Short Horizons and Sparse Concepts: a Mathematical View of the Readout in the J-lens
Aug 26, 2026Shi-Qi Yan, Kai-Xuan Ding, Chao-Hong Tan +4Transformer Interpretability Implicit Reasoning in Language Models
Alibaba Token Hub, Alibaba Group
Interpreting Language Model Hidden States at Scale
Aug 10, 2026Jordan Pettyjohn, Mansi Sakarvadia, Nathaniel Hudson +3LLM Interpretability Mechanistic Interpretability
University of Chicago · 2Illinois Institute of Technology · 3Argonne National Laboratory +1
Verbalizing Multi-Token Concepts in LLMs
Aug 31, 2026Xijie Gong, Zimeng Huang, Tonghan WangLLM Interpretability
College of AI, Tsinghua University, Beijing, China

Aug 26, 2026J/K move · Enter open · S save

Short Horizons and Sparse Concepts: a Mathematical View of the Readout in the J-lens

Shi-Qi Yan, Kai-Xuan Ding, Chao-Hong Tan, Qian Chen+3

Alibaba Token Hub, Alibaba Group

Transformer Interpretability Implicit Reasoning in Language Models Causal Reasoning in Language Models Representation Geometry in Language Models

Open paper AskPDF arXiv

The Jacobian lens (J-lens) has been proposed as a way to read verbalizable representations from language models. However, its principle and meaning lack a detailed and theoretical discussion. We provide a mathematical view of this interpretation and of its assumed causal structure. Besides treating the J-lens as a heuristic probe, we further regard it as a first-order causal transfer operator from intermediate activations to expected future readouts. We study the Jacobian matrix as the optimal local linear approximation of the downstream mapping, analyze its global approximation behavior and bias, and identify its mathematical meaning as an expectation over anticipated future readouts. Further analysis of the Jacobian energy distribution reveals that its causal geometry is highly sparse. The energy decays with depth, concentrates in an extremely small proportion, and decomposes into diagonal pathways and specific critical positions. This decomposition further resolves the expectation of the J-lens over future outputs into short-horizon and sparse concept predictions, providing a more intuitive attribution and explanation for the ability of the J-lens to visualize concepts during the thinking process. Based on the theory, we propose a simple but effective improvement strategy and decoupling method for the J-lens, which significantly enhances the ability of the J-lens to read out correct intermediate concepts.

Abstract

LLM Interpretability

Explore similar work

Short Horizons and Sparse Concepts: a Mathematical View of the Readout in the J-lens

Interpreting Language Model Hidden States at Scale

Verbalizing Multi-Token Concepts in LLMs