Emergent phases of superposition: from partial to full representation

Lihao Guo; Yizhou Liu; Jeff Gore

cs.LGSep 29, 2026

Emergent phases of superposition: from partial to full representation

Authors: Lihao Guo, Yizhou Liu, Jeff Gore

Organizations: Massachusetts Institute of Technology Cambridge, MA 02139, USA

Abstract

Large language models are thought to represent features by vectors in a hidden space of dimension given by the model's width. Superposition, in which more features are represented than the width by letting representation vectors overlap, is a leading account of how representation vectors are organized. However, how model width and data statistics determine the configuration of representation vectors and the resulting loss when the number of features and the width are large remains less understood. Here we show, in Anthropic's toy model of superposition, that increasing the width drives a continuous phase transition from a partial-representation phase, where only a subset of features receives appreciable representation vectors while the rest vanish, to a full-representation phase, where every feature is represented. Our theory via a partial random projection approximation predicts, and experiments confirm, that the critical width grows linearly with the number of active features up to a logarithmic factor. The loss scaling changes across the transition: below the critical width, the loss grows linearly with the number of active features and depends weakly on the width in a form set by data statistics; above it, the loss grows approximately quadratically with the number of active features and decays inversely with the width. Non-uniform firing probabilities delay the transition and lower the loss, as more frequent features occupy more space. Our results provide an account of how model width and data statistics jointly shape representations and loss, a step toward understanding representation scaling in large models.

Figures & tables

Figure 1: The toy model composed of embedding and unembedding layers exhibits different types of representation configurations. (a) and (b) are schematic illustrations of the toy model. Colored boxes are used to represent the element values of the input, hidden, and output vectors. (a) No superposition. Only $m$ features are represented by vectors without overlap, and the remaining features are discarded. (b) Full representation. All features are represented by vectors with appreciable norms.

Figure 2: As the width grows, the model undergoes a transition from partial to full representation. (a) Distributions of $s_{i}$ at different $m$ . At small $m$ the distribution is bimodal while at large $m$ the distribution is unimodal. (b) The distribution of $s_{i}$ and $b_{i}$ of each feature. Sample points at small $m$ form two clusters while at large $m$ they form one cluster. (c) The loss $\mathcal{L}$ and the fraction $r/n$ of represented features as functions of $m$ . (d) The ratio $r/m$ as a function of $m$ . It is always greater than 1, indicating superposition. $n=10000,\bar{E}=50$ .

Figure 3: The partial random projection approximation captures the behavior of toy model. (a) Schematic illustration of ideal partial representation. (b) The loss $\mathcal{L}$ predicted by partial RP approximation as a function of $r$ at different $m$ . (c) The predicted $r_{{\rm opt}}$ shows good consistency with the experimental $r$ . (d) The predicted $\mathcal{L}$ shows good consistency with experimental loss. $n=10000,\bar{E}=50$ .

Figure 4: The critical width $m^{*}$ increases with the active feature number $\bar{E}$ , growing asymptotically proportional to it up to a logarithmic correction. (a) Phase diagram of $r/n$ as a function of $\bar{E}$ and $m$ . The red dashed curve is the phase boundary predicted by partial RP approximation. (b) By rescaling $m$ according to Eq. ( 15 ), the critical points collapse. $n=10000$ .

Figure 5: The loss grows with number of active features, approximately proportionally in partial representations and quadratically up to a logarithmic correction in full representations. (a) As

\bar{E}

increases, the

\mathcal{L}

m

curve shifts upwards. (b) The loss

\mathcal{L}

as a function of

\bar{E}

under fixed

n

and two different

m

(before and after transition).

n=10000

Figure 6: The critical width $m^{*}$ increases with non-uniformity, i.e., the data exponent $\epsilon_{p}$ . (a) Distributions of $s_{i}$ at different $m$ . As $m$ increases, two $s_{i}$ bands change into one band. $\epsilon_{p}=0.2$ (b) Phase diagram of $r/n$ as a function of $\epsilon_{p}$ and $m$ . The red dashed curve is the phase boundary predicted by partial RP approximation. $n=10000,\bar{E}=10$ .

Figure 7: The loss decays with the data exponent $\epsilon_{p}$ , since the representation adapts to the non-uniformity. (a) As $\epsilon_{p}$ increases, the $\mathcal{L}$ - $m$ curve shifts downwards. (b) The norm decreases with $p_{i}$ in the direction predicted. The red cross point is the solution of uniform features given the same $\bar{E},m$ and $n$ . $\epsilon_{p}=0.2,m=1000$ . For (a) and (b), $n=10000,\bar{E}=10$ .

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Auxiliary function

F(x)

and

F_{{\rm asym}}(x)

Figure 9: Time sequences of trained loss. The loss converges before $t=3\times 10^{4}$ . $\bar{E}=64,n=10000,\sigma_{v}={1\over\sqrt{3}},m=200$ .

Figure 10: (a) Relative error of the loss

\mathcal{L}

. (b) Error of

r/n

. The typical value of

r/n

is of order 1. Since we focus on the transition, where

r/n

reaches 1, we report the absolute rather than the relative error of

r/n

. For both panels 10 seeds are used.

\bar{E}=50,n=10000,\sigma_{v}={1\over\sqrt{3}}

Figure 11: (a) Phase diagram of $r/n$ as a function of $m$ and $\sigma_{v}$ . The red dashed curve is the critical width $m^{*}$ predicted by partial RP approximation. The critical width $m^{*}$ decreases with $\sigma_{v}$ . (b) The $\mathcal{L}$ - $m$ at different $\sigma_{v}$ . The loss $\mathcal{L}$ increases with $\sigma_{v}$ . $\bar{E}=16,n=10000$ .

Figure 12: Losses and representation parameters of different models. (a) Losses of different models. All loss curves have the same trend and similar bending. (b) The vector norms of different models. For the original toy model, the mean norm of all features is shown, and only the data in full-representation phase (

350\lesssim m

) is shown. The error bar is too small to be seen. (c) The biases of different models. For the original toy model, the mean bias of all features is shown; only the data in full-representation phase (

350\lesssim m

) is shown.

\bar{E}=50,n=10000,\sigma_{v}={1\over\sqrt{3}}

Figure 13: (a) The mean value of

c_{ij}|_{i\neq j}

in the EN model. It reaches the theoretical lower bound

-{1\over n-1}

. (b) The variance of

c_{ij}|_{i\neq j}

in the EN model. It reaches the theoretical lower bound

{n-m\over m(n-1)}

. The curve

1/m

is plotted out as a comparison.

\bar{E}=10,n=10000,\sigma_{v}={1\over\sqrt{3}}

Figure 14: The distribution of the crosstalk noise $\eta$ reconstructed from $c_{ij}$ of the EN model. The red dashed line is the pdf of Gaussian distribution ( 10 ). $\bar{E}=10,n=10000,\sigma_{v}={1\over\sqrt{3}}$ .

Figure 15: (a) The skewness of

\eta

at different

n

. The skewness decrease with

m/n

from 0.

\bar{E}=10,\sigma_{v}={1\over\sqrt{3}}

. (b) The trained loss of the EN model and the mean field loss calculated by applying the reconstructed

\eta

distribution to Eq. ( 42 ). They are highly consistent.

\bar{E}=10,n=10000,\sigma_{v}={1\over\sqrt{3}}

Figure 16: The crosstalk noise distribution of the RP model and the EN model. The red dashed curve is the pdf of Gaussian distribution ( 10 ), and the green dashed curve is the pdf of Gaussian distribution with variance

\bar{E}/m\cdot(\bar{v}^{2}+\sigma_{v}^{2})

. The crosstalk noise distribution of the RP model is highly consistent with the Gaussian distribution with variance

\bar{E}/m\cdot(\bar{v}^{2}+\sigma_{v}^{2})

\bar{E}=10,m=1000,n=10000,\sigma_{v}={1\over\sqrt{3}}

Figure 17: (a) The represented loss $\mathcal{L}_{{\rm rep}}$ predicted by Eq. ( 52 ) at fixed $\bar{E}/n$ . Loss curves collapse after rescaling $m$ to $m/n$ . (b) The EN model loss $\mathcal{L}_{{\rm EN}}$ at different $n$ and its asymptotic value in $n\to\infty$ . For each $m/n$ , we fit $\mathcal{L}_{{\rm EN}}(n)$ by $\mathcal{L}_{{\rm EN}}=A+Bn^{\alpha}$ . If the fit matches the loss, we use $A$ from fitting as the asymptotic value, otherwise we use the minimal measured loss as the experimental asymptotic value (an upper bound on the real asymptotic value). Circles mark the points based on fit result. (c) $\mathcal{L}_{{\rm EN}}$ and the fit result at $m/n=1/16$ . $\bar{E}/n=1/2048$ , $\sigma_{v}={1\over\sqrt{3}}$ .

Figure 18: (a) The represented loss

\mathcal{L}_{{\rm rep}}

predicted by Eq. ( 52 ) at different

\bar{E}/n

n

is large enough so that the large-

n

collapse happens. (b) The asymptotic values of the EN model loss

\mathcal{L}_{{\rm EN}}

at different

\bar{E}/n

. Experimental asymptotic values are obtained by the extrapolation described in Fig. 17 . The red dashed curve is the limiting value

\lim_{\bar{E}/n\to 0}\lim_{n\to\infty}\mathcal{L}_{{\rm rep}}

computed numerically from Eq. ( 52 ).

\sigma_{v}={1\over\sqrt{3}}

Figure 19: The distribution of $s_{i}$ and $b_{i}$ of each feature. Sample points at small $m$ form two clusters while at large $m$ they form one cluster. Different panels are distributions at different $\sigma_{E}^{2}$ . $\bar{E}=20,n=10000,\sigma_{v}={1\over\sqrt{3}}$ .

Figure 20: Phase diagram of $r/n$ as a function of $m$ and $\sigma_{E}^{2}$ . The red dashed curve is the critical width $m^{*}$ predicted by partial RP approximation. The critical width $m^{*}$ increases with $\sigma_{E}^{2}$ . $\bar{E}=20,n=10000,\sigma_{v}={1\over\sqrt{3}}$ .

Figure 21: (a) The loss

\mathcal{L}

as a function of

m

at different

\sigma_{E}

. In full-representation phase, the loss

\mathcal{L}

increases with

\sigma_{E}

. (b) The loss

\mathcal{L}

m=500

, which is greater than the critical width. In full-representation phase, the loss

\mathcal{L}

is approximately linear in

\sigma_{E}^{2}

\bar{E}=20,n=10000,\sigma_{v}={1\over\sqrt{3}}

Figure 22: Distributions of $s_{i}$ at different $\epsilon_{p}$ . At small $\epsilon_{p}$ , the distribution of $s_{i}$ show clear bimodality at small $m$ ; at large $\epsilon_{p}$ , the two bands of $s_{i}$ merge at small $m$ , and features cannot be divided into represented and unrepresented groups. $\bar{E}=1,n=10000,\sigma_{v}={1\over\sqrt{3}}$ .

Figure 23: The distribution of $s_{i}$ and $b_{i}$ of each feature. Sample points at small m form two clusters while at large m they form one cluster. Different panels are distributions at different $\epsilon_{p}$ . $\bar{E}=10,n=10000,\sigma_{v}={1\over\sqrt{3}}$ .

Figure 24: Test of Eq. ( 118 ) in the trained model. Each point is one feature. The horizontal axis is $({1\over h_{i}}-1)/(n-1)$ , and the vertical axis is the variance of $c_{ij}$ over $i\neq j$ . The red dashed line is $x=y$ . $\bar{E}=10,n=10000,\sigma_{v}={1\over 3},\epsilon_{p}=0.2,m=1000$ .

Figure 25: Norms $s_{i}$ (top row), biases $b_{i}$ (middle row), and leverage scores $h_{i}$ (bottom row) as functions of the firing probability $p_{i}$ at different $m$ . Each point is one feature. The red dashed lines are the linear-response predictions of Eq. ( 132 ) around the uniform solution. The predictions capture the signs of the responses but overestimate their magnitudes. $\bar{E}=10,n=10000,\sigma_{v}={1\over 3},\epsilon_{p}=0.2$ .

Phase Transitions in Neural Networks

See all 25 papers

Jun 1, 2026

Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models

Model dimension (

d_{model}

) is a fundamental hyperparameter in transformer language models, yet its role in setting the geometric limits of feature representation remains under-explored. Grounded in the Linear Representation and Superposition Hypotheses - which propose that models encode features as near-orthogonal directions in latent space - we develop a framework for estimating how many such directions a model can support. We first establish the embedding matrix as a measurable proxy for near-orthogonality constraints across the latent space: the boundary between meaningful token relationships and incidental similarity in the pairwise cosine similarity distribution gives a concrete estimate of the model's accepted deviation

\varepsilon

from perfect orthogonality. Applying this metric across dozens of open-source models reveals two classes: models with high

\varepsilon

whose embeddings lack near-orthogonal structure, and models with low

\varepsilon

that maintain it. We then show that the standard Johnson-Lindenstrauss lemma greatly underestimates the packing efficiency of trained representations, and derive an adjusted capacity formula in which the number of near-orthogonal directions depends on the ratio of vectors to dimensions (

k/d

) rather than the raw count - a single modification that cuts prediction error by two orders of magnitude with no extra parameters. Combining these results, we define representational capacity as an upper bound on the number of distinguishable directions available for features and embeddings in a model's latent space. Capacity is exponentially sensitive to

\varepsilon

, and larger models favor tighter orthogonality constraints over maximizing raw capacity - a pattern compatible with several explanations (a stability-capacity trade-off, a ceiling on usable concepts, or confounds with model scale) that we leave to future work.

Alexander Guha

Arizona State University

Feature Superposition Transformer Expressivity Neural Representation Geometry Linear Representation Hypothesis Representation Disentanglement Representation Geometry in Language Models

Ask PDF

Sep 6, 2026

Feature Superposition in Neural Networks: From Theory to Practice

Superposition refers to neural networks representing more features than they have dimensions. It offers a possible explanation for polysemantic neurons and motivates methods for recovering interpretable features from neural activations. Theoretical models typically start with a given set of input features and assumptions about how their values vary across inputs, then study how a network encodes those values in a lower-dimensional hidden representation. Empirical work, by contrast, seeks to identify the features encoded in trained networks and determine their role in computation. In this survey, we review the geometry, learning, and computation of superposed representations, explaining how feature statistics and decoder choice affect the conclusions. To connect these theoretical accounts with evidence from trained networks, we compare practical methods for recovering and analyzing features and examine what their evaluations establish. Since accurate activation reconstruction alone does not establish feature identity or causal use, we discuss the methods' documented failures and applications in light of the evidence available for these different claims. Finally, we assess previously stated open problems and identify remaining theoretical and empirical questions about superposition in trained networks. We hope our work can pave the way for a deeper understanding of superposition and more reliable methods for interpreting neural networks.

Dai Shi, Xiaoyu Li, Andi Han +1

University of Cambridge · University of New South Wales · University of Sydney

Feature Superposition Neural Network Interpretability Neural Representation Geometry

Ask PDF

Jul 6, 2026

Compressed Computation under $L^4$ Loss is likely Computation in Superposition

Neural networks are thought to represent concepts as directions in their activation space, and superposition lets them encode more concepts than they have dimensions. It is natural to ask whether they can also compute more functions than they have neurons, i.e., perform computation in superposition. In this regime many functions of sparse inputs are evaluated by a layer with fewer neurons than there are functions to compute. Representation in superposition is by now fairly well understood, but computation in superposition is not, and there are few toy models of it arising through training rather than being hand designed. As a toy model of computation in superposition we study the compressed-computation setup: a single-hidden-layer ReLU network with 50 neurons that must compute the ReLU of each of 100 sparse input features. We show that training it under an

L^4

loss (the mean fourth power of the error), rather than the usual

L^2

, elicits a solution that appears to compute all features in superposition. We then reverse-engineer this solution. We find that the network assigns each feature a sparse binary codeword over neurons and decodes it with a pseudoinverse of the encoder. Given these codewords, a description with only three scalars recovers most of the network's performance, and we validate it by building equivalent networks from hand-designed codes.

Francisco Ferreira da Silva, Stefan Heimersheim

ReLU Neural Networks Feedforward Neural Networks Neural Network Expressivity

Ask PDF

Abstract

Phase Transitions in Neural Networks

Explore similar work

Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models

Feature Superposition in Neural Networks: From Theory to Practice

Compressed Computation under L4L^4L4 Loss is likely Computation in Superposition

Compressed Computation under $L^4$ Loss is likely Computation in Superposition