Feature Superposition in Neural Networks: From Theory to Practice

Dai Shi; Xiaoyu Li; Andi Han; José Miguel Hernández-Lobato

cs.LGSep 6, 2026

Feature Superposition in Neural Networks: From Theory to Practice

Authors: Dai Shi, Xiaoyu Li, Andi Han, José Miguel Hernández-Lobato

Organizations: University of Cambridge · University of New South Wales · University of Sydney

Abstract

Superposition refers to neural networks representing more features than they have dimensions. It offers a possible explanation for polysemantic neurons and motivates methods for recovering interpretable features from neural activations. Theoretical models typically start with a given set of input features and assumptions about how their values vary across inputs, then study how a network encodes those values in a lower-dimensional hidden representation. Empirical work, by contrast, seeks to identify the features encoded in trained networks and determine their role in computation. In this survey, we review the geometry, learning, and computation of superposed representations, explaining how feature statistics and decoder choice affect the conclusions. To connect these theoretical accounts with evidence from trained networks, we compare practical methods for recovering and analyzing features and examine what their evaluations establish. Since accurate activation reconstruction alone does not establish feature identity or causal use, we discuss the methods' documented failures and applications in light of the evidence available for these different claims. Finally, we assess previously stated open problems and identify remaining theoretical and empirical questions about superposition in trained networks. We hope our work can pave the way for a deeper understanding of superposition and more reliable methods for interpreting neural networks.

Figures & tables

Symbol	Meaning
$d$	number of features
$f=[f_{1},\dots,f_{d}]^{\top}\sim\mathcal{D}_{f}$	feature values and their distribution
$p=\mathbb{P}(f_{i}\neq 0)$ ; $k$	activation probability; hard sparsity level ( $\left\lVert f\right\rVert_{0}\leq k$ )
$\rho_{j}:\mathcal{X}\to\mathbb{R}$	interpretable property of the data inducing feature $j$
$m$	hidden width ( $m<d$ )
$W=[\,w_{1},\dots,w_{d}\,]\in\mathbb{R}^{m\times d}$	linear encoder; column $w_{i}$ is the direction of feature $i$

Table 1: Notation used throughout the survey, following Prieto et al. (2026) .

Figure 1: Feature directions in a two-dimensional hidden space and their relation to neuron responses. (a) Two features occupy the two neuron axes, so the encoding

h=Wf

gives each neuron one feature value. (b) A

45^{\circ}

rotation makes each neuron respond to both features while preserving the orthogonality of their directions and all encoded information. The rotated code therefore still fails clause 1 of Definition 2.1 . (c) Five directions share the same two-dimensional space; the displayed nonzero inner products

\left\langle w_{i},w_{j}\right\rangle

show the overlap that is unavoidable when features outnumber dimensions.

Figure 2: Reconstruction and recovery in the one-dimensional codes of Example 2.4 . (a) For two

\mathrm{Bernoulli}(p)

features, the dedicated code drops one feature and incurs risk

p

, whereas the antipodal code keeps both features and incurs

2p^{2}

. The antipodal error occurs when the two features fire together, an event of probability

p^{2}

on which the ReLU decoder errs on each feature. (b) Despite this error, the ReLU decoder achieves

R^{2}=1-\frac{p}{1-p}\to 1

as features become sparse, while the best linear decoder is capped at

R^{2}=1/2

for every

p

. For

\varepsilon<\tfrac{1}{2}

and small

p

, these features are therefore in non-linear superposition by Definition 2.2 .

Quantity	Definition and source	Constrains
Overcompleteness ratio $d/m$	Ratio of features to dimensions.	Counts candidate directions; does not by itself specify recovery or information content.
Feature capacity $C_{i}$ ; bound $\sum_{i}C_{i}\leq m$	$C_{i}=\left\lVert w_{i}\right\rVert^{4}/\sum_{j}\left\langle w_{i},w_{j}\right\rangle^{2}\in(0,1]$ ; $\sum_{i}C_{i}\leq\operatorname{rank}W$ ( Scherlis et al., 2022 ) ; identical to the feature dimensionality of Elhage et al. (2022) ( Section 3.4 ).	Geometric allocation for nonzero columns; does not bound task loss or causal use.
Welch bound	$\max_{i\neq j}\|\left\langle w_{i},w_{j}\right\rangle\|\geq\sqrt{\smash[b]{\tfrac{d-m}{m(d-1)}}}$ ( Welch, 1974 ) .	Worst-case pairwise overlap for unit-norm columns; related total-overlap bounds appear in Section 3.2 .
Linear-access width	Least $m$ such that one linear map recovers every $k$ -sparse $f\in[-1,1]^{d}$ to $\ell_{\infty}$ -error $\varepsilon$ : $O(\varepsilon^{-2}k^{2}\log d)$ and $\Omega_{\varepsilon}\!\bigl(\tfrac{k^{2}}{\log k}\log\tfrac{d}{k}\bigr)$ in the regime of Result 4.1 ( Garg et al., 2026 ) .	The cost of simultaneous uniform linear readout: quadratic in $k$ .
Sparse-recovery width	$m=O(k\log(d/k))$ suffices with a decoder in $\Psi_{\mathrm{sparse}}$ and a random or RIP dictionary ( Candès et al., 2006 ; Donoho and Elad, 2003 ) .	Linear in $k$ ; the gap to the row above is the decoder class, not a contradiction.
Information-theoretic capacity	With coordinate range $R$ and resolution $\delta$ : $\log_{2}N\leq m\log_{2}(1+2R/\delta)$ distinguishable messages.	Why finite width alone constrains nothing without a precision model ( Section 3.5 ).

Table 2: Seven quantities used in discussions of capacity, in the notation of Table 1 . Their input assumptions, recovery criteria, and resource counts differ.

Figure 3: Interference and linear-readout risk for overcomplete feature codes. (a) Arranging five features as a regular pentagon in two dimensions gives a unit-norm tight frame satisfying $WW^{\top}=(d/m)\,I_{m}$ . (b) The off-diagonal entries of its Gram matrix give the pairwise interferences, whose squared sum meets the lower bound $d(d-m)/m=7.5$ of Result 3.2 . (c) At width $m=16$ , the risk curves compare tied and untied linear readouts across overcompleteness ratios $d/m$ , with risk measured in units of $\sigma^{2}$ . Random codes read with the tied readout $V=W$ sit above the tied-readout lower bound $d(d-m)/m$ . The optimal untied readout for a full-row-rank encoder attains $d-m$ . The ratio $d/m$ is attained by the two risks at a unit-norm tight frame, rather than by every random encoder ( Result 3.3 ).

Decoder class	Guarantee considered	Conditions and limits
One linear map, $\Psi_{\mathrm{lin}}$	$m=\widetilde{\Theta}_{\varepsilon}(k^{2})$ ( Garg et al., 2026 )	Uniform $\ell_{\infty}$ approximation on $k$ -sparse inputs in $[-1,1]^{d}$ . The logarithmic dependence on $d$ and $k$ is explicit in Result 4.1 ; its lower bound requires fixed $0<\varepsilon<\tfrac{1}{2}$ , $2\leq k=O(\log d)$ , and sufficiently large $d$ .
Linear map with pointwise nonlinearity, $\Psi_{\mathrm{lin}+\sigma}$	Depends on the activation and input law	ReLU improves recovery in Example 2.4 ; that example does not give a width bound for every signed sparse input.
Sparse optimization, $\Psi_{\mathrm{sparse}}$	$m=O(k\log(d/k))$ ( Candès et al., 2006 )	Uniform exact recovery under suitable restricted-isometry conditions. Coherence alone gives a more restrictive sufficient certificate.
Unrestricted decoder, $\Psi_{\mathrm{all}}$	Depends on the information retained by the encoder	Admissible patterns mapped to the same observation cannot be distinguished, regardless of decoder complexity.

Table 3: Recovery guarantees for different decoder families. The first and third rows compare uniform sparse recovery with different error criteria and computational procedures. The remaining rows state model-dependent limits rather than intermediate width bounds.

Result	Readout and error model	Sparsity	Width (neurons)
Representation only ( Johnson and Lindenstrauss, 1984 )	Fixed linear readouts recover sparse feature values to fixed tolerance	$k=O(1)$	$m=O(\log d^{\prime})$
All pairwise ANDs, basis-aligned inputs ( Hänni et al., 2024 )	One linear readout with uniform $\ell_{\infty}$ error $\varepsilon$ ; single layer	Fixed $k$	$m=O(\varepsilon^{-2}\log^{8}d)$
All pairwise ANDs, superposed inputs ( Hänni et al., 2024 )	Same uniform guarantee, with a constructed input encoding	Fixed $k$	$m=\widetilde{O}(\varepsilon^{-2}\sqrt{d})$
Sparse Boolean circuits ( Hänni et al., 2024 )	Approximation with interleaved correction; all but a negligible fraction of admissible inputs	$k$ bounds activity throughout the circuit	$m=\widetilde{O}(n^{2/3}k^{2})$ for circuit width $n$ and polynomial depth
Ordered AND instances or permutations, upper bound ( Adler and Shavit, 2024 )	Exact thresholded recovery; shared input and output codes; constant depth per instance, chainable	$k\leq 2$	$m=O(\sqrt{d^{\prime}}\log d^{\prime})$
Same families, lower bound ( Adler and Shavit, 2024 )	Family-level requirement supported by a robust subset; error fraction $\delta<1/2$ is measured on the specified set $U$	$k=2$	$\Omega(d^{\prime}\log d^{\prime})$ parameter bits; width follows under fixed-depth, fixed-precision accounting

Table 4: Computation bounds with their task and recovery conditions. In the all-pairs rows,

d^{\prime}=\binom{d}{2}

. In the circuit row,

n

denotes the emulated circuit width, and the source provides a proof sketch with draft supporting material. The rows quantify over different tasks and resources; their widths do not form a hierarchy of readout strength.

Variant	Change to Equation 11	Difficulty addressed	Demonstrated effect	Scope
ReLU, $\ell_{1}$ ( Bricken et al., 2023 ; Cunningham et al., 2024 )	$\sigma=\mathrm{ReLU}$ ; penalty $\left\lVert z\right\rVert_{1}$	the original recipe	interpretable latents on real models	its own penalty produces the artifacts below
Gated ( Rajamanoharan et al., 2024a )	separate firing and magnitude paths; $\ell_{1}$ on the firing path only	shrinkage: the penalty also rewards scaling magnitudes down	removes aggregate magnitude bias; improves the sparsity-fidelity trade-off in compute-matched comparisons	blinded human interpretability comparison is null; ranking against TopK is protocol-dependent
TopK with auxiliary loss ( Gao et al., 2025 )	keep the largest pre-activations; dead latents fit the residual	dead latents (up to $90\%$ unmitigated); indirect sparsity control	$7\%$ dead at $16$ M latents; sparsity set directly	couples latents through an order statistic; dense, principal-component-like solutions as the active count approaches $m$
JumpReLU ( Rajamanoharan et al., 2024b ; Lieberum et al., 2024 )	per-latent learned threshold; direct $L_{0}$ objective via straight-through estimators	thresholding without batch coupling	basis of the open Gemma Scope suite; a weight-tied Gated encoder has this form with bias-determined thresholds and per-latent rescaling	the release itself reports no ground-truth or causal validation
BatchTopK; Matryoshka ( Bussmann et al., 2024 ; Bussmann et al., 2025 )	batch-level threshold; nested dictionary prefixes, each reconstructing alone	absorption, splitting, composition	reduces first-letter absorption at matched sparsity; avoids worsening absorption with width in the SAEBench comparison	the nesting is a researcher-imposed hierarchy prior; no differentiation on board-game models with known ground truth
End-to-end ( Braun et al., 2024 )	reconstruction replaced by KL divergence of spliced outputs, optionally plus downstream-activation error	reconstruction rewards variance the model never uses	reduces loss increase at matched $L_{0}$ and allocates fewer latents to a high-variance direction whose resample ablation after the first token barely changes output	the pure variant is seed-unstable and routes computation differently; latent magnitudes shrink

Table 5: Sparse-autoencoder designs and their evaluated effects. The choices of coefficient estimation, sparsity control, hierarchy, and training target can interact. Reported improvements apply to the stated objectives and evaluations; identification of model features requires the additional conditions of Section 4.5 .

	Supervised probes and dictionaries	Unsupervised dictionaries	Functional replacement	Parameter decomposition
Question answered	is this named variable accessible, at which decoder class?	what overcomplete coordinate system makes activations sparse?	can an interpretable surrogate substitute for the computation?	which parameter components participate in different computations?
Recovered object	one direction (or offset set) per named variable	dictionary of $\hat{d}$ directions with sparse latents	surrogate sublayer maps; attribution graphs with error nodes	sparsely used parameter components, built from rank-one subcomponents
Evidence sought	accessibility (L2); causal use (L5) tested separately on templatic tasks	identification (L3), conditional on a recovery reference or the assumptions of Section 4.5	substitutability (L4); mechanistic interpretation (L6) requires further tests	mechanistic components (L6 target); evaluations in toy settings and one $\sim$ $10^{8}$ -parameter model
Characteristic artifacts	the ontology is supplied in advance; dataset idiosyncrasies read as features	shrinkage, dead latents, absorption, splitting, seed instability ( Section 9 )	error nodes carry causal weight; attention routing outside the account; conclusions surrogate-relative	unaudited by third parties; interference-separation untested
Reported cost	one supervised fit per variable	hours on small models; a full open suite consumed over $20\%$ of GPT-3’s training compute ( Lieberum et al., 2024 )	surrogate training for the replaced sublayers; graph computation per prompt; cost depends on the replacement recipe	reported only for models up to $\sim$ $10^{8}$ parameters
Sources	Gurnee et al. (2023) ; Makelov et al. (2025)	Cunningham et al. (2024) ; Gao et al. (2025) ; Bussmann et al. (2025) ; Braun et al. (2024)	Dunefsky et al. (2024) ; Marks et al. (2025) ; Ameisen et al. (2025)	Braun et al. (2025) ; Bushnaq et al. (2025) ; Bushnaq et al. (2026)

Table 6: Recovery families compared by target, recovered object, and evaluation needs. The evidence categories in Section 2.5 identify what a test seeks to establish; belonging to a method family does not automatically establish a category. Section 8 examines the required references and controls.

Metric	What it measures	What it does not measure	Documented disagreement
Reconstruction error; explained variance	share of activation variance the dictionary returns	whether the returned directions carry the model’s behavior	most discriminative metric in the audit, yet $593$ against $2$ latents on a direction that is functionally inert away from the first position ( Braun et al., 2024 ; Chanin, 2026 )
$L_{0}$	mean active latents per token	whether that level matches the data’s sparsity	mixing at too low and degenerate solutions at too high $L_{0}$ ( Chanin and Garriga-Alonso, 2026a ) ; decoder-geometry and firing-time ground-truth scores peak at different $L_{0}$ ( Chanin, 2026 )
Loss recovered; KL of spliced outputs	behavior preserved when the reconstruction replaces the activation	which part of the behavior the residual carries	zero-ablation and mean-ablation normalizations are not comparable ( Karvonen et al., 2025 ; Makelov et al., 2025 )
Automated interpretability	whether a language model predicts a latent’s firing from its description	whether the description names a unit the model uses	$0.87$ for a frozen-random dictionary against $0.90$ for a trained one ( Korznikov et al., 2026 ) ; signal-to-noise ratio $1.8$ , limiting reliable selection from a single checkpoint ( Chanin, 2026 )
Sparse probing, SAEBench default	whether a named concept is readable from a few latents	whether the model uses the concept	saturated at the canonical setting, accuracy above $0.99$ while matched cosine spans $0.56$ to $0.79$ ; signal-to-noise ratio $4.2$ at top-1, about two-thirds that of the sae-probes variant ( Chanin, 2026 ) ; top-1 near parity with frozen-random dictionaries ( Korznikov et al., 2026 )
Sparse probing, sae-probes variant	the same, over $113$ datasets with cross-validated probe regularization	whether the model uses the concept	most reliable metric audited, at signal-to-noise ratio $5.4$ to $6.6$ against $1.8$ for automated interpretability ( Chanin, 2026 )

Table 7: Evaluation metrics for recovered features. The last column records where a metric has been shown to disagree with another metric or with a supervised reference, not a general claim that it is invalid.

Source	Model	Sense	Reported quantity
Scaling report ( Templeton et al., 2024 )	Claude 3 Sonnet, middle-layer residual stream	activation variance	at least $65\%$ of activation variance explained, at each of the three dictionary widths; institutional report
Engels et al. (2025b)	Gemma 2 9B, layer 20, open dictionary suite	activation variance	fraction of variance unexplained fitted as $0.186+2.708\,\hat{d}^{-0.317}$ in dictionary width $\hat{d}$ , with the asymptote extrapolated beyond the widths measured; refereed
Circuit-tracing report ( Ameisen et al., 2025 )	18-layer model, $10$ M-latent cross-layer transcoder	attribution-graph paths	replacement score $0.61$ on unpruned graphs, against $0.37$ for per-layer transcoders of the same size, so $39\%$ of the weighted path influence runs through the residual; institutional report
Case-study report ( Lindsey et al., 2025b )	Claude 3.5 Haiku	prompts yielding a satisfying explanation	attribution graphs give satisfying insight on about a quarter of the prompts tried, which the authors state as their own judgment and call difficult to quantify precisely; institutional report

Table 8: Published dark-matter quantities. The three senses are not commensurable, and the row attributed to Engels et al. (2025b) is the one refereed entry.

Activity	Setting	Reported cost
Training an open dictionary suite	more than $400$ JumpReLU dictionaries over all layers of Gemma 2 2B and 9B plus selected layers of 27B	over $20\%$ of GPT-3’s training compute ( Lieberum et al., 2024 )
Training one dictionary	Gemma-2-2B residual stream, one site	$500$ M tokens per dictionary in both protocols ( Korznikov et al., 2026 ; Karvonen et al., 2025 )
Training one dictionary, model organism	chess or Othello, $300$ M tokens	under $24$ hours on one A100; over $500$ dictionaries released ( Karvonen et al., 2024 )
Evaluating one dictionary	eight-metric suite, $16$ k latents	about $110$ minutes per dictionary plus $152$ minutes of one-time setup on one RTX 3090 ( Karvonen et al., 2025 )
Evaluating one dictionary, model organism	coverage and board reconstruction	under $5$ minutes ( Karvonen et al., 2024 )
Auditing the benchmark	snapshot panels, three seeds, plus a synthetic validity panel	roughly $500$ to $650$ H100-hours of evaluation and about $84$ H100-hours of dictionary training ( Chanin, 2026 )

Table 9: Reported compute and data for training and evaluating recovered dictionaries. Numbers are as published and are not normalized to common hardware or a common token count.

Presupposition ( Section 6.5 )	Failure when it does not hold	Evidence	Documented in
Features are linear directions ( Definition 2.3 )	a concept carried by a curved function of the activations has no direction to recover	the surveyed circular subspaces of Engels et al. (2025a) are linear; these examples do not establish a failure of linear subspace representation	Section 9.5
Atoms are one-dimensional	a group of atoms tiles the feature instead of spanning it	two-dimensional circular day, month, and year subspaces ( Engels et al., 2025a ) ; a counting task in which the model manipulates a curved low-dimensional manifold, reported by Gurnee et al. (2026a) ; five architectures on Llama-3.1-8B plateau far beyond each manifold’s ambient dimension ( Bhalla et al., 2026 )	Section 9.5
Coefficients are sparse	a sparse dictionary can leave dense structure in its residual	the residual is modeled as unlearned features, a dense term, and a term the autoencoder introduces ( Engels et al., 2025b ) ; the width scaling of reconstruction error needs a constant term ( Gao et al., 2025 ) ; separately, removing residual-stream error nodes severely disrupts a circuit model, without isolating the dense component ( Marks et al., 2025 )	Section 9.4
Atoms are layer-local	a per-layer dictionary decomposes a projection of an object that is not layer-local	The crosscoder report finds latents whose decoder norm is substantial across most layers, and states that this is consistent with cross-layer superposition without establishing it ( Lindsey et al., 2024 )	Section 7.5
The dictionary is identifiable	absorption, splitting, hedging, and seed-dependent atoms	Result 9.1 ; over-splitting preferred with two true features ( Makelov et al., 2025 ) ; stitching and meta-SAEs ( Leask et al., 2025 )	Sections 9.1 and 9.2

Table 10: Presuppositions used in dictionary learning, the failures they permit, and the evidence reported for each. A listed failure need not have been established in every setting.

Domain and architecture	Object decomposed	Reported finding	Status
Vision and image generation (InceptionV1, DINOv3, SDXL)	activations, block-sparse featurizers	concepts recovered as low-dimensional blocks rather than single directions, selected by a description-length comparison against dictionaries of directions; generation steered within a block ( Fel et al., 2026 )	preprint
Diffusion image models (SD 1.5, SDXL Turbo)	activations, autoencoder latents	latents detect and localize the target concept; multiplier interventions push activations off distribution and produce artifacts in $43.1\%$ to $61\%$ of images ( Cassano et al., 2026 )	preprint
EEG transformers (SleepFM, REVE, LaBraM)	embeddings, TopK autoencoder	three steering regimes separated by a target-versus-off-target probe statistic; interventions that collapse global performance; age and pathology inseparable ( Lehn-Schiøler et al., 2026 )	preprint
Single-cell foundation models (Geneformer V2-316M, scGPT)	residual stream, all layers	$82{,}525$ and $24{,}527$ latents, $99.8\%$ invisible to a singular-value decomposition; $29\%$ to $59\%$ annotate to biological databases; $3$ of $48$ transcription factors show target-specific responses under CRISPRi ( Kendiukhov, 2026 )	preprint
Continuum-dynamics emulator (Walrus)	one layer, over $20{,}000$ latents	feature recruitment recurs across shear-flow setups only piecewise, is intermittent, and does not map onto standard physical decompositions ( Rosenfeld and Sonnewald, 2026 )	workshop
Time-series forecasting transformer (PatchTST)	post-GELU feed-forward activations	widening the dictionary changes performance by $0.214\%$ on average; interventions on dominant latents barely move the forecast; the author concludes superposition is not necessary here ( Yıldırım, 2026 )	preprint

Table 11: Work on superposition and feature recovery outside language models that we identified. The studies use different dictionaries, evaluation targets, and ground truth, so the rows are not comparable with one another.

Question	Posed in	Verdict	Evidence and Remaining Scope
How many latents should a dictionary have?	2022 report	unique width unestablished	The studies leave dictionary width without a uniquely supported choice. Latents fitted at one width decompose into latents fitted at another ( Leask et al., 2025 ) ( Section 9.2 ).
Does dictionary learning on real activations return the model’s own directions?	2022 report	answered negatively	Recovery varies with the generator, matching rule, and threshold rather than consistently returning the stipulated directions: $3$ of $3{,}200$ directions at explained variance $0.67$ under one generator and matching rule, against almost complete recovery under another ( Korznikov et al., 2026 ) ( Section 8.1 ); chess coverage $0.48$ against $0.98$ for probes on the same activations ( Karvonen et al., 2024 ) ( Section 8.2 ).
Are the recovered atoms the model’s features or mixtures of them?	2025 agenda	mixtures can occur	The analyzed optima can favor mixtures, with reconstruction succeeding while recovery fails outside the stated extreme-sparsity conditions ( Cui et al., 2025 ; Tang et al., 2026 ) ( Section 4.5 ).
Is the sparsity objective the right training signal for recovery?	2025 agenda	answered negatively	The objective also favors artifacts. Absorption follows from the penalty ( Result 9.1 ), and splitting appears with no multi-scale structure present ( Makelov et al., 2025 ) ( Section 9.1 ).
Has a latent’s label been shown to be a property of the model?	2025 agenda	answered negatively	Confident descriptions attach to $82\%$ of random directions against $80\%$ of real coordinates ( Bolukbasi et al., 2021 ) , and a frozen random decoder matches trained dictionaries on automated interpretability ( Korznikov et al., 2026 ) ( Section 9.3 ).
Is there a test certifying from activations alone that a layer holds more features than dimensions?	2022 report	reformulated	A recovery-qualified verdict depends on the feature list, tolerance, and decoder class ( Definition 2.1 , Example 2.4 ); the packing degree at a specified layer tests whether the stated feature list contains more recoverable features than that layer has dimensions ( Definition 2.5 ).

Table 12: What became of the open questions stated by Elhage et al. (2022) and, for the superposition-relevant part of a much longer list, by Sharkey et al. (2025) . Of fifty questions assessed, seventeen are displayed: all six with negative findings, all three reformulated questions, both answered questions, three partly answered questions, and three open questions. The thirty-three omitted questions comprise twenty-eight partly answered and five open questions. Verdicts summarize the evidence in the analyzed settings.

Explore similar work

Mar 10, 2026

From Data Statistics to Feature Geometry: How Correlations Shape Superposition

A central idea in mechanistic interpretability is that neural networks represent more features than they have dimensions, arranging them in superposition to form an over-complete basis. This framing has been influential, motivating dictionary learning approaches such as sparse autoencoders. However, superposition has mostly been studied in idealized settings where features are sparse and uncorrelated. In these settings, superposition is typically understood as introducing interference that must be minimized geometrically and filtered out by non-linearities such as ReLUs, yielding local structures like regular polytopes. We show that this account is incomplete for realistic data by introducing Bag-of-Words Superposition (BOWS), a controlled setting to encode binary bag-of-words representations of internet text in superposition. Using BOWS, we find that when features are correlated, interference can be constructive rather than just noise to be filtered out. This is achieved by arranging features according to their co-activation patterns, making interference between active features constructive, while still using ReLUs to avoid false positives. We show that this kind of arrangement is more prevalent in models trained with weight decay and naturally gives rise to semantic clusters and cyclical structures which have been observed in real language models yet were not explained by the standard picture of superposition. Code for this paper can be found at https://github.com/LucasPrietoAl/correlations-feature-geometry.

Lucas Prieto, Edward Stevinson, Melih Barsbey +2

Feature Superposition Neural Network Interpretability Representation Geometry in Language Models

Ask PDF

Jun 16, 2026

Effects of sparsity and superposition on loss in simple autoencoders

One of the major difficulties in the mechanistic interpretability of neural networks is the occurrence of polysemanticity, which suggests that each neuron is typically responsible for multiple different tasks, impeding a clean interpretation of their function. The seminal paper of Elhage et al. (2022) argues that this occurs due to superposition, a phenomenon where the neural network represents distinct features as non-orthogonal directions in a lower-dimensional space, a strategy that allows much greater compression of the data without sacrificing fidelity due to the feature sparsity of input vectors. Elhage et al. (2022) empirically validates these hypotheses in a rather natural and simple autoencoder with sparse inputs. The contribution of the present work is to analyze the mathematical basis for the occurrence and optimality of superposition, while rigorously corroborating some of their findings. In particular, we provide upper and lower bounds for the L2 reconstruction loss, tight in the very sparse regime, for power activation functions. A short list of interesting open problems are also included at the end.

Mriganka Basu Roy Chowdhury, Eric McLaughlin Weiner

Department of Statistics, UC Berkeley · Department of Materials Science, UC Berkeley

Feature Superposition Activation Sparsity Autoencoders Representation Disentanglement Neural Network Interpretability Neural Network Expressivity

Ask PDF

Mar 31, 2026

Similarity of Neural Network Representations in Superposition

Comparing internal representations is a central goal in neuroscience and machine learning, but standard linear alignment metrics (Representational Similarity Analysis, Centered Kernel Alignment, and linear regression) are frequently applied to neural activity coordinates rather than on the underlying features. We show this matters when neural systems operate in superposition, encoding more features than they have neurons via linear compression. Closed-form derivations prove that these metrics depend on the Gram matrices of each system's projection, not on the latent features themselves: alignment thus combines what a system represents with how it is encoded. For those interested in what features two systems share, this is a problem: Two networks can have identical feature content yet appear more dissimilar than networks exhibiting partial feature overlap. This apparent misalignment need not reflect lost information as compressed sensing guarantees sparse features remain recoverable from the compressed activity. We confirm this by training supervised TopK sparse autoencoders that realize solvable compressed sensing by construction, finding alignment on recovered latents restored even when raw-activation alignment remains deflated. We extend the result to unsupervised SAEs trained without ground-truth latents, and to pretrained vision and language model SAEs, where SAE-latent alignment exceeds raw-activation alignment, consistent with superposition in real systems.

Sunny Liu, Habon Issa, André Longon +4

Cold Spring Harbor Laboratory · UC San Diego · Anthropic +2

Feature Superposition Representational Similarity Analysis Neural Representation Geometry Compressed Sensing Neural Encoding

Ask PDF