The term "linear representation hypothesis" (LRH) has appeared across diverse subfields of artificial intelligence, neuroscience, and cognitive science. But previous works have not consistently treated the LRH as a falsifiable scientific hypothesis; we analyze these inconsistencies and examine their implications for how prior theoretical and methodological results should be interpreted. Based on this analysis, we argue that claims regarding linear representations become well-defined only through careful examination of the model, representation location, feature definition, and evaluation dataset. We therefore propose a more rigorous formalization of the LRH that makes these dependencies explicit and allows the hypothesis to be evaluated as a falsifiable scientific claim. Finally, we identify some non-trivial open problems that warrant further attention from the research community.
Figures & tables
Year
New papers
Cumulative
1976
1
1
2001
1
2
2007
1
3
2022
2
5
2023
11
16
2024
149
165
Table 1: Number of papers explicitly employing the term “linear representation hypothesis” over time, based on Google Scholar search results (counts as of December 2025). The early history is discussed in Section 2.1 . The sharp increase after 2022 was triggered by Elhage et al. (2022) .
Table 2: Summary of studies providing empirical evidence for, or counterexamples to, the linear representation hypothesis (LRH). The Representation column indicates whether the LRH holds in the study; when it does not hold, the observed non-linear representation is specified. In the Layer column, ∙∘∘ denotes the initial (non-contextualized) embedding layer, ∘∙∘ intermediate layers, and ∘∘∙ the final layer before unembedding. Gray circles ( {\color[rgb]{0.75,0.75,0.75}\bullet} ) indicate that the corresponding layer was also examined but exhibited weaker linear representations.
Context of LRH Usage
Papers (may appear in multiple rows)
Type 1. Theoretical analyses supporting LRH
Park et al. (2024c) ; Jiang et al. (2024) ; Park et al. (2025) ; Liu et al. (2026)
Type 2. Empirical evidence: Supporting examples
Nanda et al. (2023) ; Turner et al. (2023) ; Templeton et al. (2024) ; Tigges et al. (2024) ; Park et al. (2024c) ; Jiang et al. (2024) ; Gurnee and Tegmark (2024) ; Nguyen and Leng (2025) ; Park et al. (2025) ; Kim et al. (2025) ; Agarwal et al. (2025)
Type 3. Empirical evidence: Counterexamples
Csordás et al. (2024) ; Engels et al. (2025a) ; Modell et al. (2025) ; Kantamneni and Tegmark (2025) ; Lei and Cooper (2025a) ; Walker et al. (2025)
Type 4. Empirical analysis of when LRH holds
Merullo et al. (2025)
Type 5. Statistical hypothesis testing of LRH
Reizinger et al. (2025) ; Guo et al. (2025)
Type 6. Critical discussions of LRH limitations
Marshall et al. (2024) ; Wattenberg and Viégas (2024) ; Tan et al. (2024) ; Canby et al. (2025) ; Muraleedharan (2025) ; Geiger et al. (2025) ; Tiblias et al. (2025)
Table 3: Contexts in which the term “linear representation hypothesis (LRH)” is used across the literature. A substantial fraction of prior work employs the LRH primarily as a methodological assumption ( Type 8 ) to justify linear analysis tools, such as linear probing, sparse autoencoder (SAE), and steering vector. As argued in Section 5.3 , when such assumptions are invoked across different layers or feature definitions, the resulting methodological justification may not necessarily transfer across studies.
To make claims about representations that generalize beyond a particular trained model, we need to specify when two representations should count as equivalent. The Linear Representation Hypothesis is often discussed without making this equivalence explicit. Different notions of equivalence preserve different structures, so metrics, probes, and interventions that appear to study the same representation may in fact correspond to different hypotheses. We therefore argue that the Linear Representation Hypothesis is not one hypothesis but a family of claims distinguished by representation equivalence. We formalize this idea using group actions, specifying the representation object, the procedure that produces it, and the property ultimately asserted, while accounting for equivalences imposed by the model architecture. This framework clarifies how assumptions can change across metrics, reading points, and analysis stages, and we use it to audit common representation quantities and recent interpretability analyses.
The Platonic Representation Hypothesis (PRH) holds that increasingly capable models converge toward a common representation. However, recent studies have shown different views on aligning representations via their neighborhoods, spanning trees, and distances. We show that these views can differ because the alignment criteria behind them sit at different points of one lattice, and we formalize this explanation as the Lattice Representation Hypothesis (LRH). Under the LRH, the convergence claimed by the PRH is agreement at the top of the lattice, whereas its evidence comes from neighbor overlap, a criterion near the bottom. The Aristotelian Representation Hypothesis (ARH) restricts convergence to local neighborhoods because calibration keeps the growth of neighbor overlap with model size but largely removes that of centered kernel alignment. We show that this removal can come from calibrating a maximum over layers, so global convergence remains open. Back into Plato's Cave (BPC) infers coarse shared structure from the collapse of neighbor overlap on larger evaluation sets, whereas we trace it at least partly to a fixed number of neighbors covering shorter distances. We show that the spanning-tree overlap used by Structure over Geometry (SOG) mostly repeats one-nearest-neighbor overlap. The LRH also extends to independently trained models, whose training procedure and objective decide the criteria met reliably across runs. We prove this claim for the training procedure in a linear autoencoder and for the objective in a finite contrastive Gibbs model. Experiments on dozens of pretrained vision and language models and thousands of runs of both theoretical models support these conclusions. We hope this work paves the way toward a general theory of representation hypotheses, with a framework to analyze them and compare their evidence. Project page: https://eethanshi.github.io/LRH
Dai Shi, Andi Han, Xiaoyu Li +2
University of Cambridge · University of Sydney · University of New South Wales
We investigate the Platonic Representation Hypothesis (PRH) through a tripartite statistical framework of representations: signal, bias, and noise. {1) Signal:} We propose that Platonic alignment arises from the universal relationship between objects and attributes, which is encoded linearly in representations according to the Linear Representation Hypothesis (LRH). We provide evidence that LRH helps explain PRH by extracting linear object-attribute features with sparse autoencoders and showing that these sparse representations often exhibit stronger cross-modal alignment than their dense counterparts. {2) Bias:} Models have different implicit biases due to the diverse architectures and training procedures used. We show that this difference can be partially mitigated. Centering and normalization consistently improve cross-model alignment. {3) Noise:} Finite-sample training leads to noise in representations. We provide evidence that representational noise is driven by data scarcity by revealing a strong and consistent positive correlation between word frequency and alignment in LLMs and text embedding models. Synthesizing signal, bias, and noise, we propose a statistical model that refines the Linear Representation Hypothesis and explains further phenomena related to the alignment of representations emerging from diverse modern AI architectures.
Kiril Bangachev, Guy Bresler, Yury Polyanskiy
Department of Electrical Engineering and Computer Science · Massachusetts Institute of Technology