Where Does Alignment Emerge and What Does It Mean? A Closer Look at the Platonic Representation Hypothesis
Authors: Julia Herbst, Simon Roschmann, Quentin Bouniot, Zeynep Akata
Organizations: Helmholtz Munich · Technical University of Munich · Munich Center for Machine Learning · LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France
The Platonic Representation Hypothesis (PRH) suggests that neural networks are converging toward similar representation spaces. As language models become more capable, their representations are aligning with those of vision models. However, recent work has questioned whether this trend continues to hold for newer language models. Moreover, it has been shown that representation similarity metrics can be confounded by model depth and width, requiring appropriate baselines for interpretation. In this work, we first investigate where alignment emerges between unimodal models across model layers and representation dimensions. Next, we find that the PRH capability-alignment trend persists for newer models as long as the captions are sufficiently clean and informative. Finally, we contextualize cross-modal alignment, introducing unimodal and multimodal baselines on various datasets. Our results show that the cross-modal representational alignment of unimodal models can surpass that of explicitly aligned vision-language models, but remains substantially behind the alignment observed for models of the same modality. Code is available at https://github.com/ExplainableML/prh-closer-look.
Figures & tables
Figure 1: Heatmaps with mutual k -NN alignment between every layer on WIT.
Figure 2: Effect of PCA compression on alignment between DINOv2 and language models on WIT.
Figure 3: Performance of language models on MMLU vs. alignment with vision models.
Figure 4: Performance of LLMs on ARC and GSM8K vs. alignment with vision models on DOCCI.
Figure 5: Alignment for different models and evaluation protocols: DINOv2/v3 (D2/D3), Qwen3-Emb-8B (Q3), NV-Embed-v2 (NV), CLIP, SigLIP 2, Qwen3-VL-Emb-8B (Q3-VL) (mean and standard deviation across 3 runs).
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Vision-Language
Vision-Language
Figure 8: Effect of PCA compression on alignment between DINOv2 and language models on DOCCI.
Figure 9: Effect of PCA compression on alignment between DINOv2 and language models on COCO.
Figure 10: Performance on ARC vs. alignment with DINOv3.
Figure 11: Performance on GSM8K vs. alignment with DINOv3.
WIT
COCO
DOCCI 100%
VLM
Final
Max
Final
Max
Final
Max
CLIP
0.199
0.199
0.317±0.005
0.317±0.005
0.292±0.005
0.293±0.003
SigLIP 2
0.079
0.079
0.285±0.004
0.301±0.005
0.258±0.001
0.263±0.006
Q3-VL
0.218
0.224
0.407±0.002
0.407±0.002
0.445±0.003
0.445±0.003
Appendix
Table 1: Alignment of explicitly aligned VLMs at the final layer pair (shared embedding space) and at the maximally aligned layer pair. Mean and standard deviation over three runs for COCO and DOCCI. WIT consists of a single run.
VLMs
Figure 13: Alignment for different models and evaluation protocols with DOCCI captions of varying verbosity: DINOv2/v3 (D2/D3), Qwen3-Emb-8B (Q3), NV-Embed-v2 (NV), CLIP, SigLIP 2, Qwen3-VL-Emb-8B (Q3-VL).
Figure 14: Alignment for different models and evaluation protocols with N=1024 DOCCI samples and varying k .
Figure 15: Alignment for different models and evaluation protocols on DOCCI samples with varying N and k but constant ratio Nk .
Figure 17: Example of progressively compressed DOCCI captions for the same image.
The Platonic Representation Hypothesis posits that neural networks trained on different modalities (e.g., text and images) converge toward a shared representation of reality. If true, this has significant implications for whether modality choice matters at all. In this paper, we show that the evidence for this claim is substantially weaker than subsequent work suggests. The mutual k-nearest-neighbor metric used on 1024 text-image pairs in the original study captures only coarse structure. To keep the alignment from collapsing as one scales up the data, k has to grow proportionally, undercutting the argument for fine-grained representational convergence. The reported increase in alignment with language model strength saturates for recent models. Moreover, the one-to-one text-image pairing favors alignment, while alignment decreases with non-bijective data. We further find that image and text representations indeed share coarse semantic structure, but neither stronger language models nor richer captions yield fine-grained alignment. Thus, multimodal representations share coarse structure without evidence of convergence to a shared representation -- arguably, full representational convergence would require fine-grained alignment.
A. Sophia Koepke, Daniil Zverev, Shiry Ginosar +1
UC Berkeley · Technical University Munich, MCML · University of Tübingen, Tübingen AI Center +1
We investigate the Platonic Representation Hypothesis (PRH) through a tripartite statistical framework of representations: signal, bias, and noise. {1) Signal:} We propose that Platonic alignment arises from the universal relationship between objects and attributes, which is encoded linearly in representations according to the Linear Representation Hypothesis (LRH). We provide evidence that LRH helps explain PRH by extracting linear object-attribute features with sparse autoencoders and showing that these sparse representations often exhibit stronger cross-modal alignment than their dense counterparts. {2) Bias:} Models have different implicit biases due to the diverse architectures and training procedures used. We show that this difference can be partially mitigated. Centering and normalization consistently improve cross-model alignment. {3) Noise:} Finite-sample training leads to noise in representations. We provide evidence that representational noise is driven by data scarcity by revealing a strong and consistent positive correlation between word frequency and alignment in LLMs and text embedding models. Synthesizing signal, bias, and noise, we propose a statistical model that refines the Linear Representation Hypothesis and explains further phenomena related to the alignment of representations emerging from diverse modern AI architectures.
Kiril Bangachev, Guy Bresler, Yury Polyanskiy
Department of Electrical Engineering and Computer Science · Massachusetts Institute of Technology
Whether models trained on different modalities converge to the same representation of the world is central to AI. The Platonic Representation Hypothesis (PRH) conjectures that they do as they scale, but without a proof it offers neither a guarantee nor a recipe. We turn the hypothesis into a theorem. First, we prove a Platonic Representation Theorem: Joint-Embedding Predictive Architectures (JEPAs) trained separately on any number of modalities learn the same representation up to one rotation whenever their positive pairs are two observations of one scene--good views--and their misalignment is bounded otherwise. Second, we show that next-token prediction is a JEPA with bad views, parts of a single document, which explains why language models align with vision only through scale. Third, we validate the theory from 145k captions to the texts of 940M web images, and uncover a scaling law of alignment that predicts unseen runs within seed noise. Good views need 42 times fewer training samples than next-token prediction on the same corpus to reach the same alignment, and a 109M-parameter text JEPA aligns with DINOv2 and CLIP's image encoders better than every language model we evaluated, up to 72B parameters, and than CLIP's text encoder: next-token prediction is a remarkably inefficient route to platonic representations. As a showcase, one rotation turns separately trained encoders into an image captioner (CIDEr .838 on COCO). Even that rotation needs no pair: on the latest pair-free benchmark, our text JEPA beats every zero-shot accuracy reported with language models up to 8B parameters, e.g., 64% against at most 56% top-5 on ImageNet-100. Positive views, and not scale alone, are thus the lever to build platonic representations.