Platonic Representation Hypothesis

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 11

All topics

A theory of platonic representations in language models
Oct 5, 2026Darshil Doshi, Wenjie Zhou, Corinna Elena Wegner +3Multilingual Language Models Cross-Lingual Representation Alignment
Johns Hopkins Univeristy · Johns Hopkins University & EPFL
What Converges in the Platonic Representation Hypothesis? Structure over Geometry
Sep 23, 2026Junwon You, Mihyun Jang, Sangwoo Mo +1Neural Representation Geometry Representation Alignment
KAIST · POSTECH
Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities
Sep 21, 2026Chenming Shang, Yujin Tang, Jun Jie Ou Yang +3Cross-Modal Alignment Transfer Learning
Department of Computer Science, DARTMOUTH COLLEGE
The Concept of Representation in ML: Beyond Plato and Aristotle
Jul 20, 2026Gilad Landau, Aviv KerenRepresentation Learning Representation Alignment
University of Oxford · Sett AI
PlatonicNav: Unveiling Semantic Correspondence in Navigation with Platonic Topological Maps
Jun 1, 2026Junlin Long, Zeyu Zhang, Xu Deng +5Embodied Navigation Vision-Language Navigation
1USYD · 2Maincode · 3UNSW +1
Representation Alignment Rests on Linear Structure
May 22, 2026Kiril Bangachev, Guy Bresler, Yury PolyanskiyLinear Representation Hypothesis Cross-Modal Alignment
Department of Electrical Engineering and Computer Science · Massachusetts Institute of Technology
Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning
May 22, 2026Muhammad Usama, Dong Eui ChangRepresentational Similarity Analysis LLM Interpretability
Control Laboratory, School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon 34141, Republic of Korea · Control Laboratory, School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST)2026 Daejeon 34141, Republic of Korea
Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models
May 10, 2026Aojie Yuan, Zhiyuan SuLLM Interpretability Representation Geometry in Language Models
University of Southern California · 2Duke University
Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale
Apr 20, 2026A. Sophia Koepke, Daniil Zverev, Shiry Ginosar +1Cross-Modal Learning Cross-Modal Representation Learning
UC Berkeley · Technical University Munich, MCML · University of Tübingen, Tübingen AI Center +1
Revisiting the Platonic Representation Hypothesis: An Aristotelian View
Feb 16, 2026Fabian Gröger, Shuo Wen, Maria BrbićRepresentational Similarity Analysis Neural Representation Geometry
1EPFL · University of Basel · 3HSLU
The Platonic Universe: Do Foundation Models See the Same Sky?
Sep 23, 2025UniverseTBD, :, Trinidad Borrell +11Cross-Modal Alignment Multimodal Model Evaluation
Independent Researcher · AstroAI · Harvard-Smithsonian CfA +4

Oct 5, 2026, cs.LGJ/K move · Enter open · S save

A theory of platonic representations in language models

Darshil Doshi, Wenjie Zhou, Corinna Elena Wegner, Daniel J. Korchinski+2

Johns Hopkins Univeristy · Johns Hopkins University & EPFL

Multilingual Language Models Cross-Lingual Representation Alignment Representation Geometry in Language Models Representation Alignment Platonic Representation Hypothesis

Open paper AskPDF arXiv

Representations of translated sentences are similar in the inner layers of multilingual language models -- an observation connected to the platonic representation hypothesis, yet unexplained theoretically. We provide an explanation based on the assumption that data have a hidden hierarchical structure whose abstract levels are shared across languages while surface levels are modality- or language-specific. Concretely, we generate synthetic languages from probabilistic context-free grammars sharing upper-level but not lower-level production rules. In this setting the Bayes-optimal next-token predictor is belief propagation (BP); encoding its messages in successive layers yields analytical predictions that agree well with transformers trained on the same data. The framework explains why cross-lingual similarity peaks in middle layers, coexists with language-specific structure, and strengthens with language proximity, model quality and data exposure. It distinguishes similarity (shared neighborhood geometry) from alignment (shared coordinates), showing that the latter occurs when code-switched data, i.e. mixed-language sentences, are abundant enough. It further predicts that subtracting from each layer the component linearly predictable from the preceding one increases cross-lingual similarity, which we confirm in pretrained LLMs.

Momentum

Related topics

Latest papers 11