The Platonic Universe: Do Foundation Models See the Same Sky?

UniverseTBD; :; Trinidad Borrell; Steven Dillmann; Kshitij Duraphe; Furkan Eris; Kartheik Iyer; Ashod Khederlarian

astro-ph.IMSep 23, 2025

The Platonic Universe: Do Foundation Models See the Same Sky?

Authors: UniverseTBD, :, Trinidad Borrell, Steven Dillmann, Kshitij Duraphe, Furkan Eris, Kartheik Iyer, Ashod Khederlarian, +6 more

Organizations: Independent Researcher · AstroAI · Harvard-Smithsonian CfA · University of Hertfordshire · Washington University St. Louis · Space Telescope Science Institute · Johns Hopkins University

Abstract

We investigate when foundation models converge towards shared representations, and how this convergence depends on model capacity, training regime, and model architecture. We take a `science-for-AI' approach, using astronomy as an experimental instrument to test the Platonic Representation Hypothesis and its Aristotelian refinement against an external physical reference. The historical success of astrophysics is evidence that a compact, modality-invariant description of galaxy observables exists, and so representation convergence toward reality should be measurable against the physical parameters astronomers already use. Given this framework, we evaluate eleven foundation model families (spanning classification, self-distillation, joint-embedding prediction, autoencoding, vision-language pre-training, and astro-specific architectures from $\mathcal{O}$ (10M) ${\to}\mathcal{O}$ (10B) parameters) on crossmatched JWST, HSC, and Legacy imagery, and DESI spectroscopy. All models are evaluated frozen, with no astronomy-specific fine-tuning. We probe redshift, stellar mass, and sSFR via linear probes, and local (MKNN) and global (CKA) embedding geometry within families, between modalities, and across architectures. We find that physics performance scales predictably with capacity; probe directions align consistently with expected astrophysical correlations and selection effects; and local (not global) embedding alignment tracks physics performance, including between DESI spectra and HSC imagery---modalities that share essentially no low-level statistics. Our results support the ARH over the strict PRH, demonstrate astronomy's value as an experimental framework for neural representation learning, and suggest that astro-foundation models can build on general-purpose pre-trained architectures, capitalizing on the broader open machine learning community's already-spent computational investment.

Figures & tables

Model	Training regime	Modality
Supervised
ViT ( Dosovitskiy et al., 2020 )	Supervised classification	Images
ConvNeXtv2 ( Woo et al., 2023 )	Supervised classification	Images
Self-supervised
DINOv3 ( Siméoni et al., 2025 )	Self-distillation	Images
IJEPA ( Assran et al., 2023 )	Joint-embedding prediction	Images

Table 1: Foundation models used in this study, spanning supervised, self-supervised, autoregressive, and multimodal training paradigms across vision, language, and spectral modalities.

Figure 1: Linear probe

R^{2}

as a function of model size, averaged over mass, sSFR, and redshift. We find a significant positive correlation, even though AstroPT is the only model in our basket pretrained substantially on galaxy imagery.

Figure 2: Linear probe

R^{2}

scores of JWST versus HSC modalities on a grid of physical parameters. Model performance is strongly correlated between the instruments, indicating that a model’s capacity to linearly encode physical information is a property of its representations rather than a function of the observed wavelength range or modality.

Figure 3: Cosine similarity matrices between the redshift ( $z$ ), stellar mass ( $M_{\star}$ ), and sSFR probe weight vectors for HSC (left) and JWST (right), averaged over all models. The relationships between the three probe weight vectors are consistent across all models, suggesting that the embedding spaces encode physics via similar internal organisation.

Figure 4: Intra-architectural scaling results. We plot our MKNN and CKA similarity values within each architecture family against the mean

R^{2}

probe performance across the physical tasks described in § 3.1 . In each pane we state the

\rho

and

p

values as measured via a Spearman’s rank test.

Figure 5: Crossmodal MKNN and CKA metrics vs

R^{2}

(§ 3.1 ).

R^{2}

and MKNN/CKA are demeaned per architecture family to show the trend without architecture family confounders. We state

\beta

and

p

from the ANCOVA tests

\text{MKNN}\sim R^{2}+C(\text{family})

, and

\text{CKA}\sim R^{2}+C(\text{family})

Figure 6: Cross-architecture metric distance (MKNN, CKA), plotted against average

R^{2}

over the three physical properties for HSC and JWST images (§ 3.1 ). We find that probe performance is significantly correlated with MKNN, but not with CKA.

Category	Name	Size	Hugging Face source
Models	AstroPTv2	15M (Small)	Smith42/astroPT_v2.0
		95M (Base)	Smith42/astroPT_v2.0
		850M (Large)	Smith42/astroPT_v2.0
	CLIP	86M (Base)	openai/clip-vit-base-patch16
		304M (Large)	openai/clip-vit-large-patch14
	ConvNeXtv2	15M (Nano)	facebook/convnextv2-nano-22k-224

Table 2: Foundation models and astronomical datasets used in this study.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Hardware	Approximate Usage
NVIDIA A40 / A80	300 GPU-hours
NVIDIA A100 / A800	600 GPU-hours
NVIDIA GH200 / H100	2000 GPU-hours
NVIDIA RTX 5070 and RTX 3090	50 GPU-hours
Standard CPU cores	$\approx 1$ CPU-hour per probe
Total	3,000 GPU-hours

Appendix

Table 3: Hardware usage summary by chip type.

	HSC ( $R^{2}$ )			JWST ( $R^{2}$ )
Model	$z$	$\log M_{\star}$	sSFR	$z$	$\log M_{\star}$	sSFR
AstroPTv2 Small	$0.387_{\pm 0.013}$	$0.525_{\pm 0.009}$	$0.462_{\pm 0.002}$	$0.349_{\pm 0.010}$	$0.674_{\pm 0.010}$	$0.293_{\pm 0.008}$
AstroPTv2 Base	$0.443_{\pm 0.017}$	$0.556_{\pm 0.012}$	$0.502_{\pm 0.006}$	$0.472_{\pm 0.015}$	$0.771_{\pm 0.008}$	$0.336_{\pm 0.005}$
AstroPTv2 Large	$0.453_{\pm 0.007}$	$0.559_{\pm 0.013}$	$0.510_{\pm 0.007}$	$0.531_{\pm 0.012}$	$0.802_{\pm 0.006}$	$0.360_{\pm 0.005}$
ViT Base	$0.359_{\pm 0.006}$	$0.455_{\pm 0.013}$	$0.434_{\pm 0.006}$	$0.372_{\pm 0.012}$	$0.597_{\pm 0.004}$	$0.306_{\pm 0.012}$
ViT Large	$0.369_{\pm 0.007}$	$0.474_{\pm 0.007}$	$0.440_{\pm 0.005}$	$0.408_{\pm 0.010}$	$0.638_{\pm 0.006}$	$0.340_{\pm 0.009}$

Appendix

Table 4: Downstream regression performance (

R^{2}

) when predicting redshift (

z

), stellar mass (

\log M_{\star}

), and specific star formation rate (sSFR) from frozen embeddings of each model, evaluated on 45 000 HSC and JWST galaxies. Values are mean

\pm

standard deviation over 10 k-folds.

Figure 7: Parameter count vs probe performance for our tested physical properties. Spearman’s

\rho

and

p

values are stated for each panel. In all tested cases we find a significant correlation between parameter count and physics probe performance.

Figure 8: Cosine similarity matrices between the redshift ( $z$ ), stellar mas ( $M_{\star}$ ), and sSFR probe weight vectors using HSC images.

Figure 9: Cosine similarity matrices between the redshift ( $z$ ), stellar mas ( $M_{\star}$ ), and sSFR probe weight vectors using JWST images.

		HSC MKNN (%)		JWST MKNN (%)
Model	Params (M)	Teacher	Basket	Teacher	Basket
ViT-S/16	21	$25.0\pm 0.3$	$24.3\pm 0.2$	$27.8\pm 0.3$	$23.0\pm 0.2$
ViT-S+/16	29	$25.7\pm 0.3$	$23.2\pm 0.2$	$27.8\pm 0.3$	$21.5\pm 0.2$
ViT-B/16	86	$26.0\pm 0.3$	$23.1\pm 0.2$	$31.3\pm 0.3$	$22.4\pm 0.2$
ViT-L/16	300	$29.4\pm 0.3$	$18.7\pm 0.2$	$37.4\pm 0.4$	$18.0\pm 0.2$
ViT-H+/16	840	$29.3\pm 0.3$	$15.0\pm 0.2$	$40.5\pm 0.3$	$16.0\pm 0.2$

Appendix

Table 5: DINOv3 MKNN alignment with the ViT-7B/16 teacher and the non-DINOv3 model basket, evaluated on 1.67k galaxies with paired HSC and JWST imaging. Basket alignment is averaged over non-DINOv3 models. Scores are percentages, with uncertainties estimated from 200 draws containing 80% of the sample each.

Figure 10: Physics linear

R^{2}

probing results for the PaliGemma 3B, 10B, and 28B networks on a layer-wise basis. The

k=10

-fold standard deviation is shaded. We demarcate the boundary between the fixed-size SigLIP image encoder and Gemma decoder networks with a gray dashed line.

Model Pairs	JWST	Legacy	HSC	JWST	Legacy	HSC
	MKNN (%)			CKA (%)
AstroPTv2 Small vs Base	47.2	35.5	35.7	96.9	98.4	98.1
AstroPTv2 Base vs Large	51.6	39.4	39.6	97.2	98.3	98.1
CLIP Base vs Large	30.7	5.9	6.2	66.7	69.4	68.8
ConvNeXtv2 Nano vs Tiny	29.6	4.4	4.8	66.3	70.3	70.4
ConvNeXtv2 Tiny vs Base	29.0	3.9	4.2	72.0	68.4	67.7

Appendix

Table 6: Intra-architectural embedding alignment within a model family, measured by MKNN and CKA scores. The PRH predicts that both MKNN and CKA scores will increase as we compare the embeddings of larger model pairs within a model family, since larger models will generate embeddings closer to the Platonic ideal representation.

Panel of Figure 11 — Figure 11: Intra-architectural embedding alignment (MKNN and CKA) vs probe performance for our tested physical properties. Spearman’s $\rho$ and $p$ values are stated for each panel. In all tested cases we find a significant correlation between MKNN and physics probe performance. We do not find a significant correlation between CKA and physics probe performance in any tested case.

	MKNN (%)			CKA (%)
Model	JWST	Legacy	DESI	JWST	Legacy	DESI
AstroPTv2 Small	11.62	3.36	1.25	43.4	86.6	45.66
AstroPTv2 Base	12.60	3.28	1.32	42.0	84.9	45.39
AstroPTv2 Large	14.30	3.84	1.41	41.9	84.9	44.47
CLIP Base	12.88	1.79	1.29	30.59	59.32	33.39
CLIP Large	14.07	2.28	1.24	31.89	72.73	33.85

Appendix

Table 7: Crossmodal embedding alignment between a model’s embeddings of different astronomical modalities and its embeddings of HSC imaging, measured by MKNN and CKA scores. The DESI spectra column uses Specformer embeddings for the DESI side compared against each listed vision model’s HSC image embeddings. The PRH predicts that both MKNN and CKA scores will increase as models within a family grow in size, since larger models should produce embeddings closer to the Platonic ideal representation.

Panel of Figure 12 — Figure 12: Crossmodal embedding alignment between the stated modality and HSC plotted against probe performance for our tested physical properties. ANCOVA $\beta$ and $p$ values are stated for each panel. In all tested cases we find a significant correlation between MKNN and physics probe performance. We only find a significant correlation between CKA and physics probe performance for DESI Spectra vs HSC.

	Physics subspace	Random subspace
CKA (global)	$21\%~[9,\,28]$	$0\%~[-0,\,0]$
MKNN (local)	$7\%~[2,\,12]$	$-7\%~[-12,\,-2]$

Appendix

Table 8: Effect of removing the probe-defined physics subspace from model embeddings. Values give the mean [min, max] percentage decrease in alignment over 21 tested model pairs. Negative values indicate that alignment increased after the intervention. The random control removes a subspace of equal rank.

Panel of Figure 13 — Figure 13: Cross-architectural embedding alignment between the stated modality and HSC plotted against probe performance for our tested physical properties. Spearman’s $\rho$ and $p$ values are stated for each panel. In all tested JWST cases we find a significant correlation between MKNN and physics probe performance. For HSC we find significant or borderline correlations between MKNN and physics probe performance. We only find a significant correlation between CKA and physics probe performance for HSC redshift.

Explore similar work

Sep 29, 2026astro-ph.IM

Gestalt: a meta-foundation model for astronomy

The Platonic Representation Hypothesis predicts that sufficiently scaled foundation models converge on a shared representation of the world. As each non-converged model gives a noisy view of a common structure when passed the same input, we ask whether we can combine models into a representation that outperforms its individual components. We test this on galaxies: we embed images via a basket of 22 frozen foundation models from eight families, whiten each view, and take a randomised SVD of the embedding concatenation. The resulting 1024-dimensional embedding outperforms every basket member on 19/21 of our tested metrics for physical property and galaxy morphology estimation for HSC, JWST, and DESI Legacy Survey imagery. We find that performance rises with basket size and basket architectural diversity, and that the meta-foundation model's performance transfers across astronomical surveys. We conclude that a useful astronomical foundation model can be assembled from existing generalist models with no training required beyond a single unsupervised projection. By leveraging the community's already-spent work, we save a lot of compute: a fresh pre-train of a comparable single-domain model would cost

\mathcal{O}(10^{4}

10^{5})

A100 GPU hours (emitting several tonnes of CO

_2

eq.), whereas assembling Gestalt requires minutes on a single machine.

Michael J. Smith, Shashwat Sourav

AstroAI · Center for Astrophysics | Harvard & Smithsonian · UniverseTBD +1

Astronomical Images Dimensionality Reduction

Ask PDF

Apr 20, 2026cs.CV

Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale

The Platonic Representation Hypothesis posits that neural networks trained on different modalities (e.g., text and images) converge toward a shared representation of reality. If true, this has significant implications for whether modality choice matters at all. In this paper, we show that the evidence for this claim is substantially weaker than subsequent work suggests. The mutual

k

-nearest-neighbor metric used on 1024 text-image pairs in the original study captures only coarse structure. To keep the alignment from collapsing as one scales up the data,

k

has to grow proportionally, undercutting the argument for fine-grained representational convergence. The reported increase in alignment with language model strength saturates for recent models. Moreover, the one-to-one text-image pairing favors alignment, while alignment decreases with non-bijective data. We further find that image and text representations indeed share coarse semantic structure, but neither stronger language models nor richer captions yield fine-grained alignment. Thus, multimodal representations share coarse structure without evidence of convergence to a shared representation -- arguably, full representational convergence would require fine-grained alignment.

A. Sophia Koepke, Daniil Zverev, Shiry Ginosar +1

UC Berkeley · Technical University Munich, MCML · University of Tübingen, Tübingen AI Center +1

Modalities Ridge Convergence

Ask PDF

Apr 27, 2026cs.AI

A systematic evaluation of vision-language models for observational astronomical reasoning tasks

Vision-language models (VLMs) are increasingly proposed as general-purpose tools for scientific data interpretation, yet their reliability on real astronomical observations across diverse modalities remains untested. We present AstroVLBench, a comprehensive benchmark comprising over 4,100 expert-verified instances across five tasks spanning optical imaging, radio interferometry, multi-wavelength photometry, time-domain light curves, and optical spectroscopy. Evaluating six frontier models, we find that performance is strongly modality-dependent: while one model (Gemini 3 Pro) emerges as the most consistently capable across tasks, task-specific strengths vary, and all models substantially underperform domain-specialized methods. Mechanistic ablations reveal that performance depends not only on directing attention to salient visual features but also on grounding those features in physical knowledge. Phenomenological prompts describing what to look for improve accuracy by sharpening model focus, but physical prompts explaining why those features matter perform better overall and yield more balanced classifications with reduced class-specific bias. Consistent with this picture, presenting the underlying one-dimensional measurements directly as numerical tables instead of rendered plots yields up to 13 percentage points improvement. Reasoning quality analysis further demonstrates that, without explicit physical grounding, models may reach correct predictions from phenomenologically plausible cues while providing physically imprecise justifications, establishing that accuracy alone is insufficient for trustworthy scientific deployment. These findings provide the first systematic, multi-modal baselines for VLMs in observational astronomy and identify the specific representation, grounding, and reasoning bottlenecks where current models fail.

Wenke Ren, Hengxiao Guo, Wenwen Zuo +1

Shanghai Astronomical Observatory, Chinese Academy of Sciences, Shanghai, China · Harvard University, Boston, Massachusetts, USA

Scientific Discovery

Ask PDF