Gestalt: a meta-foundation model for astronomy

Michael J. Smith; Shashwat Sourav

astro-ph.IMSep 29, 2026

Gestalt: a meta-foundation model for astronomy

Authors: Michael J. Smith, Shashwat Sourav

Organizations: AstroAI · Center for Astrophysics | Harvard & Smithsonian · UniverseTBD · Washington University in St. Louis

Abstract

The Platonic Representation Hypothesis predicts that sufficiently scaled foundation models converge on a shared representation of the world. As each non-converged model gives a noisy view of a common structure when passed the same input, we ask whether we can combine models into a representation that outperforms its individual components. We test this on galaxies: we embed images via a basket of 22 frozen foundation models from eight families, whiten each view, and take a randomised SVD of the embedding concatenation. The resulting 1024-dimensional embedding outperforms every basket member on 19/21 of our tested metrics for physical property and galaxy morphology estimation for HSC, JWST, and DESI Legacy Survey imagery. We find that performance rises with basket size and basket architectural diversity, and that the meta-foundation model's performance transfers across astronomical surveys. We conclude that a useful astronomical foundation model can be assembled from existing generalist models with no training required beyond a single unsupervised projection. By leveraging the community's already-spent work, we save a lot of compute: a fresh pre-train of a comparable single-domain model would cost $\mathcal{O}(10^{4}$ -- $10^{5})$ A100 GPU hours (emitting several tonnes of CO $_2$ eq.), whereas assembling Gestalt requires minutes on a single machine.

Figures & tables

Figure 1 : The Gestalt meta-model.

Family	Parameter counts	Pooled-output widths $d_{m}$
AstroPT ( Smith et al., 2024 )	15M, 95M, 850M	384, 768, 2048
I-JEPA ( Assran et al., 2023 )	632M, 1B	1280, 1408
V-JEPA-2 ( Assran et al., 2025 )	300M, 600M, 1B	1024, 1280, 1408
ViT ( Dosovitskiy et al., 2020 )	86M, 304M, 632M	768, 1024, 1280
ViT-MAE ( He et al., 2022 )	86M, 304M, 632M	768, 1024, 1280
CLIP ( Radford et al., 2021 )	86M, 304M	512, 768

Table 1 : Our tested basket containing 22 frozen checkpoints across eight families. ‘

d_{m}

’ is the model’s pooled-output width; counts and widths correspond left-to-right within each row.

dataset	target	metric	Gestalt	concat $\to$ PCA	best single
HSC COSMOS-Web	$z$	$R^{2}$	0.518 $\pm$ .008	0.482 $\pm$ .008	0.453 $\pm$ .008 (AstroPT 850M)
HSC COSMOS-Web	$\log M_{\star}$	$R^{2}$	0.604 $\pm$ .012	0.577 $\pm$ .009	0.560 $\pm$ .013 (V-JEPA-2 1B)
HSC COSMOS-Web	sSFR	$R^{2}$	0.581 $\pm$ .015	0.555 $\pm$ .007	0.530 $\pm$ .008 (V-JEPA-2 1B)
JWST COSMOS-Web	$z$	$R^{2}$	0.658 $\pm$ .014	0.595 $\pm$ .013	0.565 $\pm$ .012 (V-JEPA-2 1B)
JWST COSMOS-Web	$\log M_{\star}$	$R^{2}$	0.848 $\pm$ .003	0.815 $\pm$ .004	0.810 $\pm$ .004 (V-JEPA-2 1B)
JWST COSMOS-Web	sSFR	$R^{2}$	0.482 $\pm$ .007	0.458 $\pm$ .004	0.457 $\pm$ .010 (V-JEPA-2 1B)

Table 2 : Linear-probe scores (mean

\pm

standard deviation across five random train–test splits); bold marks the highest mean in each row. For Smith42/galaxies , each split is summarized by the median over 13 targets, with the best single using the fixed model with the highest value of these split-wise medians. Full results are in the appendix.

Panel of Figure 2 — Figure 2 : Experimental results investigating model performance (a), basket size scaling (b), physics subspace composition (c), and cross-survey transfer (d).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	$z$ (HSC)	$\log M_{\star}$ (HSC)	sSFR (HSC)	$z$ (JWST)	$\log M_{\star}$ (JWST)	sSFR (JWST)
Gestalt	$\mathbf{0.518\pm 0.008}$	$\mathbf{0.604\pm 0.012}$	$\mathbf{0.581\pm 0.015}$	$\mathbf{0.658\pm 0.014}$	$\mathbf{0.848\pm 0.003}$	$\mathbf{0.482\pm 0.007}$
Basket (concat $\to$ PCA)	$0.482\pm 0.008$	$0.577\pm 0.009$	$0.555\pm 0.007$	$0.595\pm 0.013$	$0.815\pm 0.004$	$0.458\pm 0.004$
AstroPT 15M	$0.387\pm 0.014$	$0.525\pm 0.010$	$0.462\pm 0.003$	$0.349\pm 0.011$	$0.674\pm 0.011$	$0.293\pm 0.009$
AstroPT 95M	$0.443\pm 0.019$	$0.556\pm 0.013$	$0.502\pm 0.007$	$0.472\pm 0.016$	$0.771\pm 0.009$	$0.336\pm 0.005$
AstroPT 850M	$0.453\pm 0.008$	$0.559\pm 0.014$	$0.510\pm 0.008$	$0.531\pm 0.013$	$0.802\pm 0.007$	$0.360\pm 0.005$
CLIP 86M	$0.433\pm 0.010$	$0.517\pm 0.009$	$0.511\pm 0.007$	$0.473\pm 0.010$	$0.706\pm 0.003$	$0.358\pm 0.012$

Appendix

Table 3 : COSMOS-Web HSC and JWST linear-probe

R^{2}

. Best per column in bold. Values use five repeated random 40 000/5 000 probe splits.

	GZ10 class	$z_{\rm phot}$
Gestalt	$0.708\pm 0.009$	$\mathbf{0.758\pm 0.008}$
Basket (concat $\to$ PCA)	$0.674\pm 0.009$	$0.729\pm 0.010$
AstroPT 15M	$0.424\pm 0.007$	$0.514\pm 0.009$
AstroPT 95M	$0.471\pm 0.007$	$0.584\pm 0.009$
AstroPT 850M	$0.480\pm 0.007$	$0.630\pm 0.017$
CLIP 86M	$0.680\pm 0.007$	$0.664\pm 0.009$

Appendix

Table 4 : GZ10 linear-probe scores: macro

F_{1}

for 10-way morphology and

R^{2}

for photometric redshift. Values use five repeated random probe splits, with 2 500 test and 15 300 training objects per split.

Modality	Property	Subset	$k{=}2$	$k{=}4$	$k{=}8$	$k{=}16$	$k{=}22$
HSC	$z_{\rm phot}$	random	$0.420\pm 0.029$	$0.462\pm 0.019$	$0.491\pm 0.013$	$0.512\pm 0.009$	$0.519\pm 0.008$
		one/family	$0.439\pm 0.019$	$0.468\pm 0.014$	$0.500\pm 0.009$	–	–
	$\log M_{\star}$	random	$0.521\pm 0.026$	$0.559\pm 0.016$	$0.582\pm 0.011$	$0.598\pm 0.011$	$0.604\pm 0.012$
		one/family	$0.541\pm 0.019$	$0.563\pm 0.014$	$0.589\pm 0.011$	–	–
	sSFR	random	$0.495\pm 0.027$	$0.533\pm 0.021$	$0.557\pm 0.012$	$0.575\pm 0.011$	$0.581\pm 0.016$
		one/family	$0.511\pm 0.017$	$0.540\pm 0.013$	$0.565\pm 0.011$	–	–

Appendix

Table 6 : COSMOS-Web basket-pruning

R^{2}

vs basket size

k

. Each probe split is 40 000/5 000. The full-basket value appears at

k{=}22

for the random series; one-per-family is undefined for

k{>}8

because the basket contains eight model families.

Figure 3 : Single-source transfer across target corpora. The two coloured square matrices have four target-metric rows and four source-corpus columns; cells show mean $\pm$ standard deviation over five probe seeds, with in-domain fits outlined. The HSC, JWST, and Smith42/galaxies rows summarize regression $R^{2}$ over target tasks; GZ10 contributes its morphology macro- $F_{1}$ row. Each single-source fit uses the first 10 000 rows and each target probe uses a 2 500-object test holdout.

Explore similar work

The Platonic Universe: Do Foundation Models See the Same Sky?
Sep 23, 2025UniverseTBD, :, Trinidad Borrell +11Astronomical Images Foundation Model
Independent Researcher · AstroAI · Harvard-Smithsonian CfA +4
A General-Purpose VLM Can Teach an Astronomy Foundation Model to Better Recognize Galaxy Morphology
Aug 3, 2026Dichang Zhang, Jiaqi Deng, Yixuan Shao +7Astronomical Images Morphology
Department of Computer Science Stony Brook University Stony Brook, New York, USA · School of Computer Science University of Technology Sydney Sydney, New South Wales, Australia · Futurewei Technologies San Jose, California, USA
TESSERA v2: Scaling Pixel-wise Earth Foundation Models
Jul 4, 2026Zhengpeng Feng, Sadiq Jaffer, Ira Shokar +10Wireless Foundation Models Model Size
University of Cambridge · 2NVIDIA · 3dClimate Labs

Sep 23, 2025, astro-ph.IMJ/K move · Enter open · S save

The Platonic Universe: Do Foundation Models See the Same Sky?

UniverseTBD, :, Trinidad Borrell, Steven Dillmann+10

Independent Researcher · AstroAI · Harvard-Smithsonian CfA+4

Astronomical Images Foundation Model Multimodal Representations Continual Fine-Tuning

Open paper AskPDF arXiv

We investigate when foundation models converge towards shared representations, and how this convergence depends on model capacity, training regime, and model architecture. We take a `science-for-AI' approach, using astronomy as an experimental instrument to test the Platonic Representation Hypothesis and its Aristotelian refinement against an external physical reference. The historical success of astrophysics is evidence that a compact, modality-invariant description of galaxy observables exists, and so representation convergence toward reality should be measurable against the physical parameters astronomers already use. Given this framework, we evaluate eleven foundation model families (spanning classification, self-distillation, joint-embedding prediction, autoencoding, vision-language pre-training, and astro-specific architectures from

\mathcal{O}

(10M)

{\to}\mathcal{O}

(10B) parameters) on crossmatched JWST, HSC, and Legacy imagery, and DESI spectroscopy. All models are evaluated frozen, with no astronomy-specific fine-tuning. We probe redshift, stellar mass, and sSFR via linear probes, and local (MKNN) and global (CKA) embedding geometry within families, between modalities, and across architectures. We find that physics performance scales predictably with capacity; probe directions align consistently with expected astrophysical correlations and selection effects; and local (not global) embedding alignment tracks physics performance, including between DESI spectra and HSC imagery---modalities that share essentially no low-level statistics. Our results support the ARH over the strict PRH, demonstrate astronomy's value as an experimental framework for neural representation learning, and suggest that astro-foundation models can build on general-purpose pre-trained architectures, capitalizing on the broader open machine learning community's already-spent computational investment.

Abstract

Explore similar work

The Platonic Universe: Do Foundation Models See the Same Sky?

A General-Purpose VLM Can Teach an Astronomy Foundation Model to Better Recognize Galaxy Morphology

TESSERA v2: Scaling Pixel-wise Earth Foundation Models