A survey detection channel overrides the pixels in an astronomical foundation model, and biases tomographic mean redshifts

Ihor Kendiukhov

cs.AIAug 23, 2026

A survey detection channel overrides the pixels in an astronomical foundation model, and biases tomographic mean redshifts

Authors: Ihor Kendiukhov

Organizations: Independent researcher

Abstract

Foundation models for astronomy are trained on survey pixels together with the catalogue products derived from those pixels. Those catalogues are incomplete at a measurable rate, and a model trained on both inherits that incompleteness as a systematic. We audit AION-1, a 39-modality transformer trained on more than 200 million objects, using causal interventions on its inputs. Holding the image tokens byte-identical and editing only the survey segmentation map changes every quantity the model reports -- flux, size, ellipticity, redshift -- by 110-4400 times a matched placebo. The mechanism is detection gating, presence at the field centre (r = 0.47), not the light the mask encloses (r = 0.30); across 322 real blends the model ignores how the pipeline partitioned the light (R = -0.006). Nor is the preference specific to that channel: contradicted catalogue photometry leaves the model nine times worse than supplying no metadata at all. The Legacy Survey pipeline leaves 3.68% of targets with no segment covering their position. Propagating that rate, with a miss represented by the fields the pipeline actually returns, shifts tomographic mean redshifts by a median 0.71 times the LSST DESC requirement over 40 assignments and exceeds it in 12; observed positional errors take the worst bin to 8.3 times. Drawing the misses by their measured magnitude dependence rather than uniformly does not change it. Spectroscopy removes the effect, withholding the detection channel removes it at no measurable cost, and the effect grows with model scale. Two further limits lie in the tokeniser: its image codec resolves 28 effective states on source patches against 934 for the spectrum codec, and the redshift readout is quantisation-limited. Sparse dictionaries are unreliable causal handles: across 15, recovery spans 26-75% and moves up to 18 points on the seed alone.

Figures & tables

Readout	Mask displaced $25\text{\,}\mathrm{p}\mathrm{x}$	Mask swapped	Placebo
flux $g$	0.630	0.110	0.0004
flux $z$	20.55	9.83	0.0047
size $R$	1.313	0.718	0.0025
ellipticity $e_{1}$	0.435	0.931	0.0036
redshift	0.580	0.312	0.0028

Table 1: Effect of altering only the segmentation map, in

\sigma

of each readout’s own spread. Image pixels are byte-identical across all columns.

n=480

, imaging input.

Figure 1: The detection gate. (a) Median

g

-band flux estimate under mask manipulations with the image pixels held byte-identical; green is another galaxy’s genuine mask (fully on-manifold), red is an erased mask (off-manifold, discounted), grey are controls. (b) Mechanism discriminator: within object and across nine intervention arms, the change in prediction tracks central mask coverage better than the light the mask encloses. (c) Every readout is gated, at 110–4400 times the placebo.

Figure 2: Real blends, no synthetic geometry: only the choice of which pipeline segments the mask includes. (a)

R=\Delta

readout

/\Delta

light is zero, not one: the model ignores how the pipeline partitioned the light. (b) The null holds across blend severity, including systems where the central segment holds under 40 % of the light.

Input configuration	$\|\Delta\mathrm{mag}\|$ vs catalogue	Shift vs clean arm
imaging only	0.0665	—
imaging + true Tractor $r,i,z$	0.0339	—
imaging + wrong $r,i,z$	0.6210	0.6121
imaging + true segmap	0.0693	—
imaging + wrong segmap	0.1682	0.1322
placebo (RA moved 250 codes)	0.0350	0.0005

Table 2: Metadata preference. Pixels identical; the target band (

g

) is never supplied.

n=480

Vulnerability (mask swapped)	aion-base ( $314$ M)	aion-large ( $860$ M)	Ratio
flux $g$	0.147 $[0.127,0.166]$ mag	0.237 $[0.170,0.267]$ mag	1.62
redshift	0.0327 $[0.0266,0.0361]$	0.0441 $[0.0377,0.0518]$	1.35

Table 3: The gate against model scale,

n=240

identical objects through both checkpoints.

	Scenario 1: measured miss rate only		Scenario 2: + displacements
Bin	Imaging only	+ spectra	Imaging only	+ spectra
$[0.00,0.15)$	0.00126	0.00000	0.02479	0.00111
$[0.15,0.25)$	0.00204	0.00002	0.00151	0.00000
$[0.25,0.35)$	0.00047	0.00000	0.00409	0.00008
$[0.35,1.00)$	0.00024	0.00000	0.01797	0.00004
worst / requirement	$0.68\times$	$0.006\times$	$8.3\times$	$0.37\times$

Table 4: Induced tomographic bias

|\Delta\langle z\rangle|/(1+z)

, against the requirement 0.003. A detection miss is represented by a segmentation field transplanted from the 184 measured misses, not by an all-zero field. Redshifts read with the posterior median. Bold exceeds the requirement.

n=960

. Per-bin values are the single realisation plotted in Fig. 3 b; the Scenario-1 ensemble over 40 miss assignments has worst-bin median

0.71\times

the requirement.

Figure 3: Incidence and consequence. (a) Distribution of central-segment centroid offsets measured from 5000 real Legacy Survey segmentation maps. (b) Induced bias in tomographic mean redshift against the LSST DESC requirement. The measured detection-miss rate alone reaches two-thirds of the requirement without breaching it; adding positional errors of the observed magnitude breaches it in three of four bins, and adding spectra removes the effect. The shaded band is the 16–84 % range of the imaging, measured-miss-rate curve over 40 miss assignments; the solid curve is one realisation from within it. The grey curve repeats the imaging-only propagation with an all-zero segmentation field instead of an empirically sampled one: the sensitivity quantified in § 4 .

Configuration	flux $g$ ( $\|\Delta\mathrm{mag}\|$ )	redshift $\|\Delta z\|$	Swap vulnerability
imaging only	0.0665	0.1169 $[0.1143,0.1200]$	0
imaging + real mask	0.0693	0.1273 $[0.1253,0.1314]$	0.132 mag / 0.031
imaging + generic disc	0.2215	0.1566 $[0.1510,0.1599]$	0
+ spectra, no mask	0.0534	0.0034 $[0.0033,0.0035]$	0
+ spectra + real mask	0.0575	0.0038 $[0.0036,0.0039]$	0.086 mag / 0.0004

Table 5: Cost of withholding the detection channel, against the vulnerability it removes.

n=480

. Bracketed values are bootstrap 16–84 % intervals; flux

g

intervals are shown in Fig. 4 a. Zero entries in the last column are not measured—no swap arm is run for configurations without a real mask.

Figure 4: (a) Withholding the detection mask costs nothing measurable, while a generic substitute mask is far worse than either option. (b) The preference is general: corrupted catalogue photometry moves the readout further than a corrupted segmentation map, and both leave the model worse than the dashed line, which is the error using no metadata at all.

	Codes emitted	Effective size (perplexity)
image, source patches	237 / 4375	28.2
image, sky patches	356 / 4375	5.2
spectrum	1024 / 1024	934
image, HSC (864 k tokens)	312 / 4375	2.5 (all-patch; cf. Legacy 7)

Table 6: Measured codebook utilisation. Perplexity is the effective number of states.

Figure 5: Tokeniser limits. (a) Effective codebook size: the image codec resolves $\sim$ 28 states on source patches against 934 for the spectrum codec. (b) The most-used image codes lie on a single brightness axis. (c) After removing the trivial flux scaling, relative structure and colour remain largely predicted by flux.

Figure 6: (a) Rest-frame stacked ablation profile of the redshift posterior, against the observed-frame version. The rest-frame peak at H

\alpha

and the secondary features are absent in the observed frame. (b) Causal steering at matched intervention norm, for all 15 dictionaries of the sweep (thin red) against the baselines. At the pre-registered

\alpha=4

(marker) every dictionary beats the random control, two of fifteen beat difference-in-means, and the spread across seeds at fixed width and sparsity reaches 18.4 points. PCA is drawn at

k=32

;

k=16

and 64 give 68.1 and 73.4 %. At

\alpha\geq 6

large-norm perturbations inflate the readout non-specifically—the random control itself reaches 48 %—so peak recovery is not the statistic.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

$r$ magnitude	$n$	$p(\text{miss})$ (%)
$13.37$ – $18.05$	605	0.33
$18.05$ – $18.67$	604	0.83
$18.67$ – $19.06$	605	0.99
$19.06$ – $19.34$	604	1.99
$19.34$ – $19.55$	604	3.81
$19.55$ – $19.81$	605	6.12

Appendix

Table 7: Detection-miss rate against

r

-band magnitude, over 5000 real cutouts (184 misses). The trend is monotone across seven of eight bins; missed objects are a median

0.49\text{\,}\mathrm{m}\mathrm{a}\mathrm{g}

fainter than detected ones, at Mann–Whitney

p=5\times 10^{-24}

Width	$k$	FVU	FVU (PCA, same $k$ )	Alive (%)	Recovery (%)
2048	16	0.065	0.350	93.7	38.7
2048	32	0.049	0.242	99.9	62.8
2048	64	0.039	0.149	100.0	63.0
8192	16	0.052	0.350	51.6	37.4
8192	32	0.032	0.242	89.0	31.5
8192	64	0.023	0.149	99.1	53.5

Appendix

Table 8: All 15 dictionaries. Recovery is the fraction of the gate-induced flux collapse restored at

\alpha=4

; bold exceeds difference-in-means (64.3 %). The first nine rows are the width-by-

k

factorial at seed 0; the last six vary the seed at

k=32

Explore similar work

Jun 5, 2026

Beyond Point Estimates: Benchmarking Uncertainty Quantification Methods on the AION-1 Astronomical Foundation Model

Foundation models for astronomical surveys offer powerful learned representations that can be transferred to downstream regression tasks such as galaxy property estimation. However, point predictions alone are insufficient for scientific inference; reliable uncertainty quantification (UQ) is essential. We compare seven UQ methods on galaxy property regression using frozen AION-1 foundation-model embeddings, predicting redshift, stellar mass, stellar-population age, gas-phase metallicity, and specific star-formation rate, from Legacy Survey photometry/imaging and DESI spectra, with PROVABGS-derived labels. Distribution-free conformal methods achieve marginal coverage within

\sim

1,pp of the nominal 90% across all properties, while non-conformal baselines (Deep Ensembles, MC~Dropout) fail to calibrate reliably. Among conformal approaches, Conformalized Quantile Regression (CQR) delivers the best coverage in the bin with the poorest model predictions. More importantly, only the Locally Valid and Discriminative (LVD) framework -- particularly when operating on AION-1 embeddings -- also provides finite-sample local validity, producing intervals that adapt to each galaxy's local prediction difficulty rather than relying on marginal guarantees alone. These results establish conformal prediction, and LVD in particular, as the preferred UQ framework for uncertainty-aware inference on foundation-model embeddings in astrophysics.

Karla Tame-Narvaez, Aleksandra Ćiprijanović, Shubhendu Trivedi

Scientific Computing Division Fermi National Accelerator Laboratory Batavia, IL 60510 · Department of Astronomy and Astrophysics University of Chicago Chicago, IL 60637 · Fermi National Accelerator Laboratory Batavia, IL 60510 +1

Uncertainty Quantification AI-Assisted Scientific Research Conformal Prediction Regression Prediction Intervals

Ask PDF

Sep 29, 2026

Gestalt: a meta-foundation model for astronomy

The Platonic Representation Hypothesis predicts that sufficiently scaled foundation models converge on a shared representation of the world. As each non-converged model gives a noisy view of a common structure when passed the same input, we ask whether we can combine models into a representation that outperforms its individual components. We test this on galaxies: we embed images via a basket of 22 frozen foundation models from eight families, whiten each view, and take a randomised SVD of the embedding concatenation. The resulting 1024-dimensional embedding outperforms every basket member on 19/21 of our tested metrics for physical property and galaxy morphology estimation for HSC, JWST, and DESI Legacy Survey imagery. We find that performance rises with basket size and basket architectural diversity, and that the meta-foundation model's performance transfers across astronomical surveys. We conclude that a useful astronomical foundation model can be assembled from existing generalist models with no training required beyond a single unsupervised projection. By leveraging the community's already-spent work, we save a lot of compute: a fresh pre-train of a comparable single-domain model would cost

\mathcal{O}(10^{4}

10^{5})

A100 GPU hours (emitting several tonnes of CO

_2

eq.), whereas assembling Gestalt requires minutes on a single machine.

Michael J. Smith, Shashwat Sourav

AstroAI · Center for Astrophysics | Harvard & Smithsonian · UniverseTBD +1

Ensemble Learning AI-Assisted Scientific Research

Ask PDF

Feb 19, 2026

Deeper detection limits in astronomical imaging using self-supervised spatiotemporal denoising

The detection limit of astronomical imaging observations is limited by several noise sources. Some of that noise is correlated between neighbouring image pixels and exposures, so in principle could be learned and corrected. We present an astronomical self-supervised transformer-based denoising algorithm (ASTERIS), that integrates spatiotemporal information across multiple exposures. Benchmarking on mock data indicates that ASTERIS improves detection limits by 1.0 magnitude at 90% completeness and purity, while preserving the point spread function and photometric accuracy. Observational validation using data from the James Webb Space Telescope (JWST) and Subaru telescope identifies previously undetectable features, including low-surface-brightness galaxy structures and gravitationally-lensed arcs. Applied to deep JWST images, ASTERIS identifies three times more redshift > 9 galaxy candidates, with rest-frame ultraviolet luminosity 1.0 magnitude fainter, than previous methods.

Yuduo Guo, Hao Zhang, Mingyu Li +10

Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing, China · Department of Astronomy, Tsinghua University, Beijing, China +6

Image Denoising Self-Supervised Image Denoising Transformer

Ask PDF