Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics

Pablo Mercader-Perez; Carolina Cuesta-Lazaro; Daniel Muthukrishna; Jeroen Audenaert; V. Ashley Villar; David W. Hogg; Marc Huertas-Company; William T. Freeman

astro-ph.IMApr 10, 2026

Learning What's Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics

Authors: Pablo Mercader-Perez, Carolina Cuesta-Lazaro, Daniel Muthukrishna, Jeroen Audenaert, V. Ashley Villar, David W. Hogg, Marc Huertas-Company, William T. Freeman

Organizations: Massachusetts Institute of Technology · Flatiron Institute, Simons Foundation · Institute for Advanced Studies · AstroAI, CfA | Harvard & Smithsonian ·

Abstract

Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. This secondary signal acts as a confounding factor, limiting our ability to extract information about the physics underlying the phenomena we observe. Furthermore, it complicates the combination of observations in heterogeneous or multi-instrument settings. We propose a deep learning framework that leverages overlapping observations, a dual-encoder architecture, and a counterfactual generation objective to disentangle these factors of variation. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference unconfounded by measurement distortions, and instrument-independent similarity search. We demonstrate the effectiveness of our approach on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey as a representative multi-instrument setting. This framework provides a general recipe for scientific and multi-modal self-supervised pretraining: construct training pairs from overlapping observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual generation.

Figures & tables

Figure 1: Counterfactual Reconstruction. The model learns to disentangle intrinsic galaxy properties from instrument systematics by reconstructing an anchor image. Training uses data triplets : an anchor observation (signal $s$ from instrument $i$ ), an instrument-augmented observation ( same source $s$ , a different instrument $i^{\prime}$ ), and a physics-augmented observation (a different source $s^{\prime}$ , same instrument $i$ ). The physics encoder extracts latents from the instrument-augmented pair to capture invariant physical features, while the instrument encoder processes the physics-augmented observation to isolate measurement artifacts. These latent variables jointly condition the flow-matching decoder. Crucially, the anchor image is never fed into the encoders; it is used only as a target for the flow-matching loss, forcing the model to generate the reconstruction counterfactually.

Figure 2: Disentanglement. UMAP projections of the physics (left) and instrument (right) latent spaces. Orange and blue points represent HSC and Legacy images, respectively. Matched markers ( $\triangle$ , $\times$ , $\square$ , $\circ$ ) denote cross-survey pairs. In the physics space , the encoders produce overlapping distributions where cross-survey pairs are mapped to similar coordinates. In contrast, the instrument space shows a clean separation between surveys and pairs, confirming that this space captures survey-specific features rather than intrinsic galaxy properties.

Figure 3: Outlier Detection. Top twelve outliers from each of the three latent spaces. Left: our physics latent reveals morphologically unusual sources (e.g., galaxy mergers), free of artifacts. Middle & Right: anomalies from AION-1 and our instrument latent are dominated by instrumental pathologies (saturated pixels, bands, noise patterns), confirming that our representation separates physical anomalies from instrument systematics.

Figure 4: Probing Latent Disentanglement via Downstream Regression. We report

R^{2}

scores for the prediction of physics-related properties (left) and instrumental properties (right) on four sets of representations: our physics embeddings (blue), our instrument embeddings (red), AION-1 embeddings which treat each survey as an independent modality (green), and a randomly initialized ResNet-18 of the same architecture as our encoders (hatched black). We distinguish physics-related properties and instrument properties, which are grouped by survey. The asymmetry between the two panels demonstrates that our architecture successfully separates physics properties from instrument-specific information.

Figure 5: Instrument-Invariant Search and Rare-Object Discovery. Top (Cross-Survey Retrieval): We search the latent spaces using an HSC observation as query. Physics Space: The pair from the alternate survey is identified as the top-1 neighbor, with additional top-ranked neighbors showing physically similar galaxies from a mixture of both instruments. Notably, the four examples shown could have been found within the top nine nearest neighbors independent of which survey the query came from. Instrument Space: Neighbors in the instrument space show similar noise characteristics independent of the underlying galactic signal. Bottom (Strong Lenses): Nearest neighbor search within the physics latent space using a known strong lens as query, focusing only on the HSC images. The retrieved top neighbors show features suggestive of strong lensing, demonstrating the model’s capacity for rare-object discovery.

Figure 6: Cross-instrument Galaxy Reconstructions. Columns 1–2 (Inputs): The target galaxy observed via an alternate instrument (input to the physics encoder) and a set of up to five different galaxies imaged by the target instrument (input to the instrument encoder). Column 3 (Ground Truth): The original target image. Columns 4–5 (Posterior Samples): Independent samples generated by the decoder, demonstrating the model’s ability to capture noise and uncertainty. Column 6 (Posterior Mean): The pixel-wise mean of the generated posterior samples.

Figure 7: Data-driven Noise Model. Top (Progressive Denoising): Reconstructions of a target galaxy from its Legacy counterpart, varying the instrument conditioning across different percentiles of the HSC SNR distribution. The model successfully generates progressively cleaner images; the average SNR shown is that of the images used to condition the reconstruction. Bottom (Systematic Artifact Removal): By conditioning the physics encoder on an artifact-corrupted HSC image and the instrument encoder on a random set of HSC images, the model successfully removes severe instrumental distortions while preserving the underlying galaxy morphology.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Data Dimensions
Parameter	Value
Image size	$48\times 48$
Input channels	4
Conditioning channels	4
UNet Architecture
Base model channels	128

Appendix

Table 1: Model architecture parameters for the conditional flow matching model with dual ResNet-18 encoders.

Optimization
Parameter	Value
Optimizer	AdamW
Learning rate	$1\times 10^{-4}$
LR scheduler	CosineAnnealingLR
Loss function	Velocity MSE
Flow Matching

Appendix

Table 2: Training configuration and hyperparameters.

Variable	Source	Description
Ellipticity ( $e_{1},e_{2}$ )	HSC catalog	Galaxy shape ellipticity components, measured via the SDSS-style second moments of the surface brightness distribution.
Redshift ( $z$ )	DESI spectra	Spectroscopic redshift.
Stellar mass ( $M_{\star}$ )	PROVABGS	Total mass in stars,, derived from Bayesian spectral energy distribution (SED) fitting to DESI spectra [ 10 ] .
Half-light radius ( $R_{\text{eff}}$ )	HSC catalog	Circularised radius enclosing half of the galaxy’s total light. A measure of galaxy size.
sSFR	PROVABGS	Specific star formation rate $\log(\text{SFR}/M_{\star})$ : the rate at which new stars are forming normalised by the galaxy’s stellar mass.
$\log Z_{\text{Met}}$	PROVABGS	Gas-phase metallicity: the abundance of elements heavier than helium in the galaxy’s gas. Traces the galaxy’s chemical enrichment history.

Appendix

Table 3: Physical parameters used for downstream regression evaluation of the physics latent space.

Variable	Bands	Description
HSC (Hyper Suprime-Cam)
psf_fwhm	$g,i,r,z$	Effective size of the point spread function (arcsec): how much the instrument and atmosphere spread the light from a point source. Analogous to Legacy PSFSIZE . $\dagger$
a	$g,i,r,z$	Milky Way dust extinction $A_{\lambda}$ per band (mag). Equivalent to Legacy EBV scaled by a filter-dependent coefficient: $A_{\lambda}=R_{\lambda}\,E(B-V)$ . $\ddagger$
Legacy Survey (DESI Legacy Imaging Surveys)
NOBS	$g,i,r,z$	Number of exposures contributing to the coadd.
GALDEPTH	$g,i,r,z$	$5\sigma$ detection depth for a round exponential galaxy with $r_{e}=0.45^{\prime\prime}$ (mag). Shallower than PSFDEPTH because galaxy flux is spread over more pixels.

Appendix

Table 4: Instrumental and observational parameters used for downstream regression evaluation of the instrument latent space. All quantities vary spatially across each survey’s footprint.

Property	Probe on cross-matched Legacy-HSC sample (Im)					Literature Reported Values
Property	Ours (Phys)	Ours (Instr)	AION-1-B	Rnd In. ResNet	X-Predict	AION-1-B (Ph+Im)	AstroCLIP (Im)
Phys.-related properties
Ellipticity $e_{2}$	0.934	0.321	0.941	0.616	—	—	—
Ellipticity $e_{1}$	0.918	0.421	0.939	0.613	—	—	—
Redshift $z$	0.801	0.535	0.857	0.514	—	0.930	0.780
$\log M_{\star}$	0.755	0.491	0.818	0.485	—	0.890	0.730

Appendix

Table 5: MLP-probe

R^{2}

on the held-out overlap -MMU sample. Bold = best in row among the five probe methods (the two right-most columns are literature reference values and are not compared against). ‘Ours (Physics)’ and ‘Ours (Instrument)’ use our flow-matching encoder’s

e_{1}

(flat) and

e_{2}

(mean-pooled) latents respectively. ‘AION-1 (Base)’ is the AION-1 Base model [ 19 ] probed identically. ‘Rand. Init. ResNet’ is a frozen randomly-initialised ResNet-18 of the same architecture as our encoders. ‘Cross-predict’ is a fresh ResNet-18 trained from scratch to predict each survey’s instrument properties from the other survey’s raw image (only defined for instrument targets). Reference values: AION-1-B (Ph+Im) from [ 19 ] Table 1; AstroCLIP image-only on Legacy

\{g,r,z\}

from [ 15 ] . Band-averaged where applicable.

Figure 8: Spatial Structure Preservation. Power spectrum (left) and autocorrelation (right) per band for ground-truth and generated images. Each plot shows the true value (blue), the posterior mean from 32 generated samples (orange dashed line), and the $1\sigma$ posterior standard deviation (orange shaded region). Strong agreement across spatial frequencies and pixel separation (lag) distances indicates that the model captures realistic morphological structure and noise properties.

Figure 9: Left: Pixel-wise Z-score distribution of generated posterior samples relative to ground-truth target images, across all pixels and held-out galaxies. Both HSC-anchored and Legacy-anchored reconstructions closely approximate a standard normal distribution, indicating that the posterior mean is approximately unbiased, the posterior variance captures the true prediction error, and that the residual uncertainty is approximately Gaussian. The standard deviations slightly above unity indicate mild overconfidence, more pronounced for HSC-anchored reconstructions, consistent with the greater ambiguity in generating higher-resolution images from lower-resolution conditioning. Right: $R^{2}$ for galaxy ellipticity from a ResNet trained on real HSC images, evaluated on real versus counterfactual HSC images. Near-identical performance shows that existing HSC pipelines can be readily applied to Legacy images through our counterfactual model, extending HSC-quality analysis across the full Legacy footprint.

Flow Architecture
Parameter	Value
Family	Neural Spline Flow (NSF, zuko )
Number of transforms	6
Hidden features per transform	$[64,64]$
Context dimension	0 (unconditional)
Input dimension	64 / 16 (Ours) / 768 (AION-1)

Appendix

Table 6: Normalizing flow architecture and training hyperparameters used for outlier detection.

Astrophysics

See all 29 papers

Apr 18, 2026

AstroSURE: Learning to Remove Noise from Astronomical Images Without Ground Truth Data

In astronomical imaging, the low photon count of exposures necessitates extensive post-processing steps, including contamination removal and denoising. This paper evaluates deep-learning denoising methods that can be trained without clean ground-truth images and assesses their utility for detection11 oriented analysis of astronomical data. We adapt and compare Noise2Noise, Stein's Unbiased Risk Estimator, and blind-spot-based methods using synthetic data and real observations from the Hubble Space Telescope (HST) and the Canada-France-Hawaii Telescope (CFHT). Performance is evaluated using object-detection metrics, including correct detection rate and false alarm rate, together with image-based metrics and pixel-distribution diagnostics. The results show that these methods can improve faint-source detectability relative to the original noisy images, with encouraging gains on HST data after domain-consistent initialization, while transfer to CFHT data is more limited, highlighting the importance of instrument/domain similarity for unsupervised adaptation.

Omid Vaheb, Sebastien Fabbro, Stark Draper

Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, 10 King’s College Road, Toronto, ON M5S 3G4, Canada · National Research Council Herzberg Astronomy an Astrophysics Research Centre, 5071 West Saanich Road, Victoria, BC V9E 2E7, Canada · Department of Computer Science, University of British Columbia, Vancouver, BC, V6T 1Z4, Canada

Self-Supervised Image Denoising Noise Robustness Image Denoising Astrophysics

Ask PDF

Feb 19, 2026

Deeper detection limits in astronomical imaging using self-supervised spatiotemporal denoising

The detection limit of astronomical imaging observations is limited by several noise sources. Some of that noise is correlated between neighbouring image pixels and exposures, so in principle could be learned and corrected. We present an astronomical self-supervised transformer-based denoising algorithm (ASTERIS), that integrates spatiotemporal information across multiple exposures. Benchmarking on mock data indicates that ASTERIS improves detection limits by 1.0 magnitude at 90% completeness and purity, while preserving the point spread function and photometric accuracy. Observational validation using data from the James Webb Space Telescope (JWST) and Subaru telescope identifies previously undetectable features, including low-surface-brightness galaxy structures and gravitationally-lensed arcs. Applied to deep JWST images, ASTERIS identifies three times more redshift > 9 galaxy candidates, with rest-frame ultraviolet luminosity 1.0 magnitude fainter, than previous methods.

Yuduo Guo, Hao Zhang, Mingyu Li +10

Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, China · Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing, China · Department of Astronomy, Tsinghua University, Beijing, China +6

Image Denoising Self-Supervised Image Denoising Astrophysics Transformer

Ask PDF

Jan 30, 2026

Denoising the Deep Sky: Physics-Based CCD Noise Formation for Astronomical Imaging

Astronomical imaging remains noise-limited under practical observing conditions. Standard calibration pipelines remove structured artifacts but largely leave stochastic noise unresolved. Although learning-based denoising has shown strong potential, progress is constrained by scarce paired training data and the requirement for physically interpretable models in scientific workflows. We propose a physics-based noise synthesis framework tailored to CCD noise formation in the telescope. The pipeline models photon shot noise, photo-response non-uniformity, dark-current noise, readout effects, and localized outliers arising from cosmic-ray hits and hot pixels. To obtain low-noise inputs for synthesis, we stack multiple unregistered exposures to produce high-SNR bases. Realistic noisy counterparts synthesized from these bases using our noise model enable the construction of abundant paired datasets for supervised learning. Extensive experiments on our real-world multi-band dataset curated from two ground-based telescopes demonstrate the effectiveness of our framework in both photometric and scientific accuracy.

Shuhong Liu, Xining Ge, Ziying Gu +7

The University of Tokyo · I2WM · Shanghai Astronomical Observatory +2

Synthetic Data Augmentation Image Denoising Astrophysics

Ask PDF

Abstract

Astrophysics

Explore similar work

AstroSURE: Learning to Remove Noise from Astronomical Images Without Ground Truth Data

Deeper detection limits in astronomical imaging using self-supervised spatiotemporal denoising

Denoising the Deep Sky: Physics-Based CCD Noise Formation for Astronomical Imaging