Catching the Imposter: Self-Supervised Learning of Physical Coherence with Cross-Entity Feature Permutations

Aleksei Rozanov; Arvind Renganathan; Vipin Kumar

cs.LGAug 14, 2026

Catching the Imposter: Self-Supervised Learning of Physical Coherence with Cross-Entity Feature Permutations

Authors: Aleksei Rozanov, Arvind Renganathan, Vipin Kumar

Organizations: Department of Computer Science & Engineering, University of Minnesota – Twin Cities

Abstract

Scientific data often describe entities whose features are jointly governed by the laws of physics, yet existing self-supervised learning (SSL) objectives largely ignore this physical coherence. We introduce imposter, a discriminative pretext task that replaces subsets of an entity's features with real observations donated by another entity and trains the encoder to identify the swapped features. Because every donated value is individually plausible, the task can only be solved by learning cross-feature physical dependencies. We evaluate the proposed objectives on global ERA5-Land reanalysis data using 21 environmental variables and assess the learned representations on seven downstream tasks spanning climate classification, carbon flux estimation, and streamflow prediction. Our study includes, to our knowledge, the first systematic comparison of self-supervised objectives for land-surface modeling under a shared architecture and pre-training budget. We find that the most effective pretext task depends on the downstream task family rather than any single objective's superiority, and that imposter provides complementary information when combined with existing SSL objectives. These results suggest that physical coherence is a valuable new source of self-supervision for scientific foundation models.

Figures & tables

Figure 1: Overview of the imposter pretext task. Given two entities representing temperate forest and desert ecosystems, the algorithm performs feature-level swap to detect incoherent features during the training stage.

	Linear Probe		CAMELS		Carbon Flux			Summary	Repr. Quality
Method	LP-30 $\uparrow$	LP-5 $\uparrow$	SF $R^{2}\uparrow$	Basin RMSE $\downarrow$	GPP $R^{2}\uparrow$	RECO $R^{2}\uparrow$	NEE $R^{2}\uparrow$	Mean Rank $\downarrow$	RankMe $\uparrow$	$\alpha$ ReQ $\downarrow$
Contrastive	0.859* $\pm$ 0.003	0.927 $\pm$ 0.001	0.370 $\pm$ 0.005	0.600 $\pm$ 0.009	0.416 $\pm$ 0.031	0.300 $\pm$ 0.056	0.135 $\pm$ 0.036	4.00	75.8 † $\pm$ 0.49	1.925 $\pm$ 0.058
NTP	0.657 $\pm$ 0.003	0.840 $\pm$ 0.002	0.352 $\pm$ 0.004	0.720 $\pm$ 0.014	0.489 † $\pm$ 0.026	0.447 $\pm$ 0.032	0.216 † $\pm$ 0.011	4.00	51.5 $\pm$ 4.18	2.238 $\pm$ 0.078
Feat-MAE	0.720 $\pm$ 0.004	0.877 $\pm$ 0.001	0.378 † $\pm$ 0.003	0.681 † $\pm$ 0.006	0.499 $\pm$ 0.027	0.418 $\pm$ 0.029	0.211 $\pm$ 0.022	2.79	70.9 $\pm$ 1.60	1.913 $\pm$ 0.074
Temp-MAE	0.657 $\pm$ 0.042	0.850 $\pm$ 0.017	0.381 $\pm$ 0.004	0.687 $\pm$ 0.009	0.472 $\pm$ 0.037	0.418 $\pm$ 0.041	0.195 $\pm$ 0.028	4.21	54.3 $\pm$ 3.35	2.179 $\pm$ 0.087
FT-MAE	0.633 $\pm$ 0.003	0.837 $\pm$ 0.003	0.384 $\pm$ 0.003	0.704 $\pm$ 0.014	0.472 $\pm$ 0.015	0.338 $\pm$ 0.016	0.203 $\pm$ 0.005	4.86	59.3 $\pm$ 0.29	2.044 $\pm$ 0.005

Table 1: Downstream evaluation of individual objectives. Linear probe accuracy reported for 5-class and 30-class Köppen classification. Streamflow (SF), basin attributes, and carbon flux (GPP, RECO, NEE) scores are median

R^{2}

/RMSE across basins/sites. RankMe and

\alpha

ReQ measure representation quality. All results reported as mean

\pm

std over 3 seeds. Bold scores represent the best score in the column, whereas

\dagger

shows the second best in the column. ∗ indicates the best score achieved across all objectives and experiments throughout this work.

	Linear Probe		CAMELS		Carbon Flux
Method	LP-30 $\uparrow$	LP-5 $\uparrow$	SF $R^{2}\uparrow$	Basin RMSE $\downarrow$	GPP $R^{2}\uparrow$	RECO $R^{2}\uparrow$	NEE $R^{2}\uparrow$
Imp. + Contrastive	-1.5%	+0.1%*	+10.8%*	-13.5%	+8.9%	+8.3%	+50.4%
Multi-Imp. + Contrastive	-1.7%	+0.1%	+8.4%	-18.5%	+5.3%	+14.7%	+54.8%
Imp. + NTP	+12.3%	+5.4%	+7.7%	-14.0%	+0.6%	-9.2%	+4.6%
Multi-Imp. + NTP	+14.5%	+6.5%	+4.3%	-20.3%	+0.2%	+0.7%*	+10.2%*
Imp. + Feat-MAE	+4.7%	+2.1%	+4.5%	-14.5%	-0.2%	-3.6%	+2.8%

Table 2: Percentage improvement of concatenated representations over the base method. Green indicates improvement, red indicates degradation. ∗ indicates the best score achieved across all objectives and experiments throughout this work.

	Linear Probe		CAMELS		Carbon Flux
Method	LP-30 $\uparrow$	LP-5 $\uparrow$	SF $R^{2}\uparrow$	Basin RMSE $\downarrow$	GPP $R^{2}\uparrow$	RECO $R^{2}\uparrow$	NEE $R^{2}\uparrow$
Imp. + Contrastive	-1.9%	-0.5%	+1.9%	+1.5%	+5.3%	+3.0%	+27.4%
Multi-Imp. + Contrastive	-0.9%	-0.1%	-2.4%	+1.8%*	+9.9%	-5.0%	+9.6%
Imp. + NTP	+22.4%	+8.9%	+3.7%	+6.8%	+3.5%	-17.0%	-0.5%
Multi-Imp. + NTP	+23.1%	+9.4%	+4.5%	+4.2%	-1.6%	-10.7%	-19.9%
Imp. + Feat-MAE	+9.2%	+3.4%	-1.1%	-9.8%	+4.0%*	-5.3%	+11.8%

Table 3: Percentage improvement of joint objectives over the base method. Green indicates improvement, red indicates degradation. ∗ indicates the best score achieved across all objectives and experiments throughout this work.

Sched.	$r$	LP-30 $\uparrow$	LP-5 $\uparrow$	SF $\uparrow$	Basin $\downarrow$	GPP $\uparrow$	RECO $\uparrow$	NEE $\uparrow$	Rank $\downarrow$	LP-30 $\uparrow$	LP-5 $\uparrow$	SF $\uparrow$	Basin $\downarrow$	GPP $\uparrow$	RECO $\uparrow$	NEE $\uparrow$	Rank $\downarrow$
		Imposter								Multi-Imposter
Const.	0.25	0.725	0.873	0.316	0.646	0.435	0.374	0.171	73	0.724	0.875	0.357	0.645	0.449	0.419	0.212	49
	0.50	0.767	0.902	0.353	0.654	0.446	0.441	0.226	36	0.761	0.891	0.343	0.642	0.448	0.433	0.217	41
	0.75	0.735	0.874	0.332	0.676	0.443	0.394	0.163	66	0.778	0.905	0.314	0.696	0.434	0.391	0.191	62
	1.00	0.651	0.837	0.373	0.878	0.472	0.505	0.248	45	0.651	0.837	0.373	0.878	0.472	0.505	0.248	49
Linear	0.25	0.568	0.816	0.312	0.712	0.433	0.468	0.188	83	0.486	0.768	0.279	0.787	0.436	0.422	0.175	89

Table 4: Swap schedule and ratio ablation (500K pre-training samples,

T=30

, patch size 2, 3 seeds per configuration). Total rank sums the per-configuration ranks across all seven metrics within each panel; lower is better. Bold marks the best value per column within a panel.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	Linear Probe		CAMELS		Carbon Flux
Method	LP-30 $\uparrow$	LP-5 $\uparrow$	SF $R^{2}\uparrow$	Basin RMSE $\downarrow$	GPP $R^{2}\uparrow$	RECO $R^{2}\uparrow$	NEE $R^{2}\uparrow$
Imp. + Contrastive	0.846 $\pm$ 0.003	0.928* $\pm$ 0.001	0.410* $\pm$ 0.008	0.681 $\pm$ 0.010	0.453 $\pm$ 0.045	0.325 $\pm$ 0.024	0.203 $\pm$ 0.035
Multi-Imp. + Contrastive	0.844 $\pm$ 0.003	0.928 $\pm$ 0.001	0.401 $\pm$ 0.007	0.711 $\pm$ 0.013	0.438 $\pm$ 0.008	0.344 $\pm$ 0.026	0.209 $\pm$ 0.015
Imp. + NTP	0.738 $\pm$ 0.005	0.885 $\pm$ 0.002	0.379 $\pm$ 0.006	0.821 $\pm$ 0.029	0.492 $\pm$ 0.018	0.406 $\pm$ 0.035	0.226 $\pm$ 0.023
Multi-Imp. + NTP	0.752 $\pm$ 0.002	0.895 $\pm$ 0.002	0.367 $\pm$ 0.006	0.866 $\pm$ 0.018	0.490 $\pm$ 0.017	0.450* $\pm$ 0.014	0.238* $\pm$ 0.004
Imp. + Feat-MAE	0.754 $\pm$ 0.002	0.895 $\pm$ 0.001	0.395 $\pm$ 0.002	0.780 $\pm$ 0.037	0.498 $\pm$ 0.014	0.403 $\pm$ 0.027	0.217 $\pm$ 0.008

Appendix

Table 5: Evaluation of the concatenated representations. Underlined values indicate an improvement over the corresponding base method, and bold values represent the best performance in each column. ∗ indicates the best score achieved across all objectives and experiments throughout this work.

	Linear Probe		CAMELS		Carbon Flux
Method	LP-30 $\uparrow$	LP-5 $\uparrow$	SF $R^{2}\uparrow$	Basin RMSE $\downarrow$	GPP $R^{2}\uparrow$	RECO $R^{2}\uparrow$	NEE $R^{2}\uparrow$
Imp. + Contrastive	0.843 $\pm$ 0.003	0.922 $\pm$ 0.001	0.377 $\pm$ 0.007	0.591 $\pm$ 0.004	0.438 $\pm$ 0.009	0.309 $\pm$ 0.014	0.172 $\pm$ 0.026
Multi-Imp. + Contrastive	0.851 $\pm$ 0.002	0.926 $\pm$ 0.001	0.361 $\pm$ 0.010	0.589* $\pm$ 0.009	0.457 $\pm$ 0.013	0.285 $\pm$ 0.032	0.148 $\pm$ 0.047
Imp. + NTP	0.804 $\pm$ 0.016	0.915 $\pm$ 0.009	0.365 $\pm$ 0.005	0.671 $\pm$ 0.002	0.506 $\pm$ 0.010	0.371 $\pm$ 0.004	0.215 $\pm$ 0.021
Multi-Imp. + NTP	0.809 $\pm$ 0.004	0.919 $\pm$ 0.001	0.368 $\pm$ 0.009	0.690 $\pm$ 0.003	0.481 $\pm$ 0.006	0.399 $\pm$ 0.006	0.173 $\pm$ 0.004
Imp. + Feat-MAE	0.786 $\pm$ 0.008	0.907 $\pm$ 0.006	0.374 $\pm$ 0.012	0.748 $\pm$ 0.012	0.519* $\pm$ 0.026	0.396 $\pm$ 0.017	0.236 $\pm$ 0.012

Appendix

Table 6: Evaluation of joint objectives. Underlined values indicate an improvement over the corresponding base method, and bold values represent the best performance in each column. ∗ indicates the best score achieved across all objectives and experiments throughout this work.

Parameter	Value
Backbone	Patch transformer
Embedding dimension $d$	128
Transformer blocks	4
Attention heads	8
MLP ratio	4
Projection dimension	64

Appendix

Table 7: Shared architecture and optimization hyperparameters.

Group	Variable
Temperature	2m air temperature
	2m dewpoint temperature
	Soil temperature (level 1)
	Soil temperature (level 4)
Radiation	Surface net solar radiation
	Surface net thermal radiation

Appendix

Table 8: ERA5-Land variables used for pre-training, grouped by physical process.

	Decoder Only		Encoder+Decoder
Method	FLOPs	Overhead	FLOPs	Overhead
Imp.	87,296	1.00 $\times$	23,872,256	1.00 $\times$
NTP	87,296	1.00 $\times$	23,872,256	1.00 $\times$
Feat-MAE	701,952	8.04 $\times$	24,486,912	1.03 $\times$
Temp-MAE	701,952	8.04 $\times$	24,486,912	1.03 $\times$
FT-MAE	701,952	8.04 $\times$	24,486,912	1.03 $\times$

Appendix

Table 9: Task-specific computational overhead (FLOPs) for the head/decoder components vs. Full-Model End-to-End FLOPs (including the shared encoder,

23,784,960

FLOPs). Overhead metrics are reported relative to imposter.

Self-Supervised Learning

Self-supervised learning uses structure within unlabeled data to train representations and models for downstream tasks.

See all 259 papers

Representation Learning for Spatiotemporal Physical Systems
Mar 13, 2026Helen Qu, Rudy Morel, Michael McCabe +4Representation Learning Scientific ML
Flatiron Institute · Princeton University · Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM +1
Probing Geospatial SSL Representations with Environmental Signals
Jul 6, 2026Rohita Mocharla, Vishal M. PatelRepresentation Geometry Geospatial Representation Learning
Johns Hopkins Applied Physics Laboratory · Johns Hopkins University · Laurel, MD +1
Learning to Perceive "Where": Spatial Pretext Tasks for Robust Self-Supervised Learning
May 11, 2026Yang Shen, Yusen Cai, Weronika Hryniewska-Guzik +2Visual Spatial Reasoning Self-Supervised Visual Representation Learning
Nanyang Technological University, Singapore · Warsaw University of Technology, Poland

Mar 13, 2026J/K move · Enter open · S save

Representation Learning for Spatiotemporal Physical Systems

Helen Qu, Rudy Morel, Michael McCabe, Alberto Bietti+3

Flatiron Institute · Princeton University · Université Paris-Saclay, Université Paris Cité, CEA, CNRS, AIM+1

Representation Learning Scientific ML Self-Supervised Time Series Representation Learning Joint-Embedding Predictive Architecture

Open paper AskPDF arXiv

Machine learning approaches to spatiotemporal physical systems have primarily focused on next-frame prediction, with the goal of learning an accurate emulator for the system's evolution in time. However, these emulators are computationally expensive to train and are subject to performance pitfalls, such as compounding errors during autoregressive rollout. In this work, we take a different perspective and look at scientific tasks further downstream of predicting the next frame, such as estimation of a system's governing physical parameters. Accuracy on these tasks offers a uniquely quantifiable glimpse into the physical relevance of the representations of these models. We evaluate the effectiveness of general-purpose self-supervised methods in learning physics-grounded representations that are useful for downstream scientific tasks. Surprisingly, we find that not all methods designed for physical modeling outperform generic self-supervised learning methods on these tasks, and methods that learn in the latent space (e.g., joint embedding predictive architectures, or JEPAs) outperform those optimizing pixel-level prediction objectives. Code is available at https://github.com/helenqu/physical-representation-learning.

Abstract

Self-Supervised Learning

Explore similar work

Representation Learning for Spatiotemporal Physical Systems

Probing Geospatial SSL Representations with Environmental Signals

Learning to Perceive "Where": Spatial Pretext Tasks for Robust Self-Supervised Learning