Sat-JEPA-Diff: Bridging Self-Supervised Learning and Generative Diffusion for Remote Sensing

Kursat Komurcu; Linas Petkevicius

cs.CVMar 14, 2026

Sat-JEPA-Diff: Bridging Self-Supervised Learning and Generative Diffusion for Remote Sensing

Authors: Kursat Komurcu, Linas Petkevicius

Organizations: Vilnius University, Faculty of Mathematics and Informatics, Institute of Computer Science, Artificial Intelligence Methods Lab, Vilnius, LT-03225, Lithuania

Abstract

Predicting satellite imagery requires a balance between structural accuracy and textural detail. Standard deterministic methods like PredRNN or SimVP minimize pixel-based errors but suffer from the "regression to the mean" problem, producing blurry outputs that obscure subtle geographic-spatial features. Generative models provide realistic textures but often misleadingly reveal structural anomalies. To bridge this gap, we introduce Sat-JEPA-Diff, which combines Self-Supervised Learning (SSL) with Hidden Diffusion Models (LDM). An IJEPA module predicts stable semantic representations, which then route a frozen Stable Diffusion backbone via a lightweight cross-attention adapter. This ensures that the synthesized high-accuracy textures are based on absolutely accurate structural predictions. Evaluated on a global Sentinel-2 dataset, Sat-JEPA-Diff excels at resolving sharp boundaries. It achieves leading perceptual scores (GSSIM: 0.8984, FID: 0.1475) and significantly outperforms deterministic baselines, despite standard autoregressive stability limits. The code and dataset are publicly available on https://github.com/VU-AIML/SAT-JEPA-DIFF.

Figures & tables

Figure 1: Overview of Sat-JEPA-Diff. The IJEPA module (left) predicts future semantic embeddings $\hat{z}_{t+1}$ from input $I_{t}$ . These embeddings, combined with coarse spatial structure, condition a frozen SD3.5 backbone via a learned adapter to generate $\hat{I}_{t+1}$ .

Deterministic Baselines
Model	L1 $\downarrow$	MSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	GSSIM $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$
Default	0.0131	0.0008	37.52	0.9361	0.7858	0.0708	0.6959
PredRNN Wang et al. (2022)	0.0117	0.0005	38.38	0.9476	0.7836	0.0726	9.9720
SimVP v2 Gao et al. (2022)	0.0131	0.0006	37.63	0.9391	0.7719	0.0928	18.7208
Generative Models
SD 3.5 Esser et al. (2024)	0.0175	0.0005	32.98	0.8398	0.8711	0.4528	0.1533

Table 1: Quantitative comparison on the test set. Arrows indicate whether lower (

\downarrow

) or higher (

\uparrow

) values are better. Best results are highlighted in bold .

Figure 2: Qualitative comparison of next-frame predictions ( $t\to t+1$ ). While deterministic baselines (PredRNN, SimVP) suffer from spectral blurring, Sat-JEPA-Diff preserves high-frequency details and geospatial boundaries.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Geographical distribution of the 100 selected Regions of Interest (RoIs).

Figure 4: Systematic IJEPA Loss Ablation. Validation metrics over 100 epochs demonstrate that our full objective function (E curve) uniquely avoids representation collapse and maintains high embedding variance compared to near-zero variance in the reduced underlying models (A-D curves). Despite higher total loss, the full model maintains high cosine similarity and achieves superior spatial variance.

Figure 5: Long-horizon autoregressive rollout comparison ( $2018\to 2024$ ) on the Rio de Janeiro Coast. Top Row: Ground Truth. Rows 2-3: Deterministic baselines rapidly degrade into spectral blurring (spatial collapse) after 2-3 steps. Bottom Row: Sat-JEPA-Diff maintains high contrast and structural sharpness throughout the 7-year horizon.

Spatiotemporal Forecasting

Spatiotemporal forecasting predicts how quantities change across locations and time, from traffic and weather to disease and environmental systems.

See all 423 papers

TerraDiT: Point-Conditioned Diffusion Transformer for Satellite Image Synthesis
Mar 2, 2026Srikumar Sastry, Dan Cher, Brian Wei +4Diffusion Transformer Controllable Image Generation
Washington University, St. Louis MO 63130, USA
Remote Sensing Image Super-Resolution for Imbalanced Textures: A Texture-Aware Diffusion Framework
Apr 15, 2026Enzhuo Zhang, Sijie Zhao, Dilxat Muhtar +3Remote Sensing Image Super-Resolution Image Super-Resolution
Nanjing University, China
DiffuSAM: Diffusion Guided Zero-Shot Object Grounding for Remote Sensing Imagery
Apr 20, 2026Geet Sethi, Panav Shah, Ashutosh Gandhe +1Remote Sensing Image Understanding Diffusion Model Guidance
Indian Institute of Technology Bombay

Mar 2, 2026J/K move · Enter open · S save

TerraDiT: Point-Conditioned Diffusion Transformer for Satellite Image Synthesis

Srikumar Sastry, Dan Cher, Brian Wei, Aayush Dhakal+3

Washington University, St. Louis MO 63130, USA

Diffusion Transformer Controllable Image Generation T2I Diffusion Models

Open paper AskPDF arXiv

We introduce TerraDiT, a diffusion transformer designed for text-to-satellite image generation with point-based control. Existing controlled satellite image generative models often require pixel-level maps that are time-consuming to acquire, yet semantically limited. To address this limitation, we introduce a novel point-based conditioning framework that controls the generation process through the spatial location of the points and the textual description associated with each point, providing semantically rich control signals. This approach enables flexible, annotation-friendly, and computationally simple inference for satellite image generation. To this end, we introduce an adaptive local attention mechanism that effectively regularizes the attention scores based on the input point queries. We systematically evaluate various domain-specific design choices for training TerraDiT, including the selection of satellite image representation for alignment and geolocation representation for conditioning. Our experiments demonstrate that TerraDiT achieves impressive generation performance, surpassing the state-of-the-art remote sensing generative models. Our models, dataset, and code are available at https://github.com/mvrl/TerraDiT.

Abstract

Spatiotemporal Forecasting

Explore similar work

TerraDiT: Point-Conditioned Diffusion Transformer for Satellite Image Synthesis

Remote Sensing Image Super-Resolution for Imbalanced Textures: A Texture-Aware Diffusion Framework

DiffuSAM: Diffusion Guided Zero-Shot Object Grounding for Remote Sensing Imagery