S2S-JEPA: Predicting the Predictable at Subseasonal-to-Seasonal Timescales

Chenyu Dong; Gianmarco Mengaldo

physics.ao-phOct 2, 2026

S2S-JEPA: Predicting the Predictable at Subseasonal-to-Seasonal Timescales

Authors: Chenyu Dong, Gianmarco Mengaldo

Organizations: Department of Mechanical Engineering, College of Design and Engineering, National University of Singapore, 9 Engineering Drive 1, 117575, Singapore.

Abstract

The subseasonal-to-seasonal (S2S) timescale, roughly from two weeks to two months ahead, is a critical forecast window for sectors such as agriculture, energy, and water management. Yet, it is widely known as the `predictability desert'. Recent AI weather models excel up to two weeks ahead but deteriorate beyond, largely because they are trained to predict fine-scale details that are neither predictable nor essential at S2S timescales. We argue that a more physically grounded objective is to forecast only the slowly varying components that remain predictable. Computer vision reached the same conclusion with the Joint-Embedding Predictive Architecture (JEPA), which predicts in latent space, discarding unpredictable details. In this work, we introduce S2S-JEPA, which brings the JEPA paradigm to S2S forecasting. It is tailored to this task through design elements from state-of-the-art AI weather models. S2S-JEPA achieves comparable skill to the gold-standard ECMWF physics-based ensemble and surpasses it on multiple metrics at weeks 5 to 6.

Figures & tables

Figure 1: Two-stage training of S2S-JEPA. (a) Stage 1: one-step latent prediction. The encoder embeds the context week $X^{(0)}$ (daily anomaly fields and static variables) into a latent state, and the predictor, conditioned on the context-week climatology $C^{(0)}$ , predicts the latent state of the next week. The prediction is matched under an $\ell_{1}$ loss to the embedding of the true next week under an EMA target encoder, which receives no gradient. (b) Stage 2: multi-step rollout training. Starting from the context-week latent, the predictor rolls out week-by-week predictions up to week 6, conditioned at each step on the climatology of the input week $C^{(k-1)}$ , and each predicted latent is matched to the now-frozen target encoder’s embedding of the observed week ( $\mathcal{L}_{1},\dots,\mathcal{L}_{6}$ ). Gradients do not traverse the rollout chain; the context encoder is updated through the first step only.

		Lead (week)
Variable	System	3	4	5	6
T2M	ECMWF (ens. mean)	0.349	0.244	0.202	0.181
T2M	S2S-JEPA (ours)	0.289 ↓	0.251	0.238 ↑	0.230 ↑
MSLP	ECMWF (ens. mean)	0.467	0.338	0.288	0.260
MSLP	S2S-JEPA (ours)	0.335 ↓	0.304 ↓	0.283	0.277
TP	ECMWF (ens. mean)	0.270	0.204	0.168	0.161

Table 1: Deterministic skill at sub-seasonal lead times (weeks 3–6) on the 102 common Wednesday initializations (13 Dec 2023–19 Nov 2025): temporal correlation coefficient (TCC) of weekly-mean anomalies, area-weighted over land for T2M and TP and over the globe for MSLP. ECMWF is the 101-member ensemble mean; S2S-JEPA is a single deterministic forecast. Anomalies are computed relative to the 2004–2023 ERA5 daily climatology. Best values per column are shown in bold. An upward arrow marks where S2S-JEPA is significantly better than ECMWF, a downward arrow where it is significantly worse, based on a bootstrap significance test over initializations (Appendix D ).

Figure 2: Spatial distribution of deterministic skill at weeks 5–6. Grid-point TCC of weekly-mean anomalies for the ECMWF ensemble mean (left) and S2S-JEPA (right), computed jointly over weeks 5 and 6 on the same initializations as Tab. 1 . Numbers below each panel give the area-weighted average, over land for T2M and TP and over the globe for MSLP. Stippling marks grid points where the correlation is significant.

		Lead (week)
Variable	System	3	4	5	6
T2M (detrended)	ECMWF	0.092	0.049	0.040	0.031
T2M (detrended)	S2S-JEPA + EDM	0.079 ↓	0.058	0.049	0.043 ↑
MSLP	ECMWF	0.152	0.075	0.053	0.041
MSLP	S2S-JEPA + EDM	0.078 ↓	0.052 ↓	0.044	0.035
TP	ECMWF	0.048	0.021	0.008	0.005

Table 2: Probabilistic skill at weeks 3–6: fair RPSS of quintile probabilities relative to the uniform forecast under the FuXi-S2S protocol, with each system’s quintile edges taken from its own 2004–2023 reforecasts (11 members per initialization). S2S-JEPA + EDM has 50 members. T2M is scored after removing each system’s own linear trend and the observed trend; MSLP and TP are scored without detrending. TP grid points with a degenerate quintile range are masked.

Figure 3: As in Fig. 2 , but for the RPSS of quintile forecasts relative to climatology. For TP, grid points where the climatological quintiles are degenerate due to near-zero precipitation are masked.

Forecasting formulation
	Lead (week)
Model	1	2	3	4	5	6
S2S-JEPA (final)	0.609	0.335	0.271	0.241	0.226	0.219
Lead-conditioned	0.422	0.265	0.209	0.175	0.152	0.141
Per-lead experts	0.620	0.299	0.232	0.208	0.198	0.187
Prediction objective

Table 3: Ablations of the key design choices in S2S-JEPA. Values report the mean TCC across T2M, MSLP, and TP at each lead time.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Type	Variable	Short name	Fields
Pressure level	Geopotential	z	12
	Temperature	t	12
	U component of wind	u	12
	V component of wind	v	12
	Specific humidity	q	12
Single level	2 m temperature	t2m	1

Appendix

Table 4: Variables used by S2S-JEPA. All fields are daily means of ERA5 on the global

1.5^{\circ}

grid, and short names follow the ERA5 convention. Pressure-level variables are taken at 1000, 925, 850, 700, 600, 500, 400, 300, 200, 100, 50 and 10 hPa. The column Fields gives the number of two-dimensional fields that a variable contributes, which adds up to 79 for the time-varying inputs. The static fields enter the encoder and the insolation enters the predictor. The forecast targets are t2m, msl and tp.

Latent level	Group	Variables	Static Input	Total
Boundary	Ocean	sst , siconc	Orography, land–sea mask, soil type	5
Boundary	Land	sd , stl1-4 , swvl1-4	Orography, land–sea mask, soil type	12
Surface	Near-surface	t2m , d2m , msl , u10 , v10	Orography, land–sea mask, soil type	8
Surface	Column	tcwv , tp , ttr	Orography, land–sea mask, soil type	6
Lower troposphere	1000, 925, 850, 700 hPa	q , t , u , v , z at each level	Orography	21
Free troposphere	600, 500, 400, 300, 200 hPa	q , t , u , v , z at each level	–	25

Appendix

Table 5: The 79 ERA5 variables are first partitioned into 16 physically motivated groups, which are further organized into five latent levels. Static inputs are appended to the groups indicated in the fourth column. Total denotes the number of distinct fields in each row, including dynamic variables and static inputs. For each group, the seven input days are stacked channel-wise and embedded with a

6\times 6

patch convolution. The resulting group embeddings within each latent level are then concatenated and projected to a single token per patch.

	Stage 1: pre-training	Stage 2: rollout fine-tuning
Initialisation	random	Stage 1, iteration 10,000
Rollout steps per sample	1	6 (leads 7–42 days)
Objective	JEPA (L1 in latent space)	JEPA at every step, detached hand-offs
Teacher	EMA, momentum 0.999 $\rightarrow$ 1	frozen
Context / target length	7 / 7 days	7 / 7 days
Batch size	32	32

Appendix

Table 6: Training settings of the forecast model. Both stages use the same architecture (91.0M trainable parameters including the readout head), the same data and the same optimiser.

Setting	Value
Noise distribution $(P_{\mathrm{mean}},P_{\mathrm{std}})$	T2M $(-0.6,1.4)$ ; MSLP $(0.5,1.7)$ ; TP $(-1.8,1.0)$
Data scale $\sigma_{\mathrm{data}}$	1 (residual normalised per lead and variable)
Training years / validation years	1979–2018 / 2019–2021
Batch size	32
Learning rate	$2\times 10^{-4}$ , warmup 200, decay over the last 600 of 3,000 iterations
Weight decay	0.1

Appendix

Table 7: Training settings of the diffusion model (one head per variable, 9.3M parameters each).

Figure 4: As in Fig. 3 , but with system-specific linear trends removed for MSLP and TP.

Explore similar work

AIFS-SUBS: Extending Data-Driven Forecasting to Sub-Seasonal Timescales
Jul 6, 2026Jakob Schloer, Steffen Tietsche, Christopher D. Roberts +6Probabilistic Forecasting Weather Forecasting
1European Centre for Medium-Range Weather Forecasts (ECMWF)
Dual-Scale Temporal Fusion Reveals Structured Predictability in Subseasonal-to-Seasonal Temperature Prediction
May 7, 2026Elnaz Bashir, Jiali Wang, Lin YanTime Series Forecasting Multi-Scale Feature Fusion
Department of Computer Science, Iowa State University, Ames, IA 50011, USA · Environmental Science Division, Argonne National Laboratory, Lemont, IL 60439, USA
SimCast-S2S: A Computationally Efficient Diffusion Model for Subseasonal Precipitation Forecasting
Aug 27, 2026Hiep V. Dang, Antonios MamalakisPrecipitation Forecasting Latent Diffusion Models
Department of Environmental Sciences, University of Virginia, Charlottesville, VA 22903, United States · School of Data Science, Department of Environmental Sciences, University of Virginia, Charlottesville, VA 22903, United States

Jul 6, 2026, physics.ao-phJ/K move · Enter open · S save

AIFS-SUBS: Extending Data-Driven Forecasting to Sub-Seasonal Timescales

Jakob Schloer, Steffen Tietsche, Christopher D. Roberts, Lorenzo Zampieri+5

1European Centre for Medium-Range Weather Forecasts (ECMWF)

Probabilistic Forecasting Weather Forecasting Autoregressive Modeling

Open paper AskPDF arXiv

Data-driven models now rival numerical weather prediction in the medium range, but extending them to sub-seasonal lead times raises challenges absent at shorter horizons. Errors accumulate over long autoregressive rollouts, systematic biases grow with lead time, and several years of data must be held out for independent verification, even though machine-learning models otherwise benefit from longer training records. To address these challenges, we adapt ECMWF's AIFS-CRPS medium-range model. AIFS-SUBS adopts a 24h autoregressive time step to reduce error accumulation, adds stratospheric levels and top-of-atmosphere thermal radiation as predictors, and reserves 2007--2011 as an independent verification window. We evaluate two config-durations: AIFS-SUBS, fine-tuned on operational analyses, and AIFS-SUBS-ERA5, trained on ERA5 alone. Across weeks 2--6, AIFS-SUBS matches the operational Integrated Forecasting System (IFS) in probabilistic skill while reducing systematic biases. For the convective (OLR) component of the Madden--Julian Oscillation (MJO), AIFS-SUBS extends skilful forecasts (correlation > 0.5) by eight days relative to the IFS, while matching or exceeding the IFS for the full multivariate RMM index. AIFS-SUBS also reproduces the observed MJO modulation of tropical cyclone activity comparably. Stratospheric skill is particularly strong with AIFS-SUBS reproducing sudden stratospheric warming (SSW) frequency and surface impact. In the AI Weather Quest, AIFS-SUBS-ERA5 attains a variable-averaged ranked probability skill score slightly ahead of the IFS at weeks 3 and 4. At inference, AIFS-SUBS uses about 200 times less energy than the IFS, opening the door to much larger real-time ensembles. AIFS-SUBS is ECMWF's first machine-learning model targeted at sub-seasonal time-scales.

Abstract

Explore similar work

AIFS-SUBS: Extending Data-Driven Forecasting to Sub-Seasonal Timescales

Dual-Scale Temporal Fusion Reveals Structured Predictability in Subseasonal-to-Seasonal Temperature Prediction

SimCast-S2S: A Computationally Efficient Diffusion Model for Subseasonal Precipitation Forecasting