Probabilistic neural surrogates for PDEs enable ensemble forecasting, but often exhibit miscalibration when used for uncertainty quantification, especially in compute-constrained, few-step inference regimes. Existing generative approaches typically rely on fixed global stochasticity, which cannot adapt predictive uncertainty to the local dynamical state. We propose TORNADO, a latent probabilistic forecasting framework that formulates next-step prediction as stochastic transport between VAE encoder posteriors. Unlike existing approaches with globally prescribed stochasticity, TORNADO learns a SDE with state-dependent diffusion, enabling the predictive uncertainty to adapt to the local latent dynamics. The model is trained using analytic conditional endpoint targets derived from a Gaussian interpolant, enabling simulation-free learning. An additional energy distance regularization term improves distributional alignment. Empirically, TORNADO improves uncertainty calibration over representative generative baselines, including stochastic interpolants, diffusion models, and flow matching, while maintaining comparable predictive accuracy. Our results show that adapting stochasticity to the latent state, rather than prescribing it globally, is an effective mechanism for improving calibration in practical neural PDE forecasting settings. Project page: https://tornado-sde.github.io.
Figures & tables
Figure 1: Inference pipeline of TORNADO. The encoder Eϕ maps the current PDE state ut to a latent posterior qϕ(zt∣ut) , from which a latent sample zt is drawn. Starting from the encoded latent state, we draw K independent realizations of the learned latent SDE over the auxiliary time τ∈[0,1] , using drift fθ and diagonal diffusion gθ′ predicted by PΘ . The decoder Dϕ maps the resulting latent ensemble to a predictive distribution over ut+1 , yielding a mean u^ˉt+1 and a 95% confidence interval CI95(u^t+1) .
Dataset
Model
MSE (10−3)(↓)
CRPS (10−2)(↓)
RMSCE (10−2)(↓)
∣log(SSR)∣(↓)
Kolmogorov
DDIM
178 ± 4
18.84 ± 0.14
9.52 ± 0.05
0.628 ± 0.005
FM
107 ± 3
16.33 ± 0.15
20.33 ± 0.03
1.494 ± 0.006
SI
220 ± 5
22.43 ± 0.14
10.56 ± 0.06
0.653 ± 0.005
TORNADO
120 ± 2
14.53 ± 0.12
3.95 ± 0.06
0.268 ± 0.006
KS
DDIM
33.88 ± 0.14
11.12 ± 0.02
15.051 ± 0.015
0.8773 ± 0.0014
FM
9.138 ± 0.015
5.152 ± 0.005
5.483 ± 0.010
0.2984 ± 0.0007
Table 1: Interval evaluation at NFE =5 over each dataset’s horizon: steps 1–20, except ERA5 (1–40). Mean ± std over the test set, except Combustion, where ± is the spread across the four held-out conditions. ERA5 aggregates its four variables in normalised, cos(lat) -weighted units. Best and second-best per dataset and metric are highlighted ( green / blue ).
Figure 3: Kolmogorov flow rollout metrics vs. horizon t at fixed compute (NFE=5): MSE, CRPS, RMSCE, and SSR. Lower is better for all metrics except SSR (ideal value 1).
Method
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
Euler
0.439
0.351
0.286
9.674
Constant Amplitude
2.701
0.810
0.085
0.602
Fixed Schedule
0.488
0.291
0.067
0.361
TORNADO
0.371
0.241
0.039
0.264
Table 2: Learned state-dependent adaptive diffusion (TORNADO) vs. other integration methods for a same drift, matched NFE=5, averaged over rollout time T=33 for the Kolmogorov Flow dataset.
Figure 4: Rollout visualization for T=50 on the Kolmogorov Flow dataset. Rows top to bottom: ground truth trajectory, TORNADO mean prediction, pixel-wise mean prediction error, pixel-wise predictive ensemble 95% confidence interval.
121×240 (latitude × longitude), poles included, longitude periodic
Cadence
6-hourly
Channels
4 ( T2m , U10 , V10 , Z500 )
Frames
8764 train / 1464 validation / 2920 test
Split (by year)
2010–2015 / 2016 / 2017–2018
Encoder padding
121→128 latitudes: circular in longitude, replicate at
the poles; cropped back after decoding
Appendix
Table 4: WeatherBench-2 ERA5 at 1.5∘ : dataset specifications.
Loss setting
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
λED=0,λvar=1
0.0245
0.0766
0.1004
0.6240
λED=0.1,λvar=0
0.0244
0.0744
0.0420
0.1520
λED=1,λvar=0
0.0388
0.0929
0.0583
0.1273
λED=0.1,λvar=1
0.0196
0.0662
0.0309
0.2335
Appendix
Table 5: Diffusion-loss ablation on Kolmogorov Flow at NFE=5. Lower is better.
Figure 5: Hyperparameter sensitivity sweep for λED∈[0,1.0] . The method exhibits a stability plateau for λED∈[0.1,0.5] where calibration is minimized without degrading accuracy.
β
VAE RMSCE ↓
TORNADO 1-step RMSCE ↓
10−7
0.284
0.098
10−6
0.100
0.073
10−5
0.169
0.148
10−4
0.218
0.201
Appendix
Table 6: Calibration results for the VAE’s reconstruction task and TORNADO’s one-step prediction task averaged over the validation set for different values of β
β
Recon. vRMSE ↓
KL (nats/sample)
Rollout MSE ↓
Rollout RMSCE ↓
Sharpness ↓
10−7
0.0018
11031.9
0.0216
0.0514
0.0907
10−6
0.0024
5376.7
0.2060
0.0393
0.4301
10−5
0.0084
2004.0
0.2177
0.0287
0.3467
10−4
0.0357
635.4
2.0723
0.0388
1.7371
Appendix
Table 7: VAE regularization sweep on Kolmogorov flow: latent statistics of each VAE and 20-step rollout metrics of the TORNADO model trained on it (NFE =5 , K=16 ).
Method
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
TORNADO-EM
0.155
0.167
0.053
0.376
TORNADO-Milstein
0.155
0.167
0.053
0.376
FM-Euler
0.107
0.163
0.203
1.500
FM-Heun
0.106
0.161
0.195
1.404
Appendix
Table 8: Results for Kolmogorov at T=20 with different ODE/SDE solvers for TORNADO (Euler-Maruyama and Milstein) and FM (Euler and Heun)
Setting
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
w/ detach
0.0195
0.0662
0.0309
0.2334
w/o detach
0.0203
0.0672
0.0304
0.2296
Appendix
Table 9: Gradient-stop ablation (detach diffusion head) on Kolmogorov Flow at NFE=5, averaged over rollout horizon T=10 . Lower is better.
Method
NFE=2
NFE=5
NFE=10
DDIM
555.00
1387.70
2773.38
FM
555.16
1387.80
2775.52
SI
554.70
1387.57
2775.01
TORNADO
602.41
1505.95
3011.94
Appendix
Table 10: Compute cost (ms per forecast step) at batch size B=32 and ensemble size K=16 . Lower is better.
Figure 6: RMSCE and ∣log(SSR)∣ at σγ∈{0,0.1,0.5,1,10.0} for the Kolmogorov Flow, Burgers and KS datasets averaged over 10 rollout steps.
Figure 7: Calibration, predictive skill, and spread across ensemble sizes. Lower RMSCE and CRPS are better; the target spread–skill ratio is one.
Method / training regime
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
TORNADO (TF)
0.120
0.145
0.0395
0.268
TORNADO-AR (H=2)
0.139
0.157
0.0368
0.225
TORNADO-AR (H=4)
0.111
0.141
0.0318
0.180
Appendix
Table 11: Autoregressive extension of TORNADO on the Kolmogorov Flow dataset. Extending training beyond teacher forcing preserves the benefits of the base method and further improves calibration, with H=4 achieving the best overall performance.
Method
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
TORNADO (NFE=5)
0.120
0.145
0.039
0.268
NLL
0.326
0.243
0.192
2.344
VAE matching
0.687
0.360
0.199
2.628
MC Dropout
0.321
0.250
0.075
0.447
Appendix
Table 12: Comparison with direct one-step latent baselines on the Kolmogorov Flow dataset.
Figure 8: 1D Wasserstein distance between the predicted and ground-truth total-enstrophy distributions over the rollout windows of length 20, aggregated over the test set for Kolmogorov Flow over 100 step trajectories. Lower is better.
Figure 9: Normalized total enstrophy over two representative Kolmogorov flow rollouts. Red curves show the ground truth; gray curves and shaded regions show the predictive ensemble mean and one standard deviation for each method.
Figure 10: 20 step pooled calibration error versus mean 90% predictive interval width on Kolmogorov flow.
Method
RMSCE ↓
PI width@95%
cov.@95%
SSR →1
CRPS ↓
DDIM
0.095
0.521
0.632
0.533
0.188
FM
0.203
0.156
0.284
0.225
0.163
SI
0.106
2.037
0.990
1.921
0.224
TORNADO
0.039
0.624
0.800
0.765
0.145
Appendix
Table 13: Calibration, sharpness, and predictive skill on Kolmogorov flow.
Figure 11: Reliability diagrams on Kolmogorov flow (NFE =5 , K=16 ), one-step (left) and over the 20-step rollout (right). Curves above the diagonal over-cover, below under-cover.
Table 14: Within-method tercile analysis of latent-space error and variance for TORNADO and SI.
Variable
Model
MSE (10−3)(↓)
CRPS (10−2)(↓)
RMSCE (10−2)(↓)
∣log(SSR)∣(↓)
T2m
DDIM
13.116 ± 2.634
5.60 ± 0.44
17.72 ± 2.16
0.787 ± 0.051
FM
13.367 ± 2.569
5.52 ± 0.43
15.91 ± 2.22
0.637 ± 0.046
SI
11.604 ± 2.214
4.90 ± 0.38
8.44 ± 1.48
0.292 ± 0.045
TORNADO
11.885 ± 2.212
4.94 ± 0.35
7.53 ± 1.21
0.265 ± 0.042
U10
DDIM
451.1 ± 40.7
35.06 ± 1.67
12.35 ± 0.51
0.655 ± 0.044
FM
460.3 ± 48.3
35.08 ± 1.89
11.00 ± 0.65
0.578 ± 0.048
Appendix
Table 15: ERA5 per-variable results on the held-out years (NFE =5 , K=16 , steps 1–40, 70 test trajectories). Metrics are in normalised, cos(lat) -weighted units; mean ± std over the test set. Best and second-best per variable and metric are highlighted ( green / blue ).
Figure 15: Kolmogorov flow rollout metrics vs. horizon t at fixed compute (NFE=2): MSE, CRPS, RMSCE, and SSR.
Figure 16: Kolmogorov flow rollout metrics vs. horizon t at fixed compute (NFE=10): MSE, CRPS, RMSCE, and SSR.
Figure 17: NFE sensitivity on Kolmogorov Flow. We report rollout MSE, CRPS, RMSCE, and SSR as a function of NFE used by method. Metrics are averaged over the same rollout horizon T=50 .
Figure 18: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Kolmogorov Flow data set.
Figure 19: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Kolmogorov Flow data set.
Figure 20: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Burgers data set.
Figure 21: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Burgers data set.
Figure 22: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the KS data set.
Figure 23: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the KS data set.
Figure 24: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the GS data set.
Figure 25: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the GS data set.
Figure 26: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the RealPDEBench Combustion data set.
Figure 27: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the RealPDEBench Combustion data set.
Figure 28: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the T2m variable WeatherBench ERA5 data set.
Figure 29: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the T2m variable WeatherBench ERA5 data set.
Figure 30: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the U10 variable WeatherBench ERA5 data set.
Figure 31: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the U10 variable WeatherBench ERA5 data set.
Figure 32: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the V10 variable WeatherBench ERA5 data set.
Figure 33: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the V10 variable WeatherBench ERA5 data set.
Figure 34: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Z500 variable WeatherBench ERA5 data set.
Figure 35: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Z500 variable WeatherBench ERA5 data set.