Probabilistic neural surrogates for PDEs enable ensemble forecasting, but often exhibit miscalibration when used for uncertainty quantification, especially in compute-constrained, few-step inference regimes. Existing generative approaches typically rely on fixed global stochasticity, which cannot adapt predictive uncertainty to the local dynamical state. We propose TORNADO, a latent probabilistic forecasting framework that formulates next-step prediction as stochastic transport between VAE encoder posteriors. Unlike existing approaches with globally prescribed stochasticity, TORNADO learns a SDE with state-dependent diffusion, enabling the predictive uncertainty to adapt to the local latent dynamics. The model is trained using analytic conditional endpoint targets derived from a Gaussian interpolant, enabling simulation-free learning. An additional energy distance regularization term improves distributional alignment. Empirically, TORNADO improves uncertainty calibration over representative generative baselines, including stochastic interpolants, diffusion models, and flow matching, while maintaining comparable predictive accuracy. Our results show that adapting stochasticity to the latent state, rather than prescribing it globally, is an effective mechanism for improving calibration in practical neural PDE forecasting settings. Project page: https://tornado-sde.github.io.
Figures & tables
Figure 1: Inference pipeline of TORNADO. The encoder Eϕ maps the current PDE state ut to a latent posterior qϕ(zt∣ut) , from which a latent sample zt is drawn. Starting from the encoded latent state, we draw K independent realizations of the learned latent SDE over the auxiliary time τ∈[0,1] , using drift fθ and diagonal diffusion gθ′ predicted by PΘ . The decoder Dϕ maps the resulting latent ensemble to a predictive distribution over ut+1 , yielding a mean u^ˉt+1 and a 95% confidence interval CI95(u^t+1) .
Dataset
Model
MSE (10−3)(↓)
CRPS (10−2)(↓)
RMSCE (10−2)(↓)
∣log(SSR)∣(↓)
Kolmogorov
DDIM
178 ± 4
18.84 ± 0.14
9.52 ± 0.05
0.628 ± 0.005
FM
107 ± 3
16.33 ± 0.15
20.33 ± 0.03
1.494 ± 0.006
SI
220 ± 5
22.43 ± 0.14
10.56 ± 0.06
0.653 ± 0.005
TORNADO
120 ± 2
14.53 ± 0.12
3.95 ± 0.06
0.268 ± 0.006
KS
DDIM
33.88 ± 0.14
11.12 ± 0.02
15.051 ± 0.015
0.8773 ± 0.0014
FM
9.138 ± 0.015
5.152 ± 0.005
5.483 ± 0.010
0.2984 ± 0.0007
Table 1: Interval evaluation at NFE =5 over each dataset’s horizon: steps 1–20, except ERA5 (1–40). Mean ± std over the test set, except Combustion, where ± is the spread across the four held-out conditions. ERA5 aggregates its four variables in normalised, cos(lat) -weighted units. Best and second-best per dataset and metric are highlighted ( green / blue ).
Figure 3: Kolmogorov flow rollout metrics vs. horizon t at fixed compute (NFE=5): MSE, CRPS, RMSCE, and SSR. Lower is better for all metrics except SSR (ideal value 1).
Method
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
Euler
0.439
0.351
0.286
9.674
Constant Amplitude
2.701
0.810
0.085
0.602
Fixed Schedule
0.488
0.291
0.067
0.361
TORNADO
0.371
0.241
0.039
0.264
Table 2: Learned state-dependent adaptive diffusion (TORNADO) vs. other integration methods for a same drift, matched NFE=5, averaged over rollout time T=33 for the Kolmogorov Flow dataset.
Figure 4: Rollout visualization for T=50 on the Kolmogorov Flow dataset. Rows top to bottom: ground truth trajectory, TORNADO mean prediction, pixel-wise mean prediction error, pixel-wise predictive ensemble 95% confidence interval.
121×240 (latitude × longitude), poles included, longitude periodic
Cadence
6-hourly
Channels
4 ( T2m , U10 , V10 , Z500 )
Frames
8764 train / 1464 validation / 2920 test
Split (by year)
2010–2015 / 2016 / 2017–2018
Encoder padding
121→128 latitudes: circular in longitude, replicate at
the poles; cropped back after decoding
Appendix
Table 4: WeatherBench-2 ERA5 at 1.5∘ : dataset specifications.
Loss setting
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
λED=0,λvar=1
0.0245
0.0766
0.1004
0.6240
λED=0.1,λvar=0
0.0244
0.0744
0.0420
0.1520
λED=1,λvar=0
0.0388
0.0929
0.0583
0.1273
λED=0.1,λvar=1
0.0196
0.0662
0.0309
0.2335
Appendix
Table 5: Diffusion-loss ablation on Kolmogorov Flow at NFE=5. Lower is better.
Figure 5: Hyperparameter sensitivity sweep for λED∈[0,1.0] . The method exhibits a stability plateau for λED∈[0.1,0.5] where calibration is minimized without degrading accuracy.
β
VAE RMSCE ↓
TORNADO 1-step RMSCE ↓
10−7
0.284
0.098
10−6
0.100
0.073
10−5
0.169
0.148
10−4
0.218
0.201
Appendix
Table 6: Calibration results for the VAE’s reconstruction task and TORNADO’s one-step prediction task averaged over the validation set for different values of β
β
Recon. vRMSE ↓
KL (nats/sample)
Rollout MSE ↓
Rollout RMSCE ↓
Sharpness ↓
10−7
0.0018
11031.9
0.0216
0.0514
0.0907
10−6
0.0024
5376.7
0.2060
0.0393
0.4301
10−5
0.0084
2004.0
0.2177
0.0287
0.3467
10−4
0.0357
635.4
2.0723
0.0388
1.7371
Appendix
Table 7: VAE regularization sweep on Kolmogorov flow: latent statistics of each VAE and 20-step rollout metrics of the TORNADO model trained on it (NFE =5 , K=16 ).
Method
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
TORNADO-EM
0.155
0.167
0.053
0.376
TORNADO-Milstein
0.155
0.167
0.053
0.376
FM-Euler
0.107
0.163
0.203
1.500
FM-Heun
0.106
0.161
0.195
1.404
Appendix
Table 8: Results for Kolmogorov at T=20 with different ODE/SDE solvers for TORNADO (Euler-Maruyama and Milstein) and FM (Euler and Heun)
Setting
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
w/ detach
0.0195
0.0662
0.0309
0.2334
w/o detach
0.0203
0.0672
0.0304
0.2296
Appendix
Table 9: Gradient-stop ablation (detach diffusion head) on Kolmogorov Flow at NFE=5, averaged over rollout horizon T=10 . Lower is better.
Method
NFE=2
NFE=5
NFE=10
DDIM
555.00
1387.70
2773.38
FM
555.16
1387.80
2775.52
SI
554.70
1387.57
2775.01
TORNADO
602.41
1505.95
3011.94
Appendix
Table 10: Compute cost (ms per forecast step) at batch size B=32 and ensemble size K=16 . Lower is better.
Figure 6: RMSCE and ∣log(SSR)∣ at σγ∈{0,0.1,0.5,1,10.0} for the Kolmogorov Flow, Burgers and KS datasets averaged over 10 rollout steps.
Figure 7: Calibration, predictive skill, and spread across ensemble sizes. Lower RMSCE and CRPS are better; the target spread–skill ratio is one.
Method / training regime
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
TORNADO (TF)
0.120
0.145
0.0395
0.268
TORNADO-AR (H=2)
0.139
0.157
0.0368
0.225
TORNADO-AR (H=4)
0.111
0.141
0.0318
0.180
Appendix
Table 11: Autoregressive extension of TORNADO on the Kolmogorov Flow dataset. Extending training beyond teacher forcing preserves the benefits of the base method and further improves calibration, with H=4 achieving the best overall performance.
Method
MSE ↓
CRPS ↓
RMSCE ↓
∣log(SSR)∣↓
TORNADO (NFE=5)
0.120
0.145
0.039
0.268
NLL
0.326
0.243
0.192
2.344
VAE matching
0.687
0.360
0.199
2.628
MC Dropout
0.321
0.250
0.075
0.447
Appendix
Table 12: Comparison with direct one-step latent baselines on the Kolmogorov Flow dataset.
Figure 8: 1D Wasserstein distance between the predicted and ground-truth total-enstrophy distributions over the rollout windows of length 20, aggregated over the test set for Kolmogorov Flow over 100 step trajectories. Lower is better.
Figure 9: Normalized total enstrophy over two representative Kolmogorov flow rollouts. Red curves show the ground truth; gray curves and shaded regions show the predictive ensemble mean and one standard deviation for each method.
Figure 10: 20 step pooled calibration error versus mean 90% predictive interval width on Kolmogorov flow.
Method
RMSCE ↓
PI width@95%
cov.@95%
SSR →1
CRPS ↓
DDIM
0.095
0.521
0.632
0.533
0.188
FM
0.203
0.156
0.284
0.225
0.163
SI
0.106
2.037
0.990
1.921
0.224
TORNADO
0.039
0.624
0.800
0.765
0.145
Appendix
Table 13: Calibration, sharpness, and predictive skill on Kolmogorov flow.
Figure 11: Reliability diagrams on Kolmogorov flow (NFE =5 , K=16 ), one-step (left) and over the 20-step rollout (right). Curves above the diagonal over-cover, below under-cover.
Table 14: Within-method tercile analysis of latent-space error and variance for TORNADO and SI.
Variable
Model
MSE (10−3)(↓)
CRPS (10−2)(↓)
RMSCE (10−2)(↓)
∣log(SSR)∣(↓)
T2m
DDIM
13.116 ± 2.634
5.60 ± 0.44
17.72 ± 2.16
0.787 ± 0.051
FM
13.367 ± 2.569
5.52 ± 0.43
15.91 ± 2.22
0.637 ± 0.046
SI
11.604 ± 2.214
4.90 ± 0.38
8.44 ± 1.48
0.292 ± 0.045
TORNADO
11.885 ± 2.212
4.94 ± 0.35
7.53 ± 1.21
0.265 ± 0.042
U10
DDIM
451.1 ± 40.7
35.06 ± 1.67
12.35 ± 0.51
0.655 ± 0.044
FM
460.3 ± 48.3
35.08 ± 1.89
11.00 ± 0.65
0.578 ± 0.048
Appendix
Table 15: ERA5 per-variable results on the held-out years (NFE =5 , K=16 , steps 1–40, 70 test trajectories). Metrics are in normalised, cos(lat) -weighted units; mean ± std over the test set. Best and second-best per variable and metric are highlighted ( green / blue ).
Figure 15: Kolmogorov flow rollout metrics vs. horizon t at fixed compute (NFE=2): MSE, CRPS, RMSCE, and SSR.
Figure 16: Kolmogorov flow rollout metrics vs. horizon t at fixed compute (NFE=10): MSE, CRPS, RMSCE, and SSR.
Figure 17: NFE sensitivity on Kolmogorov Flow. We report rollout MSE, CRPS, RMSCE, and SSR as a function of NFE used by method. Metrics are averaged over the same rollout horizon T=50 .
Figure 18: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Kolmogorov Flow data set.
Figure 19: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Kolmogorov Flow data set.
Figure 20: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Burgers data set.
Figure 21: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Burgers data set.
Figure 22: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the KS data set.
Figure 23: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the KS data set.
Figure 24: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the GS data set.
Figure 25: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the GS data set.
Figure 26: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the RealPDEBench Combustion data set.
Figure 27: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the RealPDEBench Combustion data set.
Figure 28: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the T2m variable WeatherBench ERA5 data set.
Figure 29: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the T2m variable WeatherBench ERA5 data set.
Figure 30: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the U10 variable WeatherBench ERA5 data set.
Figure 31: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the U10 variable WeatherBench ERA5 data set.
Figure 32: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the V10 variable WeatherBench ERA5 data set.
Figure 33: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the V10 variable WeatherBench ERA5 data set.
Figure 34: Visualization of rollout predictions at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the ground truth trajectory for the Z500 variable WeatherBench ERA5 data set.
Figure 35: Visualization of the 95% confidence interval of the predictive ensemble distribution at NFE=5 for methods TORNADO, FM, DDIM and SI alongside the RMSE of the mean of the predictive ensemble distribution with respect to the ground truth for the Z500 variable WeatherBench ERA5 data set.
Many scientific systems exhibit uncertainty from stochastic forcing, unresolved degrees of freedom, or imperfect observations, making reliable surrogate forecasting fundamentally distributional rather than pointwise. For such systems, deterministic neural surrogates fail to capture statistical measures and forecast uncertainty. We introduce TRIE, an evaluation framework for stochastic PDE surrogates that asks whether models reproduce invariant measures, provide trustworthy predictive uncertainty, and scale to efficient probabilistic generation. We demonstrate TRIE on two stationary chaotic spatially extended SPDEs, stochastic Kuramoto--Sivashinsky and stochastic Kolmogorov flow, across 11 parameter values. Our evaluation shows that standard pointwise-trained neural surrogates can produce plausible short rollouts while failing to match long-time statistical structure. Approximate uncertainty methods such as Monte Carlo dropout and heteroscedastic Gaussian likelihoods produce stochastic forecasts, but are often miscalibrated and overconfident under temporal and spatial uncertainty diagnostics. Across these criteria, generative models provide the most consistent performance, accurately capturing invariant measure statistics and achieving the lowest CRPS in all reported probabilistic settings. Finally, we show that latent generative models with automatic dimension discovery retain much of this statistical fidelity while reducing Kolmogorov inference time by roughly 12×. We release our code and data at https://github.com/scailab/TRIE-SPDE-Bench to support reproducible evaluation of stochastic PDE forecasting models.
Bharat Srikishan, Javier E. Santos, Nikhil Muralidhar +1
Stevens Institute of Technology · Los Alamos National Laboratory
Dominant approaches for modelling Partial Differential Equations (PDEs) rely on deterministic predictions, yet many physical systems of interest are inherently chaotic and uncertain. While training probabilistic models from scratch is possible, it is computationally expensive and fails to leverage the significant resources already invested in high-performing deterministic backbones. In this work, we adopt a training-efficient strategy to transform pre-trained deterministic models into probabilistic ones via retrofitting with a proper scoring rule: the Continuous Ranked Probability Score (CRPS). Crucially, this approach is architecture-agnostic: it applies the same adaptation mechanism across distinct model backbones with minimal code modifications. The method proves highly effective across different scales of pre-training: for models trained on single dynamical systems, we achieve 20-54% reductions in rollout CRPS and up to 30% improvements in variance-normalised RMSE (VRMSE) relative to compute-matched deterministic fine-tuning. We further validate our approach on a PDE foundation model, trained on multiple systems and retrofitted on the dataset of interest, to show that our probabilistic adaptation yields an improvement of up to 40% in CRPS and up to 15% in VRMSE compared to deterministic fine-tuning. Validated across diverse architectures and dynamics, our results show that probabilistic PDE modelling need not require retraining from scratch, but can be unlocked from existing deterministic backbones with modest additional training cost.
Cristiana Diaconu, Miles Cranmer, Richard E. Turner +2
The Polymathic AI Collaboration · University of Cambridge · Alan Turing Institute
Deterministic neural networks and neural operators provide point predictions with no intrinsic measure of reliability. Yet, predictive uncertainty may stem from irreducible outcome variability, finite data, or limitations of the chosen model class. Monte Carlo dropout offers a computationally convenient way to construct a predictive distribution through stochastic feature masking, without training multiple independent networks or explicitly inferring a posterior over model parameters. However, its perturbation law is largely prescribed a priori and typically factorised across latent coordinates. We introduce Optimal Transport Dropout (OTD), which instead learns the predictive mapping and the law of its latent perturbations jointly. Starting from a simple independent reference distribution, OTD transports latent perturbations through a learnable flow and propagates them through the predictive neural network, thereby inducing a structured predictive law. Training uses the strictly proper Energy Score, while a kinetic-action term geometrically regularises the transport. Synthetic benchmarks show that OTD captures multimodal predictive distributions, generates meaningful dispersion when the model is misspecified, and exhibits contracting dispersion as more training data or greater model capacity are provided. For a field-valued partial differential equation surrogate, predictive dispersion strongly aligns with the spatial pattern of prediction errors. On this task, compared with Monte Carlo dropout, OTD yields more accurate predictions and better-calibrated, substantially narrower intervals. On real-world regression benchmarks, it further shows competitive accuracy and better probabilistic predictions compared to established baselines. OTD therefore offers a way to learn structured predictive uncertainty without explicit posterior inference or ensembles of independently trained predictors.
Giacomo Lorenzon, Francesco Regazzoni
MOX–Department of Mathematics, Politecnico di Milano, Italy