Diffusion Distillation for Efficient Weather Ensembles

Yiming Yang; Valentin Brekke; James Briant; Serge Guillas

cs.LGAug 27, 2026

Diffusion Distillation for Efficient Weather Ensembles

Authors: Yiming Yang, Valentin Brekke, James Briant, Serge Guillas

Organizations: Department of Statistical Science University College London · MPLS Doctoral Training Centre University of Oxford

Abstract

Diffusion models generate skillful weather ensembles but require costly iterative sampling. We introduce a supervised energy-distance distillation method that compresses a multi-step diffusion teacher into a single-step student by aligning student forecasts with teacher samples and ground-truth observations. Experiments on global forecasting and typhoon-track prediction show that our student outperforms existing distillation methods and preserves skill for extreme events. It matches or surpasses the teacher across key metrics using only one neural function evaluation per autoregressive step.

Figures & tables

Figure 1: Overview of our distillation framework. Our objective aligns the single-pass student with teacher forecasts through distribution distance and improves accuracy using ground-truth supervision.

Figure 3: Global forecast skill across representative variables. Shading denotes

\pm 1

standard deviation; lower RMSE/CRPS and a spread–skill ratio closer to one are better. ENS (WB2) denotes the 50-member ECMWF IFS ensemble.

Figure 4: Typhoon Nanmadol track and peak-intensity forecasts. Top: observed and predicted tracks, with thin lines showing ensemble members and bold lines their means. Bottom: MSLP and 10-m winds at peak intensity, with crosses marking cyclone centers.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Peak learning rate	$5.0\times 10^{-5}$
Weight decay	$0.01$
Learning rate schedule	Cosine decay
Total training steps	$20{,}000$
EMA decay	$0.999$

Appendix

Table 1: Optimizer hyperparameters.

Figure 5: Typhoon Nanmadol track errors against the IBTrACS best track. Top: position error. Bottom: ensemble position CRPS, computed as the multivariate energy score over track locations.

Figure 6: Relative RMSE and CRPS of our student compared with the GenCast teacher across variables, pressure levels, and lead times. Blue indicates improved skill and red indicates degradation.

Figure 7: Spatial forecast errors for MSLP, Q700, T850, Z500, U10M, and V10M at lead times of 1, 3, 5, and 10 days, comparing the GraphCast , GenCast teacher, our student, CM, and DMD against ERA5.

Figure 8: Spatial ensemble means and variances for Z500, U10M, and V10M. Columns show lead times of 1, 3, 5, and 10 days, and rows compare the multi-step GenCast teacher, our distilled student, CM, and DMD.

Figure 9: Spatial ensemble means and variances for MSLP, Q700, and T850. Columns show lead times of 1, 3, 5, and 10 days, and rows compare the multi-step GenCast teacher, our distilled student, CM, and DMD.

	GraphCast	GenCast (Original, NFE=39)	GenCast (Ours, NFE=1)
Time (s)	$9.83\pm 0.14$	$112.46\pm 0.19$	$21.44\pm 0.16$
Ensemble Size	1	20	20

Appendix

Table 2: Wall-clock time per autoregressive step on one NVIDIA GH200 with batch size 1. Values report mean

\pm

standard deviation over 10 runs, excluding JIT compilation, for the ensemble sizes shown.

Diffusion Model Distillation

Diffusion model distillation transfers a large model’s generative behavior to faster or smaller models that need fewer sampling steps.

See all 189 papers

Proper Scoring Rule-based Diffusion for Probabilistic Weather Forecasting
Sep 29, 2026Joonhyeong Park, Giung Nam, Hyungi Lee +3Probabilistic Forecasting Proper Scoring Rules
KAIST · Kookmin University · New York University
Long-Range Distillation: Distilling 10,000 Years of Simulated Climate into Long Timestep AI Weather Models
Dec 28, 2025Scott A. Martin, Noah Brenowitz, Dale Durran +1Weather Forecasting Spatiotemporal Forecasting
NVIDIA Research, Santa Clara, CA, USA · School of Oceanography, University of Washington, Seattle, WA, USA · Department of Atmospheric & Climate Science, University of Washington, Seattle, WA, USA
Just Weather Scoring: Efficient End-to-end Nowcasting with Distributional Diffusion
Oct 8, 2026Jannik Wiese, Johannes Schusterbauer, Tommaso Martorella +1Precipitation Nowcasting Probabilistic Forecasting

Sep 29, 2026J/K move · Enter open · S save

Proper Scoring Rule-based Diffusion for Probabilistic Weather Forecasting

Joonhyeong Park, Giung Nam, Hyungi Lee, Kyunghyun Cho+2

KAIST · Kookmin University · New York University

Probabilistic Forecasting Proper Scoring Rules Conditional Diffusion Models Weather Forecasting

Open paper AskPDF arXiv

Recent probabilistic weather forecasters train stochastic predictors with the continuous ranked probability score (CRPS) to generate each ensemble member in a single forward pass. These models learn the predictive distribution from the forecast context alone, which becomes difficult at longer forecast horizons where uncertainty is high. To learn the predictive distribution more effectively, we introduce auxiliary conditional denoising tasks that predict the same future state from the context and its corrupted version, which provides partial future information that can reduce prediction ambiguity. Building on distributional diffusion models, we learn the conditional distributions of these tasks with a single stochastic predictor by minimizing a proper scoring rule across noise levels. At inference, the predictor can still generate each ensemble member in a single forward pass at the fully corrupted endpoint. Standard CRPS training is recovered as the endpoint-only special case of our formulation, so our framework extends existing CRPS-based forecasters with only additional conditioning inputs. Controlled experiments show that the auxiliary tasks improve one-step forecasting across architectures, with larger gains at longer forecast horizons. The gains extend to high-dimensional global weather forecasting under both training from scratch and fine-tuning, along with improved calibration and potential benefits for generalization under distribution shift.

Abstract

Diffusion Model Distillation

Explore similar work

Proper Scoring Rule-based Diffusion for Probabilistic Weather Forecasting

Long-Range Distillation: Distilling 10,000 Years of Simulated Climate into Long Timestep AI Weather Models

Just Weather Scoring: Efficient End-to-end Nowcasting with Distributional Diffusion