Developing Machine Learning-Based Watch-to-Warning Severe Weather Guidance from the Warn-on-Forecast System
Authors: Montgomery Flora, Samuel Varga, Corey Potvin, Noah Lang
Organizations: Cooperative Institute for Severe and High-Impact Weather Research and Operations, University of Oklahoma, Norman, Oklahoma · NOAA/OAR/National Severe Storms Laboratory, Norman, Oklahoma · AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography · Current Affiliation: The Weather Company, Atlanta, Georgia · School of Meteorology, University of Oklahoma, Norman, Oklahoma · Current Affiliation: Versik Analytics, Boston, Massachusetts · Valparaiso University, Valparaiso, Indiana · Current Affiliation: University of Wisconsin-Madison, Madison, Wisconsin
While machine learning (ML) post-processing of convection-allowing model (CAM) output for severe weather hazards (large hail, damaging winds, and/or tornadoes) has shown promise for very short lead times (0-3 hours), its application to slightly longer forecast windows remains relatively underexplored. In this study, we develop and evaluate a grid-based ML framework to predict the probability of severe weather hazards over the next 2-6 hours using forecast output from the Warn-on-Forecast System (WoFS). Our dataset includes WoFS ensemble forecasts valid every 5 minutes out to 6 hours from 108 days during the 2019--2023 NOAA Hazardous Weather Testbed Spring Forecasting Experiments. We train ML models to generate probabilistic forecasts of severe weather akin to Storm Prediction Center outlooks (i.e., likelihood of a tornado, severe wind, or severe hail event within 36 km of each point). We compare a histogram gradient-boosted tree (HGBT) model and a deep learning U-Net approach against a carefully calibrated baseline generated from 2-5 km updraft helicity. Results indicate that the HGBT and U-Net outperform the baseline, particularly at higher probability thresholds. The HGBT achieves the best performance metrics, but predicted probabilities cap at 60% while the U-net forecasts extend to 100%. Similar to previous studies, the U-Net produces spatially smoother guidance than the tree-based method. These findings add to the growing evidence of the effectiveness of ML-based CAM post-processing for providing short-term severe weather guidance.
Figures & tables
Figure 1: Heatmap showing the frequency of a 1 ° × 1 ° region within a 900 x 900 km WoFS domain during the 2019–2023 HWT-SFEs.
Figure 2: Illustration of the data pre-processing workflow for the tabular model inputs (left) and the U-net inputs (right). The data arrays sizes ( n ) for a single sample are given. The subscripts are t = time, ( y,x )= (latitude, longitude), e = ensemble, v = variable/channel. The combined xy indices indicate the common 2D array flattening to create tabular inputs. The ensemble statistics used are provided in the bottom table.
Intra-storm
Environment
Updraft Helicity (0–2 km, 2–5 km)
Lapse Rates (0–3 km, 700–500 mb)
Composite Reflectivity
0–3-km Storm-Relative Helicity
0–2-km Average Vertical Vorticity
U Shear (0–6 km, 0–1 km, 3–6 km)
Column-maximum Updraft
V Shear (0–6 km, 0–1 km, 3–6 km)
80-m Wind Speed
Significant Tornado Parameter
HAILCAST
Supercell Composite Parameter
Table 1: Input variables from the WoFS. CAPE is convective available potential energy, CIN is convective inhibition, and LCL is the lifted condensation level. HAILCAST refers to the maximum hail diameter from WRF-HAILCAST ( Adams-Selin and Ziegler, 2016 ; Adams-Selin et al., 2019 ) . Parentheses indicate different vertical levels or layers where data were extracted.
Figure 3: Illustration of the U-net architecture used in this study. Image and channel sizes are provided on each rectangle’s lower left-hand side and top. A DoubleConv layer comprises 2D convolution, batch normalization, ReLU, dropout, 2D convolution, batch normalization, and then ReLU again. For skip connections, the output of a given encoder layer is passed directly to the corresponding decoding layer and concatenated to the UpConv layer input. Skip connections are only connected across the U-net to the level with a similar spatial dimension.
Figure 4: Heatmaps of the NMEP cross-validation (cv)-mean Brier Skill Score (BSS) for choice of 2–5 km UH threshold and neighborhood size. The BSS is calculated as the mean BSS across all validation folds using 5-fold cross-validation on the training set. The highest BSS is highlighted with a black outline.
Figure 5: Example forecasts from the U-Net (upper left), HGBT (upper right), and WoFS baseline NMEP (lower left). The guidance was issued for 12 May 2023 22-01 UTC from WoFS forecasts initialization at 20:00 UTC. SPC categorical outlooks issued at 1630 UTC the same day are overlaid as contours. The MRMS composite reflectivity at model initialization time is also shown. NCEI Storm Data reports of severe hail, severe wind, and tornadoes are shown in green, blue, and red markers, respectively.
Figure 6: The performance curves of the U-net, HGBT, and baseline (BL) are shown in blue, red, and black, respectively. POD,SR points of the three models are shown for thresholds of 5%, 20%, 50%, and 75%. The 95% confidence interval is provided for each curve. The no-skill region, defined by the base rate, is shaded. The NAUPDC and maximum NCSI are provided in the inset table.
Figure 7: The reliability curves of the U-net, HGBT, and baseline (BL) are shown in blue, red, and black, respectively. The background histograms provide the sample size in each bin (defined every 10%) for all three models (values shown the right y-axis).
Hyperparameter
Values
Learning Rate
0.0001, 0.001, 0.01, 0.1
Max. Leaf Nodes
5 , 10, 20, 30, 40, 50
Max. Depth
4, 6 , 8, 10
Min. Samples Leaf
5, 10 ,15,20,30, 40, 50
L2 Regularization
0.001, 0.01 , 0.1
Max. Bins
15, 31, 63 , 127
Table 2: Hyperparameter values attempted for the HGBT in the hyperparameter search. Selected values are bolded.
Hyperparameter
Values
Kernel Size
3
Dropout Rate
U(0,0.4)→ 0.3
Initial Filter Size
32, 64 , 128, 256
Num. of Layers
1,2,3, 4
Learning Rate
0.0001, 0.001 , 0.01, 0.1
Table 3: Hyperparameter values attempted for the U-net in the hyperparameter search. Selected values are bolded.
AI-based weather forecasting now rivals traditional physics-based ensembles, but state-of-the-art (SOTA) models rely on specialized architectures and massive computational budgets, creating a high barrier to entry. We demonstrate that such complexity is unnecessary for frontier performance. We introduce \ours, a probabilistic forecaster built on a standard U-Net backbone trained with a simple recipe: deterministic pre-training on Mean Absolute Error followed by short probabilistic fine-tuning on the Continuous Ranked Probability Score (CRPS) using Monte Carlo Dropout for stochasticity. As a result, our model matches or exceeds the probabilistic skill of GenCast and IFS ENS at 1.5∘ resolution while reducing training compute by over 10× compared to leading CRPS-based models and inference latency by over 10× compared to diffusion-based models. U-Cast trains in under 12 H200 GPU-days and generates a 15-day ensemble forecast in 3 seconds. These results suggest that scalable, general-purpose architectures paired with efficient training curricula can match complex domain-specific designs at a fraction of the cost, opening the training of frontier probabilistic weather models to the broader community.
Salva Rühling Cachay, Duncan Watson-Parris, Rose Yu
To address insufficient contextualization, weak generalization, and poor scenario adaptation in tourism meteorological services, we propose SmartWeatherAgent--a unified three-stage architecture integrating intent recognition, hazard prediction, and reasoning-enhanced generation. The system fuses rule-based methods with large language models to parse queries at multiple granularities and employs a LightGBM model enriched with highland-specific features (e.g., wind speed abruptness rate), achieving an F1-Macro score of 0.605 with 1.60 ms latency on high-wind, precipitation, and low-temperature events. A 12-round micro-step prompt self-optimization loop boosts the composite warning quality score S_final from 4.2 (B01) to 8.9 (B12, +112%). Key improvements include a sharp rise in B08 from data source citation (6.5 -> 8.5), sustained high performance in B10 via physical mechanism explanation, and a peak scientific rigor score of 9.2 in B12 through explicit uncertainty statements. The system autonomously generates structured warnings that integrate causal mechanisms, spatiotemporal evolution, quantitative evidence, regulatory references, and confidence statements--enhancing professional depth, logical rigor, and scientific soundness, and advancing meteorological services toward proactive perception, explainable decision-making, and intelligent agency.
Shuai Yan, Yang Xu, Shan He
Chengdu Jincheng College College of Computer and Software Chengdu, Sichuan, China
Accurate short-term forecasting of atmospheric seeing is of significant interest for astronomical observations and free-space optical communication systems, where rapid variations in optical turbulence can affect system performance and operational planning. In this work, we benchmark machine learning models for predicting atmospheric seeing up to two hours in advance, with a particular focus on probabilistic forecasting and uncertainty quantification. We compare two probabilistic approaches, Gaussian Processes (GPs) and a normalizing-flow- based model (FloTS), against deterministic recurrent neural network (RNN) and long short-term memory (LSTM) baselines. All models are trained and evaluated on seeing measurements from the Maunakea summit. We assess the models using both deterministic metrics, including the root mean square error and the Pearson correlation coefficient, and probabilistic metrics that evaluate the quality and calibration of predictive uncertainty estimates. The results show that deep learning approaches outperform standard recurrent networks for short-term forecasting, with LSTM and FloTS achieving the best overall predictive accuracy. While Gaussian Processes provide a useful probabilistic baseline, their predictive distributions are limited by the Gaussian assumption. FloTS combines competitive forecasting accuracy with flexible uncertainty quantification and produces the most informative probabilistic forecasts.
Mary-Joe Medlej, Rahul Srinivasan, Simon Prunet +2
Universit´e Cˆote d’Azur, Observatoire de la Cˆote d’Azur, J.L. Lagrange, Parc Valrose 06108 Nice Cedex 2, France · SISSA, Via Bonomea 265, 34136 Trieste, Italy · IFPU - Institute for Fundamental Physics of the Universe, Via Beirut 2, 34014 Trieste, Italy