Data scarcity is a primary obstacle in developing robust Machine Learning (ML) models for detecting rapidly intensifying tropical cyclones. Traditional data augmentation techniques (rotation, flipping, brightness adjustment) fail to preserve the physical consistency and high-intensity gradients characteristic of rare Category 4-equivalent events, which constitute only 0.14% of our dataset (202 of 140,514 samples). We propose a physics-informed diffusion model based on the Context-UNet architecture to generate synthetic, multi-spectral satellite imagery of extreme weather events. Our model is conditioned on critical atmospheric parameters such as average wind speed, type of Ocean and stage of development (early, mature, late etc) -- the known drivers of rapid intensification. Using a controlled pre-generated noise sampling strategy and mixed-precision training, we generated 16×16 wind-field samples that are cropped from multi-spectral satellite imagery which preserve realistic spatial autocorrelation and physical consistency. Results demonstrate that our model successfully learns discriminative features across ten distinct context classes, effectively mitigating the data bottleneck. Specifically, we address the extreme class imbalance in our dataset, where Class 4 (Ocean 2, early stage with average wind speed 50kn hurricane) contains only 202 samples compared to 79,768 samples in Class 0. This generative framework provides a scalable solution for augmenting training datasets for operational weather detection algorithms. The average Results yield an average Log-Spectral Distance (LSD) of 4.5dB, demonstrating a scalable framework for enhancing operational weather detection algorithms.
Figures & tables
Figure 1: Figure illustrates the three-phase pipeline for generating synthetic extreme events: Forward Process: x0 (original wind field) is transformed into Gaussian noise xt using a pre-generated noise strategy. Context-UNet Training: A UNet ( n_feat=64 ) predicts noise by conditioning the noisy input ( xt ) on sinusoidal time steps ( t ) and physics-based context ( c ), such as one-hot encoded wind shear classes. Reverse Process: The trained model performs 500 iterative denoising steps starting from xT to produce a synthetic extreme event x0^ .
Figure 2: Visual and statistical verification of eight random noise samples. Each patch represents a 16×16 realization of ϵ∼N(0,1) . The consistent Gaussian distribution across samples ensures that rare event detection is not biased by noise artifacts.
Parameter
Value
Batch size
64
Training epochs
120
Initial learning rate ( ηmax )
1×10−4
Minimum learning rate ( ηmin )
1×10−6
Optimizer
Adam
EMA decay coefficient ( βEMA )
0.995
Table 1: Core Training Hyperparameters
Figure 3: Cyclone event capture from emergence to dissipation. Note: Images were originally 1024×1024 pixels and have been cropped to 16×16 pixels for training.
Figure 4: Storm Wind Fields representing physical intensity on a 16×16 grid. Samples are conditioned on specific atmospheric parameters (Param 0 and 1) representing Rapid Intensification (RI) conditions. The colormap highlights the high-intensity gradients preserved by the physics-informed model.
Figure 5: Distribution of unique physical parameter labels within the dataset. The extreme scarcity of samples in Class 4 (202 samples) demonstrates the data bottleneck for rare weather events that this research aims to mitigate through synthetic augmentation.
Parameter
Value
Architecture
Context U-Net
Input channels
1 (grayscale)
Feature dimension ( nfeat )
64
Context features ( ncfeat )
5 (standard), 10 (context)
Image resolution
16 × 16 pixels
Activation function
SiLU (Swish)
Table 2: Model architecture configuration parameters.
Parameter
Value
Batch size
64
Training epochs
120
Initial learning rate
10−4
Optimizer
Adam
Adam β1
0.9
Adam β2
0.999
Table 3: Training hyperparameters and optimization settings.
Component
Specification
Model Architecture
Model
Context U-Net
Input channels
1 (grayscale)
Base features
64
Context features
5
Spatial resolution
16×16
Table 4: Complete Training Configuration Summary
Context
Ocean
Stage
Avg. Wind Speed
Description
7, 9
1
Early
47 kn
Initial formation/Early development
0
1
Mid-Early
51 kn
Intensifying tropical system
1
1
Mature
56 kn
Peak intensity phase
6, 5
1
Late
42 kn
Long-duration, weakening storms
3
1
Extreme
43 kn
Rare, ultra-long-lived systems
2, 4
2
Early
50 kn
Standard development in Ocean 2
Table 5: Storm lifecycle characteristics across different contexts and ocean basins
Figure 6: Context-conditioned generation for labels 0-9. Each image represents a wind field pattern generated conditioned on its corresponding class label using one-hot encoding. The model successfully produces distinct patterns for different context inputs.
Figure 7: Multiple samples generated for specific context labels. (a) Context label 1 produces predominantly low-intensity, smooth gradient fields. (b) Context label 5 generates high-contrast patterns with distinct spatial structures and intensity clusters.
Figure 8: As the conditioning increases to epoch 116, the U-Net backbone successfully synthesizes well-defined localized gradients and cellular structures. At lower conditioning levels, the model produces high-frequency, low-contrast outputs indicative of early-stage training or weak prior guidance.
While machine learning-based weather models hold significant promise, they struggle to predict the detailed structure of large-scale weather systems such as cyclonic storms. Regional models are constrained by limited historical records within fixed geographic boundaries, while global models are computationally expensive and often operate at resolutions too coarse to capture fine-grained storm dynamics. To bridge this gap, we introduce the Geospatial Diffusion-based Evolution Synthesis (GeoDES) model, a custom image-to-video diffusion model. By focusing generation strictly on the evolving storm structure, GeoDES synthesizes physically consistent, high-fidelity weather events suitable for stress-testing forecast models and expanding meteorological datasets. Evaluations demonstrate that GeoDES outperforms prior methods on key metrics, achieving 52% lower Peak Vorticity Error and 8% higher Anomaly Correlation Coefficient than the next strongest methods on the North Atlantic test set.
Sonia Cromp, Satya Sai Srinath Namburi GNVV, Youran Wang +4
University of Wisconsin-Madison · GE HealthCare · University of Georgia +2
Microwave satellite imagery plays a crucial role in monitoring tropical cyclone precipitation and intensity worldwide, but suffers from long revisit times, potentially missing rapid storm evolution phases. While this raises the need for an interpolation method, it is made challenging by the high level of heterogeneity of microwave data coming from different instruments. In this work, we introduce the first generative model that can be applied to multiple geospatial sources that change across samples, occur at irregular time intervals, are misaligned geographically, and come from instruments with varying characteristics. We apply this model to the case of spatio-temporal interpolation of tropical cyclone microwave images from other microwave and infrared instruments. We train using a self-supervised task in which a random source is masked and reconstructed, and show that it leads to a significant decrease in Continuous Ranked Probability Score over supervised training. We show a further improvement by combining infrared and microwave data compared to microwave only. Using these improvements, the generative model produces an ensemble mean on par with that of a deterministic model, while generating a power spectrum significantly closer to that of true observations. To the best of our knowledge, this is the first generative model that interpolates microwave images of cyclones by combining multiple microwave instruments and infrared observations at irregular time intervals.
Clément Dauvilliers, Claire Monteleoni
Inria · INRIA, Paris, Île-de-France, France · University of Colorado Boulder, Boulder, CO, USA.
Class imbalance is a persistent challenge in visual recognition, particularly in safety-critical domains where collecting positive examples is expensive and rare events are inherently underrepresented. We propose a lightweight synthetic data augmentation pipeline that fine-tunes a LoRA adapter on as few as 20-50 real images of a rare class and uses a pretrained diffusion model to generate synthetic samples for training. We systematically vary the synthetic-to-real ratio and evaluate the approach across two structurally different domains: chest X-ray pathology classification (NIH ChestX-ray14) and industrial surface crack detection (Magnetic Tile Defect dataset). All evaluations are performed on held-out sets of real images only. Across both domains, synthetic augmentation consistently improves rare-class recall and F1 compared to training with real data alone. Performance improves with moderate synthetic augmentation and shows diminishing returns as the synthetic ratio increases. These results suggest that LoRA-adapted diffusion models provide a simple and scalable mechanism for augmenting rare classes, enabling effective learning in data-scarce scenarios across heterogeneous visual domains.
Daniil Dushenev, Nazariy Karpov, Daniil Zinovjev +2
National University of Science and Technology MISIS, Moscow, Russia