Precipitation nowcasting (short-term forecasting) is still often performed using numerical solvers for physical equations, which are computationally expensive and make limited use of the large volumes of available weather data. Deep learning models have shown strong potential for precipitation nowcasting, offering both accuracy and computational efficiency. Among these models, convolutional neural networks (CNNs) are particularly effective for image-to-image prediction tasks. The SmaAt-UNet is a lightweight CNN based architecture that has demonstrated strong performance for precipitation nowcasting. This paper introduces the Multimodal Advection-Guided Small Attention GNet (MAD-SmaAt-GNet), which extends the core SmaAt-UNet by (i) incorporating an additional encoder to learn from multiple weather variables and (ii) integrating a physics-based advection component to ensure physically consistent predictions. We show that each extension individually improves rainfall forecasts and that their combination yields further gains. MAD-SmaAt-GNet reduces the mean squared error (MSE) by 8.9% compared with the baseline SmaAt-UNet for four-step precipitation forecasting up to four hours ahead. Additionally, experiments indicate that multimodal inputs are particularly beneficial for short lead times, while the advection-based component enhances performance across both short and long forecasting horizons.
Figures & tables
Figure 1: The architecture of (a) MAD-SmaAt-GNet when given 4 rain images and corresponding images of other weather variables and a target of 4 output images (best viewed in colour). Additionally, the architecture of (b) the evolution network with the same rain inputs. Each bar represents a sequence of images or feature maps with corresponding dimensions. The number of channels is given above the bars and the height and width of the images on the left of the bars (unless unchanged).
Model
Number of parameters
Relative model size
SmaAt-UNet
4,110,400
1×
MAD-SmaAt- GNet
7,453,676
1.81×
SmaAt-UNet with Evo-Net
3,745,936
0.91×
SmaAt-UNet with 2-stream
4,766,624
1.16×
Evo-Net
2,219,500
0.54×
Persistence
−
−
Table 1: Overview of the models and their model sizes in number of parameters.
Figure 2: Plots of a rain image before and after the cropping. Note that the coordinates start in the top-left corner and that the images are vertically flipped compared to a standard map of the Netherlands.
Figure 3: Plots of the other weather variables used as inputs for the models with two streams. Note that the coordinates start in the top-left corner and that the images appear vertically flipped (this is best seen in the image of relative humidity where the map of the Netherlands is somewhat visible from the image).
Dataset
Filter
Number of training samples
Number of test samples
%
No filter
33,743
8,270
100%
20%>0.1mm/h
5,925
1,883
18.6%
Table 2: Size of the filtered dataset compared to the whole dataset.
Model
MSE ↓
Accuracy ↑
Precision ↑
Recall ↑
F1 ↑
CSI ↑
MCC ↑
Persistence
1.0625
0.7989
0.4271
0.5248
0.4709
0.3080
0.3512
SmaAt-UNet
0.4721
0.8586
0.5932
0.5440
0.5675
0.3962
0.4839
MAD-SmaAt-GNet
0.4299
0.8672
0.6123
0.6033
0.6078
0.4366
0.5279
SmaAt-UNet with Evo-Net
0.4352
0.8652
0.6034
0.6125
0.6079
0.4367
0.5266
SmaAt-UNet with 2-stream
0.4599
0.8591
0.5861
0.5914
0.5888
0.4172
0.5038
Evo-Net
0.4884
0.8581
0.5835
0.5874
0.5854
0.4139
0.4998
Table 3: Test results for the models on the task of predicting a sequence of four consecutive rain images. All classification metrics were computed using the threshold of 0.5mm/h . Metrics where lower values indicate better performance are marked with a ↓ symbol, while those where higher is better are marked with a ↑ symbol. The best results are shown in bold, and the second-best results are underlined.
Figure 4: Plot of the MSEs of the models per time step. The persistence model is not included in the plot due to its much higher MSEs which would skew the plot.
Figure 5: The predictions of the models for a certain test sample and the corresponding ground-truth images. The images form a sequence of predictions from 1 to 4 hours ahead.
Weather forecasting supports critical socioeconomic activities and complements environmental protection, yet operational Numerical Weather Prediction (NWP) systems remain computationally intensive, thus being inefficient for certain applications. Meanwhile, recent advances in deep data-driven models have demonstrated promising results in nowcasting tasks. This paper presents SmaAT-QMix-UNet, an enhanced variant of SmaAT-UNet that introduces two key innovations: a vector quantization (VQ) bottleneck at the encoder-decoder bridge, and mixed kernel depth-wise convolutions (MixConv) replacing selected encoder and decoder blocks. These enhancements both reduce the model's size and improve its nowcasting performance. We train and evaluate SmaAT-QMix-UNet on a Dutch radar precipitation dataset (2016-2019), predicting precipitation 30 minutes ahead. Three configurations are benchmarked: using only VQ, only MixConv, and the full SmaAT-QMix-UNet. Grad-CAM saliency maps highlight the regions influencing each nowcast, while a UMAP embedding of the codewords illustrates how the VQ layer clusters encoder outputs. The source code for SmaAT-QMix-UNet is publicly available on GitHub: https://github.com/nstavr04/MasterThesisSnellius.
Nikolas Stavrou, Siamak Mehrkanoon
Department of Information and Computing Sciences, Utrecht University, Utrecht, Netherlands
Accurate and timely rainfall nowcasting is crucial for disaster mitigation and water resource management. Despite recent advances in deep learning, precipitation prediction remains challenging due to limitations in effectively leveraging diverse multimedia data sources. We introduce M3R, a Meteorology-informed MultiModal attention-based architecture for direct Rainfall prediction that synergistically combines visual NEXRAD radar imagery with numerical Personal Weather Station (PWS) measurements, using a comprehensive pipeline for temporal alignment of heterogeneous meteorological data. With specialized multimodal attention mechanisms, M3R novelly leverages weather station time series as queries to selectively attend to spatial radar features, enabling focused extraction of precipitation signatures. Experimental results for three spatial areas of 100 km * 100 km centered at NEXRAD radar stations demonstrate that M3R outperforms existing approaches, achieving substantial improvements in accuracy, efficiency, and precipitation detection capabilities. Our work establishes new benchmarks for multimedia-based precipitation nowcasting and provides practical tools for operational weather prediction systems. The source code is available at https://github.com/Sanjeev97/M3Rain
Sanjeev Panta, Rhett M Morvant, Xu Yuan +2
University of Louisiana at Lafayette, Lafayette, LA, USA · University of Delaware, Newark, DE, USA
Proper short-term forecasting of precipitation is crucial in disaster management and preparedness. Nonetheless, the variability and nonlinearity of precipitation make short-term forecasting challenging for meteorologists. Moreover, capturing temporal dependencies in spatiotemporal data is a challenge in precipitation nowcasting. In this article, we introduce a lightweight deep learning model for half-hourly precipitation nowcasting. This model has been designed by incorporating the DenseNet architecture, residual connections, and transformer encoders for effective precipitation nowcasting with reduced model parameters. The North-Eastern region of India has been selected as the area of interest for our study. The region receives the highest precipitation during the months of June-September due to the monsoon season. The proposed model takes the previous five time-steps of half-hourly precipitation as inputs and predicts the precipitation in the next two half-hours. The GPM IMERG precipitation dataset with a 30-minute cadence has been used in this study for training and testing the model. The proposed architecture achieves best MAE of 0.235 millimetres, RMSE of 0.735 millimetres, and KGE score of 0.816 at an interval of 30 minutes.
Gourav Jyoti Kalita, Hidam Kumarjit Singh
Department of Electronics and Communication Technology, Gauhati University, Guwahati-781014, Assam, India