Precipitation nowcasting (short-term forecasting) is still often performed using numerical solvers for physical equations, which are computationally expensive and make limited use of the large volumes of available weather data. Deep learning models have shown strong potential for precipitation nowcasting, offering both accuracy and computational efficiency. Among these models, convolutional neural networks (CNNs) are particularly effective for image-to-image prediction tasks. The SmaAt-UNet is a lightweight CNN based architecture that has demonstrated strong performance for precipitation nowcasting. This paper introduces the Multimodal Advection-Guided Small Attention GNet (MAD-SmaAt-GNet), which extends the core SmaAt-UNet by (i) incorporating an additional encoder to learn from multiple weather variables and (ii) integrating a physics-based advection component to ensure physically consistent predictions. We show that each extension individually improves rainfall forecasts and that their combination yields further gains. MAD-SmaAt-GNet reduces the mean squared error (MSE) by 8.9% compared with the baseline SmaAt-UNet for four-step precipitation forecasting up to four hours ahead. Additionally, experiments indicate that multimodal inputs are particularly beneficial for short lead times, while the advection-based component enhances performance across both short and long forecasting horizons.
Figures & tables
Figure 1: The architecture of (a) MAD-SmaAt-GNet when given 4 rain images and corresponding images of other weather variables and a target of 4 output images (best viewed in colour). Additionally, the architecture of (b) the evolution network with the same rain inputs. Each bar represents a sequence of images or feature maps with corresponding dimensions. The number of channels is given above the bars and the height and width of the images on the left of the bars (unless unchanged).
Model
Number of parameters
Relative model size
SmaAt-UNet
4,110,400
1×
MAD-SmaAt- GNet
7,453,676
1.81×
SmaAt-UNet with Evo-Net
3,745,936
0.91×
SmaAt-UNet with 2-stream
4,766,624
1.16×
Evo-Net
2,219,500
0.54×
Persistence
−
−
Table 1: Overview of the models and their model sizes in number of parameters.
Figure 2: Plots of a rain image before and after the cropping. Note that the coordinates start in the top-left corner and that the images are vertically flipped compared to a standard map of the Netherlands.
Figure 3: Plots of the other weather variables used as inputs for the models with two streams. Note that the coordinates start in the top-left corner and that the images appear vertically flipped (this is best seen in the image of relative humidity where the map of the Netherlands is somewhat visible from the image).
Dataset
Filter
Number of training samples
Number of test samples
%
No filter
33,743
8,270
100%
20%>0.1mm/h
5,925
1,883
18.6%
Table 2: Size of the filtered dataset compared to the whole dataset.
Model
MSE ↓
Accuracy ↑
Precision ↑
Recall ↑
F1 ↑
CSI ↑
MCC ↑
Persistence
1.0625
0.7989
0.4271
0.5248
0.4709
0.3080
0.3512
SmaAt-UNet
0.4721
0.8586
0.5932
0.5440
0.5675
0.3962
0.4839
MAD-SmaAt-GNet
0.4299
0.8672
0.6123
0.6033
0.6078
0.4366
0.5279
SmaAt-UNet with Evo-Net
0.4352
0.8652
0.6034
0.6125
0.6079
0.4367
0.5266
SmaAt-UNet with 2-stream
0.4599
0.8591
0.5861
0.5914
0.5888
0.4172
0.5038
Evo-Net
0.4884
0.8581
0.5835
0.5874
0.5854
0.4139
0.4998
Table 3: Test results for the models on the task of predicting a sequence of four consecutive rain images. All classification metrics were computed using the threshold of 0.5mm/h . Metrics where lower values indicate better performance are marked with a ↓ symbol, while those where higher is better are marked with a ↑ symbol. The best results are shown in bold, and the second-best results are underlined.
Figure 4: Plot of the MSEs of the models per time step. The persistence model is not included in the plot due to its much higher MSEs which would skew the plot.
Figure 5: The predictions of the models for a certain test sample and the corresponding ground-truth images. The images form a sequence of predictions from 1 to 4 hours ahead.