Organizations: School of Computer Science and Artificial Intelligence, Wuhan University of Technology · School of Earth and Space Science and Technology, Wuhan University
Deep learning has significantly improved the accuracy of precipitation nowcasting. However, most existing multimodal models typically use simple channel concatenation or interpolation methods for data fusion, which often overlook the feature differences between different modalities. This paper therefore proposes a novel precipitation nowcasting optimisation framework called FusionCast. This framework incorporates three types of data: historical precipitable water vapour (PWV) data derived from global navigation satellite system (GNSS) inversions, historical radar based quantitative precipitation estimation (QPE), and forecasted radar QPE serving as a future prior. The FusionCast model comprises two core modules: the future prior radar QPE processing Module, which forecasts future radar data; and the Radar PWV Fusion (RPF) module, which uses a gate mechanism to efficiently combine features from various sources. Experimental results show that FusionCast significantly improves nowcasting performance.
Figures & tables
Figure 1: (a) Incorporates sparse GNSS PWV stations, historical radar QPE data, and future radar QPE data as model inputs. (b) Different data types are encoded separately through a multi-branch architecture. (c) Information from diverse data sources is fused to enhance forecasting performance. (d) The final decoder generates forecast outputs, providing projections of future precipitation.
Figure 2: (a) A spatial attention mechanism is applied to PWV features to emphasise critical spatial information. (b) The integration of GNSS and radar data through feature fusion via a gating mechanism is demonstrated. (c) Channel attention mechanisms extract key features from radar data to optimise information representation.
Figure 3: (a) The geographic location of the study area (red bounding box) within the Contiguous United States. (b) Zoomed-in view showing the spatial distribution of the GNSS stations (red triangles) used in this work.
Figure 4: Comparison: Interpolated Image vs. Ground Truth in the PWV Dataset at 2023-02-01 00:00:00 with 496 Valid Stations. (left) Interpolated Image, (right) Ground Truth.
Threshold
Model
CSI
T=10 min
T=40 min
T=80 min
T=120 min
τ=0.1
OpticalFlow
0.680
0.493
0.397
0.340
ConvLSTM
0.602
0.460
0.383
0.336
PredRNN
0.365
0.136
0.130
0.109
NowcastNet
0.478
0.477
0.477
0.477
FusionCast
0.784
0.661
0.614
0.534
Table 1: Performance comparison of CSI(↑ ) scores across different precipitation thresholds ( τ ) and forecasting horizons. The best results for each column are highlighted in bold .
Model
RMSE (mm/h)
MAE (mm/h)
OpticalFlow
1.598
0.214
ConvLSTM
1.181
0.356
PredRNN
1.835
0.743
NowcastNet
1.500
0.179
Ours
1.004
0.103
Table 2: Comparison of continuous metrics (RMSE, MAE) ( ↓ ) comparison of precipitation nowcasting performance across the test dataset. The best results are highlighted in bold .
Figure 5: Visualisation comparison of precipitation intensity forecast results for the Mississippi River basin over the next two hours, at 12:40 UTC on 22 February 2023. From top to bottom: Ground Truth (GT), FusionCast (Ours), NowcastNet, Optical Flow, ConvLSTM and PredRNN. The highlighted area indicates where our model demonstrates significant improvement over all other models. FusionCast preserves both the spatial structure and intensity of precipitation more consistently, especially at longer lead times.
Figure 6: Ablation study across different thresholds ( τ ) and lead times. The inferior performance of ”w/o PWV” and ”w/o Prior” variants validates that integrating multimodal priors is indispensable for accurate precipitation nowcasting.
Figure 7: The proposed Gating strategy effectively mitigates the feature blurring observed in the Concatenation baseline, leading to superior robustness in forecasting heavy precipitation ( τ=4.0 mm/h) over long lead times.
Precipitation nowcasting is critical for disaster mitigation and aviation safety. However, radar-only models frequently suffer from a lack of large-scale atmospheric context, leading to performance degradation at longer lead times. While integrating meteorological variables predicted by weather foundation models offers a potential remedy, existing architectures fail to reconcile the profound representational heterogeneities between radar imagery and meteorological data. To bridge this gap, we propose PW-FouCast, a novel frequency-domain fusion framework that leverages Pangu-Weather forecasts as spectral priors within a Fourier-based backbone. Our architecture introduces three key innovations: (i) Pangu-Weather-guided Frequency Modulation to align spectral magnitudes and phases with meteorological priors; (ii) Frequency Memory to correct phase discrepancies and preserve temporal evolution; and (iii) Inverted Frequency Attention to reconstruct high-frequency details typically lost in spectral filtering. Extensive experiments on the SEVIR and MeteoNet benchmarks demonstrate that PW-FouCast achieves state-of-the-art performance, effectively extending the reliable forecast horizon while maintaining structural fidelity. Our code is available at https://github.com/Onemissed/PW-FouCast.
Yuze Qin, Qingyong Li, Zhiqing Guo +3
Key Laboratory of Big Data & Artificial Intelligence in Transportation (Ministry of Education), Beijing Jiaotong University, Beijing, China · CMA Key Laboratory of Transportation Meteorology, Nanjing Innovation Institute for Atmospheric Sciences, Nanjing, China
Precipitation nowcasting is a vital spatio-temporal prediction task for meteorological applications but faces challenges due to the chaotic property of precipitation systems. Existing methods predominantly rely on single-source radar data to build either deterministic or probabilistic models for extrapolation. However, the single deterministic model suffers from blurring due to MSE convergence. The single probabilistic model, typically represented by diffusion models, can generate fine details but suffers from spurious artifacts that compromise accuracy and computational inefficiency. To address these challenges, this paper proposes a novel coarse-to-fine Vision Mamba Unet and residual Diffusion (VMU-Diff) based precipitation nowcasting framework. It realizes precipitation nowcasting through a two-stage process, i.e., a deterministic model-based coarse stage to predict global motion trends and a probabilistic model-based fine stage to generate fine prediction details. In the coarse prediction stage, rather than single-source radar data, both radar and multi-band satellite data are taken as input. A spatial-temporal attention block and several Vision mamba state-space blocks realize multi-source data fusion, and predict the future echo global dynamics. The fine-grained stage is realized by a spatio-temporal refine generator based on residual conditional diffusion models. It first obtains spatio-temporal residual features based on coarse prediction and ground truth, and further reconstructs the residual via conditional Mamba state-space module. Experiments on Jiangsu SWAN datasets demonstrate the improvements of our method over state-of-the-art methods, particularly in short-term forecasts.
Chunlei Shi, Hao Li, Yufeng Zhu +6
Department of Automation, Southeast University, Nanjing 210096, China
Accurate and timely rainfall nowcasting is crucial for disaster mitigation and water resource management. Despite recent advances in deep learning, precipitation prediction remains challenging due to limitations in effectively leveraging diverse multimedia data sources. We introduce M3R, a Meteorology-informed MultiModal attention-based architecture for direct Rainfall prediction that synergistically combines visual NEXRAD radar imagery with numerical Personal Weather Station (PWS) measurements, using a comprehensive pipeline for temporal alignment of heterogeneous meteorological data. With specialized multimodal attention mechanisms, M3R novelly leverages weather station time series as queries to selectively attend to spatial radar features, enabling focused extraction of precipitation signatures. Experimental results for three spatial areas of 100 km * 100 km centered at NEXRAD radar stations demonstrate that M3R outperforms existing approaches, achieving substantial improvements in accuracy, efficiency, and precipitation detection capabilities. Our work establishes new benchmarks for multimedia-based precipitation nowcasting and provides practical tools for operational weather prediction systems. The source code is available at https://github.com/Sanjeev97/M3Rain
Sanjeev Panta, Rhett M Morvant, Xu Yuan +2
University of Louisiana at Lafayette, Lafayette, LA, USA · University of Delaware, Newark, DE, USA