stat.APSep 24, 2026

Suitable Measures for the Potential Operational Utility of AI NWP Rainfall Forecasts Over Africa

Authors: Shruti Nath, Docko Sow, Koomi Toussaint Amoussouvi, Fenwick Cooper, Josiah Kiarie Kimani, John Bagiliko, Florian Pappenberger, Rendani Mbuvha

Organizations: Department of Physics, University of Oxford, United Kingdom · AfriClimate AI, South Africa · ECMWF, Reading, United Kingdom · School of Statistics and Actuarial Science, University of Witwatersrand, South Africa

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Artificial intelligence (AI)-based weather prediction is approaching the skill of physical numerical weather prediction (NWP) systems at a fraction of the computational cost. This is particularly promising for Africa, where rainfall extremes are intensifying and many forecasting centres lack the infrastructure to run physical models at extended lead times. We present a calibrated comparison of GraphCast, GenCast and the Functional Generative Network (FGN) against the physical NWP model IFS for rainfall prediction across Africa. Deterministic and probabilistic forecasts are postprocessed using Isotonic Distributional Regression and evaluated with the Continuous Ranked Probability Score against IMERG, RFEv2 and CHIRPS across seasons, wet and dry regimes, elevation zones and lead times. All models retain skill beyond climatology across most seasons and at extended lead times. AI models generally outperform IFS in wet regions, whereas IFS performs better in dry, high-elevation areas, where its finer resolution better represents orographic controls on rainfall. Across observational datasets and seasons, AI models achieve a median improvement of approximately 5% over IFS. GraphCast achieves calibrated skill comparable to the ensemble-based FGN, although FGN provides greater significant skill at longer lead times. These results highlight the potential of calibrated AI weather prediction to provide accessible and computationally efficient rainfall forecasts across Africa, while demonstrating the continuing importance of spatial resolution, ensemble design and regional characteristics.

Figures & tables

Model202020212022202320242025
IFS-ENS✓✓✓x✓✓
GraphCast✓✓✓✓✓✓
GenCast✓✓✓✓✓✓
FGNxx✓✓✓✓
Table 1: Years available for each model dataset, we only consider years that are within the AI models test set (2020 and above)
Figure 1: Areas receiving significant rainfall across seasons of February March April and May (FMAM, left); June, July, August and September (JJAS, middle); and October, November, December and January (ONDJ, right). For each season we consider the breakdown of months receiving significant rainfall, defined as on average 1 mm/day or more for a given month. This is calculated for satellite products of IMERG (top), RFEv2 (middle) and CHIRPS (bottom).
Figure 2: Elevation map over Africa binned according to the stratification of <500<500 m, 500500 – 10001000 m, 10001000 – 20002000 m, and >2000>2000 m used for verification in this study.
Figure 3: Annual difference in potential Continuous Ranked Probability Score (CRPS) with respect to climatology plotted against lead time and calculated for IMERG (top), RFEv2 (middle) and CHIRPS (bottom). We consider potential CRPS values as averaged across all grid-cells in the domain (All, left), areas only receiving significant rainfall for each month (Wet, middle) and vice versa (Dry, right). A more negative difference indicates lower CRPS with respect to climatology and better forecast resolution. Uncertainty bands are provided by bootstrap resampling of monthly values across all years considered for each model, therefore representing inter-annual and intra-seasonal uncertainty.
Figure 4: Same as Figure 3 , but showing seasonal performance when only considering areas receiving significant rainfall for each month instead. Seasons of February March April and May (FMAM, left); June, July, August and September (JJAS, middle); and October, November, December and January (ONDJ, right) are shown.
Figure 5: Same as Figure 3 , but showing elevation-wise performance when only considering areas receiving significant rainfall for each month instead. Elevation bins of << 500 m, 500 m-1000 m, 1000 m-2000 m and >> 2000 m are considered.
Figure 6: Same as Figure 5 , but now considering only dry areas.
Figure 7: Spatial maps of potential Continuous Ranked Probability Skill Score (CRPSS) for FGN and GraphCast with reference to IFS and evaluated against IMERG for February March April and May (FMAM). 1- (top), 5- (middle) and 9- (bottom) day lead times are shown. A more positive CRPSS indicates greater skill improvement over IFS and vice versa for negative. Dotted areas show where the null hypothesis from the Diebold-Mariano test is not rejected and the skill differential therefore could be due to random chance.
Figure 8: Same as Figure 7 but for June, July, August and September (JJAS).
Figure 9: Same as Figure 7 but for October, November, December and January.
Figure 10: Box-plots of the potential Continuous Ranked Probability Skill Score (CRPSS) for FGN and GraphCast with reference to IFS evaluated across 1-, 3-, 5-, 7- and 9- day lead times. Seasonal performance for February March April and May (FMAM, left); June, July, August and September (JJAS, middle); and October, November, December and January (ONDJ, right) when considering observations of IMERG (top), RFEv2 (middle) and CHIRPS (bottom) are shown. Only grid-cells with a significant loss differential as obtained by the Diebold-Mariano test are considered. Whiskers indicate 90 %\% spatial spread across grid cells.
Figure 11: Box-plots of p-values obtained from the Diebold-Mariano test applied to FGN and GraphCast with reference to IFS evaluated across 1-, 3-, 5-, 7- and 9- day lead times. Seasonal performance for February March April and May (FMAM, left); June, July, August and September (JJAS, middle); and October, November, December and January (ONDJ, right) when considering observations of IMERG (top), RFEv2 (middle) and CHIRPS (bottom) are shown. P-values lower than 0.1 (dotted horizontal line) indicates that the potential CRPS loss differential to IFS is significant. Whiskers indicate 90 %\% confidence intervals.
Fig. S1: CRPS vs lead time for raw (red) and Isotonic Distribution Regression based (black) outputs of GraphCast, GenCast, FGN and IFS. CRPS is averaged over all grid points, climatology is provided as the dashed line.
Fig. S2: Same as Figure S1 , but CRPS is averaged over only significantly wet areas for a given month, before then taking the annual average.
Fig. S3: CRPS maps of IFS calibrated with Isotonic Distributional Regression towards IMERG. Lead times of 1-, 5- and 9- day lead (columns) and seasons of February, March, April and May (FMAM), June, July, August and September (JJAS) and October, November, December and January (ONDJ) (rows) are considered.
Fig. S4: Same as Figure S3 but for RFEv2
Fig. S5: Same as Figure S3 but for CHIRPS

Explore similar work

CardsList
Sep 26, 2026

Global Evaluation of AI and NWP Precipitation Forecasts During Atmospheric River Events

Atmospheric rivers (ARs) produce many of the world's most extreme precipitation events and hydrometeorological hazards. Although artificial intelligence weather prediction (AIWP) models have demonstrated skill comparable to or exceeding numerical weather prediction (NWP) systems for large-scale atmospheric variables, their ability to forecast AR-related precipitation remains insufficiently characterized globally. Here, we evaluate 24-hour precipitation forecasts from the Global Forecast System (GFS), Global Ensemble Forecast System (GEFS), GraphCast, and Artificial Intelligence Forecasting System (AIFS) from Day 1 through Day 10 globally and across North America, Europe, and Australia and New Zealand. Using the Extreme Weather Bench framework, forecasts are evaluated against Integrated Multi-satellitE Retrievals for GPM (IMERG) observations using measures of precipitation magnitude, spatial structure, and localization. GraphCast and AIFS exhibit greater spatial skill than GFS and GEFS, particularly for heavy precipitation and at longer lead times, and better preserve the spatial organization of AR-related precipitation through Day 10. However, this improved spatial skill does not translate into accurate precipitation magnitudes. AIWP models tend to overpredict moderate-to-heavy accumulations while underpredicting the heaviest precipitation at longer lead times, whereas NWP systems develop pronounced dry biases. These results reveal distinct strengths and limitations of AIWP for high-impact precipitation forecasting and provide a reproducible benchmark.

Marina Vicens-Miquel, Taylor Mandelbaum, Amy McGovern +2

School of Meteorology and School of Computer Science, University of Oklahoma, Norman, 73072, OK, USA. · NSF AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography, Norman, 73072, OK, USA. · Brightband, San Francisco, CA, USA.

Precipitation ForecastingWeather ForecastingForecasting Benchmarks
AskPDF
Sep 2, 2026

Improving global precipitation forecasts with an AI weather model trained on satellite observations

Precipitation forecasts shape decision-making across the global economy, particularly in sectors such as agriculture. However, unlike variables such as temperature, precipitation is highly intermittent and localized, making it difficult to forecast. While recent advances in AI weather prediction systems have enabled them to surpass physical models on globally averaged metrics, improvements in mean error rarely translate to actionable forecasts of severe flooding or dry crop fields. Furthermore, most of these models are trained and evaluated against a reanalysis data product, ERA5, which has well-known biases. Here we retrain AIFS, ECMWF's widely-used, open-source operational 0.25° probabilistic graph-transformer weather model, on satellite-based precipitation observations. Our model, Laxmi, improves global probabilistic accuracy by 19% and resolves systematic distributional biases in ERA5. Specifically, Laxmi reduces drizzle overprediction by 33% for amounts less than 3 mm per day. It also mitigates extreme rainfall underprediction, improving the global 95th percentile Brier skill score by 57%. Across a case study of 10 Indian tropical storms, Laxmi delivered the most accurate forecast of 150 mm event-total precipitation in 7 events, compared to 1 for AIFS and 2 for the leading physical model, IFS. Our results demonstrate that incorporating observation-based precipitation data directly into training can substantially improve forecasts.

Julian F. Schmitt, Bertrand Delorme, Robert C. King +4

X, The Moonshot Factory, Google LLC, Mountain View, CA, USA · Google, Google LLC, Mountain View, CA, USA · Environmental Science and Engineering, California Institute of Technology, Pasadena, CA, USA +1

Precipitation ForecastingProbabilistic ForecastingWeather Forecasting
AskPDF
Mar 13, 2026

MR-GNF: Multi-Resolution Graph Neural Forecasting on Ellipsoidal Meshes for Efficient Regional Weather Prediction

Weather forecasting offers an ideal testbed for artificial intelligence (AI) to learn complex, multi-scale physical systems. Traditional numerical weather prediction remains computationally costly for frequent regional updates, as high-resolution nests require intensive boundary coupling. We introduce Multi-Resolution Graph Neural Forecasting (MR-GNF), a lightweight, physics-aware model that performs short-term regional forecasts directly on an ellipsoidal, multi-scale graph of the Earth. The framework couples a 0.25° region of interest with a 0.5° context belt and 1.0° outer domain, enabling continuous cross-scale message passing without explicit nested boundaries. Its axial graph-attention network alternates vertical self-attention across pressure levels with horizontal graph attention across surface nodes, capturing implicit 3-D structure in just 1.6 M parameters. Trained on 40 years of ERA5 reanalysis (1980-2024), MR-GNF delivers stable +6 h to +24 h forecasts for near-surface temperature, wind, and precipitation over the UK-Ireland sector. Despite a total compute cost below 80 GPU-hours on a single RTX 6000 Ada, the model matches or exceeds heavier regional AI systems while preserving physical consistency across scales. These results demonstrate that graph-based neural operators can achieve trustworthy, high-resolution weather prediction at a fraction of NWP cost, opening a practical path toward AI-driven early-warning and renewable-energy forecasting systems. Project page and code: https://github.com/AndriiShchur/MR-GNF

Andrii Shchur, Inna Skarga-Bandurova

Kyiv School of Economics · Oxford Brookes University

Weather ForecastingGraph Neural NetworksSpatiotemporal GNNs
AskPDF