cs.LGMar 9, 2026
SaveDesigning probabilistic AI monsoon forecasts to inform agricultural decision-making
Organizations: Development Innovation Lab, University of Chicago, IL, 60637 · Department of the Geophysical Sciences, University of Chicago, IL, 60637 · Data Science Institute, University of Chicago, IL, 60637 · Development Innovation Lab India, University of Chicago Trust, India, 560025 ·
Abstract
Hundreds of millions of farmers make high-stakes decisions under uncertainty about future weather. Forecasts can inform these decisions, but available choices and their risks and benefits vary between farmers. We introduce a decision-theory framework for designing useful forecasts in settings where the forecaster cannot prescribe optimal actions because farmers' circumstances are heterogeneous. We apply this framework to the case of seasonal onset of monsoon rains, a key date for planting decisions and agricultural investments in many tropical countries. We develop a system for tailoring forecasts to the requirements of this framework by blending systematically benchmarked artificial intelligence (AI) weather prediction models with a new "evolving farmer expectations" statistical model. This statistical model applies Bayesian inference to historical observations to predict time-varying probabilities of first-occurrence events throughout a season. The blended system yields more skillful Indian monsoon forecasts at longer lead times than its components or any multi-model average. In 2025, this system was deployed operationally in a government-led program that delivered subseasonal monsoon onset forecasts to 38 million Indian farmers, skillfully predicting that year's early-summer anomalous dry period. This decision-theory framework and blending system offer a pathway for developing climate adaptation tools for large vulnerable populations around the world.
Figures & tables
| Model Result | Implication for Forecast Design and Evaluation |
| Probabilistic forecasts better allow heterogeneous farmers to make their own decisions based on information relevant to farming that the forecaster does not have (Proposition 1 ). | • Provided probabilistic forecasts. • Designed messages in part based on focus groups with farmers aimed at increasing comprehension of probabilistic information. |
| If some farmers take forecasts at face value, guarantees on benefits of forecasts require forecasts to be well-calibrated and incorporate farmers’ other sources of information about weather outcomes (Proposition 2 ). | • Designed a new “evolving expectations" statistical model as both a baseline and a component of the blended model. • Used the blended model, which met the criteria for dissemination, rather than AIWP model outputs on their own, which did not. • Tested calibration of blended model across multiple time periods. |
| Experiments measuring yield and profit outcomes do not capture the value of forecasts if farmers are risk-averse. Whether forecasts have value to farmers can be measured by observing whether farmers change decisions in response to forecasts (Propositions 3 - 6 ). | • Identified monsoon onset as valuable target based on previous research finding that farmers change decisions after receiving monsoon onset forecasts [ 10 ] . • Chose an agriculturally-relevant monsoon onset definition, capturing the start of continuous local rainfall not followed by a potentially crop-damaging dry spell, to specifically target farmer decisions. |
Table 1: Implications of decision-theory framework and operational decisions informed by the results, and actions we took as a result.
Figure 1: Static climatology and the evolving-expectations model. A) Median onset date for the grid cells used in dissemination. B) The distribution of possible onset dates predicted by the evolving-expectations model for a grid cell centered at latitude 26 ∘ N and longitude 78 ∘ E. Shaded distribution shows the unconditional probability of onset; lines show the probability from the evolving expectations model if the onset has not occurred by the date signified by the dot. Illustrative probabilities are calculated over the shaded grey region. As the season progresses without an onset, the probability of onset in any particular future week increases. C) The distribution of possible onset dates predicted by the evolving-expectations model for a grid cell centered at 18 ∘ N and 80 ∘ E. The decision-theory framework implies that models that cannot outperform this baseline should not be used on their own for dissemination.
Figure 2: Evaluation of temporal components of models’ skill for the 2000-2024 period. A) Area under ROC curve (AUC) by lead time. The baseline for the bars is chosen to be 0.5, indicating the AUC of a forecast with no ability to distinguish onsets from non-onsets. B) Brier skill score by lead time, computed relative to a traditional (static) climatology model. C) AUC by year across all lead times. The 2000-2024 scores are computed via cross-validation. The 2025 scores are only for forecasts that were actually disseminated before Moron-Robertson onset in each grid cell. Dissemination began in late May, so the set of initialization dates is smaller than in other years. D) Brier skill score by year across all lead times, calculated relative to a traditional (static) climatology model. A deterministic version of AIFS was used in both the blended model and benchmarking, so NGCM is shown here as a representative AIWP model as it performs better on these probabilistic metrics (Fig. 4 ). See Extended Data Fig. 2 for results for the 1965-1978 period.
Figure 3: Reliability diagram and histogram of probabilities assigned by different models during the 2000-2024 cross-validation period. If the model is well-calibrated the points should line up with the dashed line. Each point represents a decile of probabilities assigned by the model, and compares the average predicted probability in that decile and the fraction of events which occurred in observation. Histograms are normalized so that the bin capturing probabilities between 0 and 10% has height equal to one. A deterministic version of AIFS was used for both benchmarking and the blended model, so NGCM is used as a representative AIWP model for this figure. See Extended Data Fig. 3 for the results from the 1965-1978 period.
Figure 4: Performance of models during the primary 2000-2024 period, including submodels of the blended model. Vertical lines represent the evolving expectations model, which the decision-theory model implies should be a baseline for dissemination. The interaction models, indicated with colons, are submodels of the final blended model combining the evolving expectations model (EE) with 5-day rainfall variables from one or more AIWP models but not 10-day rainfall variables. The best multimodel ensemble (MME) is selected as a linear combination of probabilities from AIFS, NGCM, and the evolving expectations model maximizing the RPSS post-hoc on the 2000-2024 period. All other models (including calibration) are trained via leave-one-year-out cross-validation. Skill scores and differences in AUC are computed relative to unconditional climatology, which has an AUC of .810, a Brier Score of 0.583, and a RPS of 0.611. See Extended Data Fig. 4 for the results from the 1965-1978 period.
Figure 1: Skill scores aggregated by test period. The blended model is cross-validated during the 2000-2024 period, and trained on 2000-2024 data during the other periods. The climatology model and the evolving expectations model are cross-validated by year using 1900-2024 IMD data. Skill scores are all computed relative to static climatology.
Figure 2: Temporal components of model skill, 1965-1978. A) Area under ROC curve (AUC) by lead time during the 1965-1978 pre-satellite-era period. The baseline for the bars is chosen to be 0.5, indicating the AUC of a forecast with no ability to distinguish onsets from non-onsets. B) Brier skill score by lead time, computed relative to a traditional (static) climatology model. C) Area under ROC curve by year across all lead times. The 2000-2024 scores are computed via cross-validation. D) Brier skill score by year across all lead times, calculated relative to a traditional (static) climatology model.
Figure 3: Reliability diagram and histogram of probabilities assigned by different models during the 1965-1978 pre-satellite-era period. If the model is well-calibrated the points should line up with the dashed line. Each point represents a decile of probabilities assigned by the model, and compares the average predicted probability in that decile and the fraction of events which occurred in observation. Histograms are normalized so that the bin capturing probabilities between 0 and 10% has height equal to one.
Figure 4: Performance of models during the 1965-1978 pre-satellite-era period, including submodels of the blended model. Vertical lines represent the evolving-expectations (EE) model, which our decision-theory model implies should be a baseline for dissemination. The interaction models, indicated with colons, are submodels of the final blended model using 5-day rainfall variables from one or more AIWP models but not 10-day rainfall variables. The best multimodel ensemble is selected as a linear combination of probabilities from AIFS, NGCM, and the evolving expectations model maximizing the RPSS post-hoc on the 2000-2024 period. All other models (including calibration) are trained via leave-one-year-out cross-validation. Skill scores and differences in AUC are computed relative to unconditional climatology, which has an AUC of 0.822, a Brier Score of 0.567, and a Ranked Probability Score of 0.594.
Figure 5: Skills of the evolving expectations model and the blended model using different training periods for the evolving expectations model. Skill scores are computed relative to static climatology trained on all years 1900-2024. Bars indicate scores on the 2000-2024 period, and circles represent scores in the 1965-1978 period.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Supplementary Figure S1: An illustration of the decision-theory model’s implication that probabilistic forecasts provide more value than deterministic forecasts.
Supplementary Figure S1: Map of Brier skill scores of blended model relative to the evolving expectations model by grid cell during the 2000-2024 cross-validation period. Grid cells used for model training and intended for dissemination are depicted with thick outlines.
Supplementary Figure S2: Map of RPS skill scores of blended model relative to the evolving expectations model by grid cell during the 2000-2024 cross-validation period. Grid cells used for model training and intended for dissemination are depicted with thick outlines.
Supplementary Figure S3: Map of differences between the AUC of the blended model and that of the evolving expectations model by grid cell during the 2000-2024 cross-validation period. Grid cells used for model training and intended for dissemination are depicted with thick outlines.
Supplementary Figure S4: Reliability diagram and histogram of probabilities assigned by different models during the 2000-2024 cross-validation period, removing the MOK filter from the onset definition. If the model is well-calibrated the points should line up with the dashed line. Each point represents a decile of probabilities assigned by the model, and compares the average predicted probability in that decile and the fraction of events which occurred in observation. Histograms are normalized so that the bin capturing probabilities between 0 and 10% has height equal to one.
Supplementary Figure S5: Reliability diagram and histogram of probabilities assigned by different models during the 1965-1978 pre-satellite-era period, removing the MOK filter from the onset definition. If the model is well-calibrated the points should line up with the dashed line. Each point represents a decile of probabilities assigned by the model, and compares the average predicted probability in that decile and the fraction of events which occurred in observation. Histograms are normalized so that the bin capturing probabilities between 0 and 10% has height equal to one.
Supplementary Figure S6: Reliability diagram and histogram of probabilities assigned by different models during the 2000-2024 cross-validation period, using a climatological MOK date as part of the onset definition. If the model is well-calibrated the points should line up with the dashed line. Each point represents a decile of probabilities assigned by the model, and compares the average predicted probability in that decile and the fraction of events which occurred in observation. Histograms are normalized so that the bin capturing probabilities between 0 and 10% has height equal to one.
Supplementary Figure S7: Reliability diagram and histogram of probabilities assigned by different models during the 1965-1978 pre-satellite-era period, using a climatological MOK date as part of the onset definition. If the model is well-calibrated the points should line up with the dashed line. Each point represents a decile of probabilities assigned by the model, and compares the average predicted probability in that decile and the fraction of events which occurred in observation. Histograms are normalized so that the bin capturing probabilities between 0 and 10% has height equal to one.
Supplementary Figure S8: Performance of models during the 2000-2024 cross-validation period, removing the MOK filter from the onset definition. Vertical lines represent the evolving expectations model, which our decision-theory model implies should be a baseline for dissemination. The interaction models, indicated with colons, are submodels of the final blended model using 5-day rainfall variables from one or more AIWP models but not 10-day rainfall variables. The best multimodel ensemble is selected as a linear combination of probabilities from AIFS, NGCM, and the evolving expectations model maximizing the Ranked Probability Skill Score post-hoc on the 2000-2024 period. All other models (including calibration) are trained via leave-one-year-out cross-validation. Skill scores and differences in AUC are computed relative to unconditional climatology, which has an AUC of .790, a Brier Score of .612, and a Ranked Probability Score of .682.
Supplementary Figure S9: Performance of models during the 1965-1978 pre-satellite-era period, removing the MOK filter from the onset definition. Vertical lines represent the evolving expectations model, which our decision-theory model implies should be a baseline for dissemination. The interaction models, indicated with colons, are submodels of the final blended model using 5-day rainfall variables from one or more AIWP models but not 10-day rainfall variables. The best multimodel ensemble is selected as a linear combination of probabilities from AIFS, NGCM, and the evolving expectations model maximizing the Ranked Probability Skill Score post-hoc on the 2000-2024 period. All other models (including calibration) are trained via leave-one-year-out cross-validation. Skill scores and differences in AUC are computed relative to unconditional climatology, which has an AUC of .807, a Brier Score of .582, and a Ranked Probability Score of .646.
Supplementary Figure S10: Performance of models during the 2000-2024 cross-validation period, using a climatological MOK date as part of the onset definition. Vertical lines represent the evolving expectations model, which our decision-theory model implies should be a baseline for dissemination. The interaction models, indicated with colons, are submodels of the final blended model using 5-day rainfall variables from one or more AIWP models but not 10-day rainfall variables. The best multimodel ensemble is selected as a linear combination of probabilities from AIFS, NGCM, and the evolving expectations model maximizing the Ranked Probability Skill Score post-hoc on the 2000-2024 period. All other models (including calibration) are trained via leave-one-year-out cross-validation. Skill scores and differences in AUC are computed relative to unconditional climatology, which has an AUC of .821, a Brier Score of .574, and a Ranked Probability Score of .595.
Supplementary Figure S11: Performance of models during the 1965-1978 pre-satellite-era period, using a climatological MOK date as part of the onset definition. Vertical lines represent the evolving expectations model, which our decision-theory model implies should be a baseline for dissemination. The interaction models, indicated with colons, are submodels of the final blended model using 5-day rainfall variables from one or more AIWP models but not 10-day rainfall variables. The best multimodel ensemble is selected as a linear combination of probabilities from AIFS, NGCM, and the evolving expectations model maximizing the Ranked Probability Skill Score post-hoc on the 2000-2024 period. All other models (including calibration) are trained via leave-one-year-out cross-validation. Skill scores and differences in AUC are computed relative to unconditional climatology, which has an AUC of .827, a Brier Score of .561, and a Ranked Probability Score of .590.
Supplementary Figure S12: Temporal components of model skill, using a climatological MOK date as part of the onset definition. (A) Area under ROC curve (AUC) by lead time during the 2000-2024 cross-validation period. The baseline for the bars is chosen to be 0.5, indicating the AUC of a forecast with no ability to distinguish onsets from non-onsets. (B) Brier skill score by lead time, computed relative to a traditional (static) climatology model. (C) Area under ROC curve by year across all lead times. The 2000-2024 scores are computed via cross-validation. The 2025 scores are only for forecasts that were actually disseminated before MR onset in each grid cell. Dissemination began in late May, so the set of initialization dates is smaller than in other years. (D) Brier skill score by year across all lead times, calculated relative to a traditional (static) climatology model.
Supplementary Figure S13: Temporal components of model skill, removing the MOK filter from the onset definition. (A) Area under ROC curve (AUC) by lead time during the 2000-2024 cross-validation period. The baseline for the bars is chosen to be 0.5, indicating the AUC of a forecast with no ability to distinguish onsets from non-onsets. (B) Brier skill score by lead time, computed relative to a traditional (static) climatology model. (C) Area under ROC curve by year across all lead times. The 2000-2024 scores are computed via cross-validation. The 2025 scores are only for forecasts that were actually disseminated before MR onset in each grid cell. Dissemination began in late May, so the set of initialization dates is smaller than in other years. (D) Brier skill score by year across all lead times, calculated relative to a traditional (static) climatology model.
Supplementary Figure S14: Temporal components of model skill, using a climatological MOK date as part of the onset definition, 1965-1978. (A) Area under ROC curve (AUC) by lead time during the 1965-1978 pre-satellite-era period. The baseline for the bars is chosen to be 0.5, indicating the AUC of a forecast with no ability to distinguish onsets from non-onsets. (B) Brier skill score by lead time, computed relative to a traditional (static) climatology model. (C) Area under ROC curve by year across all lead times. (D) Brier skill score by year across all lead times, calculated relative to a traditional (static) climatology model.
Supplementary Figure S15: Temporal components of model skill, removing the MOK filter from the onset definition, 1965-1978. (A) Area under ROC curve (AUC) by lead time during the 1965-1978 pre-satellite-era period. The baseline for the bars is chosen to be 0.5, indicating the AUC of a forecast with no ability to distinguish onsets from non-onsets. (B) Brier skill score by lead time, computed relative to a traditional (static) climatology model. (C) Area under ROC curve by year across all lead times. (D) Brier skill score by year across all lead times, calculated relative to a traditional (static) climatology model.
Supplementary Figure S16: Skill scores aggregated by test period, using a climatological MOK date as part of the onset definition. The blended model is cross-validated during the 2000-2024 period, and trained on 2000-2024 data during the other periods. The climatology model and the evolving expectations model are cross-validated by year using 1900-2024 IMD data. Skill scores are all computed relative to static climatology.
Supplementary Figure S17: Skill scores aggregated by test period, removing the MOK filter from the onset definition. The blended model is cross-validated during the 2000-2024 period, and trained on 2000-2024 data during the other periods. The climatology model and the evolving expectations model are cross-validated by year using 1900-2024 IMD data. Skill scores are all computed relative to static climatology.
More in this topic
Weather Forecasting
Weather forecasting predicts future atmospheric conditions, from local storms to global patterns, using physical models, observations, and machine learning.
See all 176 papers