RainCheck: Harnessing Historical Weather Data with Machine Learning for Rainfall Forecasting in Bangladesh

Shahran Rahman Alve; Muhammad Zawad Mahmud; Samiha Islam; Al Jubayer Pial

cs.LGOct 28, 2026

RainCheck: Harnessing Historical Weather Data with Machine Learning for Rainfall Forecasting in Bangladesh

Authors: Shahran Rahman Alve, Muhammad Zawad Mahmud, Samiha Islam, Al Jubayer Pial

Organizations: Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka-1229, Bangladesh.

Abstract

Machine-learning rainfall forecasts are often evaluated by shuffling station records into random training and test sets. For rainfall, this lets a model see what happened at neighbouring stations on the very day it is asked to forecast, and it hides the fact that a seasonal lookup table already explains most of the variance. We quantify both effects on 618,650 station-days from 35 Bangladesh Meteorological Department stations (1948-2014). We compare random, temporal (train <=1999, test 2005-2014) and spatio-temporal (unseen stations and unseen years) evaluation, calendar-only versus antecedent-rainfall features, and six learners against climatology, persistence and SARIMA. Three findings stand out. First, a station-month climatology alone classifies high-rainfall months with 88.4% accuracy (AUC 0.950), and no learner improved on it by more than 0.8 points under temporal evaluation. Second, a calendar-only random forest reached a next-day wet-day AUC of 0.889 under a random split, higher than any model with real predictors under temporal evaluation, yet fell to 0.835, below climatology (0.859), on future years. Third, models with lagged rainfall add genuine but modest skill for next-day forecasts: a Brier skill score of 0.074 (95% CI 0.064-0.088) over climatology, rising to 0.091 when station and year inputs are removed and to 0.131 when other stations' same-day observations are added. A GRU sequence model matches but does not exceed gradient boosting, and skill disappears beyond a two-day lead. For monthly totals, no learner beat climatology significantly. We release a leakage-aware protocol and code so that future station-based studies report skill against climatology on held-out years.

Figures & tables

Panel of Figure 1 — Figure 1: Workflow of the study. Raw BMD records are cleaned so that missing days remain missing, then turned into daily and monthly forecasting tasks. Each model is trained with calendar, antecedent or network inputs and evaluated under three protocols, always against reference forecasts and with skill scores relative to climatology. The dashed red path marks the combination that inflates reported accuracy: calendar inputs with a random split, which lets a model recover the target day’s rainfall from neighbouring stations.

Figure 2: Data. Left: mean monthly total across stations (bars) with 10th–90th percentile range; over 70% of annual rainfall falls in June–September. Right: number of stations with a valid monthly total per year, with the training (1950–1999) and test (2005–2013) periods shaded.

Model	Setting
Logistic / ridge regression	median imputation, standardisation; L2 penalty, $C=1$ / $\alpha=1$
$k$ -nearest neighbours	$k=50$ , standardised inputs, fitted on 100,000 random training rows
Decision tree	maximum depth 12, at least 20 samples per leaf
Random forest	daily: 100 trees, 20% rows per tree, 50% features per split; monthly: 300 trees; at least 5 samples per leaf
Histogram gradient boosting	300 iterations, learning rate 0.05; early stopping on 10% of training rows in the main table, none in Section 6.6
MLP	two hidden layers of 64 ReLU units, Adam, early stopping, at most 50 epochs

Table 1: Hyperparameters (fixed in advance).

		Random split			Temporal split
Model	Feat.	Acc.	AUC	F1	Acc.	AUC	F1
Climatology	–	87.8	0.946	84.6	88.4	0.950	84.9
Logistic	Cal	58.9	0.609	0.0	49.1	0.607	0.1
Random forest	Cal	88.2	0.950	84.9	87.2	0.942	85.0
Grad. boosting	Cal	89.3	0.958	86.5	87.6	0.951	85.5
Logistic	Ant	88.0	0.950	84.5	88.8	0.956	85.6

Table 2: High- vs. low-rainfall month classification (threshold 203 mm). Random-split values are means over three seeds (s.d.

\leq

0.8 points). Acc. and F1 in %. Features: Cal = calendar only, Ant = antecedent rainfall.

Temporal split
Model	Feat.	MAE	RMSE	$R^{2}$
Climatology	–	80.3	129.1	0.718
Persistence	–	132.7	212.7	0.236
Seasonal naive	–	108.6	182.4	0.436
SARIMA †	–	76.4	132.2	0.708
Ridge	Cal	186.3	241.2	0.016

Table 3: One-month-ahead station totals (mm). Temporal split: train 1950–1999, test 2005–2013 (3,743 station-months). † SARIMA covers 32 stations (3,419 station-months); on that subset climatology has MAE 79.2 and RMSE 126.7. Random split: mean over three seeds, with climatology refitted on the same random training rows.

Split	Features	AUC	MAE	$R^{2}$
Random	Cal	0.867	7.86	0.164
Random	Cal + network	0.916	6.33	0.343
Temporal	Cal	0.849	8.34	0.081
Temporal	Cal + network	0.921	6.13	0.366

Table 4: Leakage probe (next day, gradient boosting). “+ network adds the oracle mean target-day rainfall at all other stations, information a random split exposes implicitly.

Figure 3: Temporal evaluation, gradient boosting with antecedent features. Left: next-day occurrence AUC by season (winter, pre-monsoon, monsoon, post-monsoon), compared with climatology. Right: reliability diagram; points are 10 probability bins with at least 100 cases.

	Occurrence: AUC						Amount: $R^{2}$
	Rand.		Temp.		Sp.-temp.		Rand.		Temp.		Sp.-temp.
Model	Cal	Ant	Cal	Ant	Cal	Ant	Cal	Ant	Cal	Ant	Cal	Ant
Climatology	–	0.847	–	0.859	–	0.854	–	0.128	–	0.140	–	0.097
Persistence	–	0.758	–	0.763	–	0.763	–	-0.255	–	-0.205	–	-0.244
Logistic / ridge	0.562	0.868	0.580	0.876	0.578	0.873	0.003	0.201	0.003	0.219	-0.005	0.184
$k$ -NN	0.843	0.865	0.843	0.870	0.842	0.868	0.104	0.191	0.054	0.208	0.024	0.170

Table 5: Next-day forecasts. ROC AUC for occurrence (

\geq

1 mm) and

R^{2}

for amount, under random, temporal and spatio-temporal evaluation, with calendar-only (Cal) or antecedent-rainfall (Ant) features. Random split: mean of three seeds (s.d.

\leq

0.002 AUC). Spatio-temporal: five held-out station groups, weighted by test size. Reference forecasts use no learned features; on held-out stations climatology is the network-wide average. Bold: best per column. Underlined: calendar-only models whose random-split score exceeds the best temporal score with real predictors.

Figure 4: Next-day forecasts under three evaluation protocols. Hatched bars use calendar features only; solid bars add antecedent rainfall. The dashed line is climatology under temporal evaluation. Calendar-only tree models look strong under the random split and fall below climatology on future years and new stations.

Figure 5: Further analyses (temporal evaluation, next-day occurrence unless stated). (a) Brier skill score against lead time. (b) Skill against the length of the training record, with test years fixed at 2005–2014 and the full-record climatology as reference. (c) Skill after removing one feature group from the gradient-boosting model; the dotted line is the full model. (d) Skill at each of the 35 stations, with and without network observations.

Model	AUC	BSS	MSESS
Climatology	0.859	0	0
Logistic / ridge (Ant)	0.876	0.063	0.092
GB (Ant, all inputs)	0.880	0.074	0.077
GB (Ant, no st./yr)	0.883	0.091	0.104
GRU (30-day seq.)	0.884 $\pm$ .000	0.092 $\pm$ .001	0.095 $\pm$ .003
GB (no st./yr) + network	0.896	0.131	0.140

Table 6: Next-day skill on future years (temporal split, test 2005–2014). Skill scores are relative to climatology. GB = gradient boosting. “No st./yr” drops the station code and year inputs. “+ network” adds rainfall observed on day

t

at the other stations, which is available when the forecast is issued. GRU: mean

\pm

s.d. over three seeds; GB (no st./yr): s.d. over five bootstrap refits

\leq

0.001.

Year	2005	2006	2007	2008	2009	2010	2011	2012	2013	2014
Wet-day rate	0.32	0.30	0.33	0.33	0.29	0.31	0.32	0.32	0.32	0.20
BSS	0.112	0.054	0.081	0.056	0.054	0.060	0.082	0.068	0.091	0.092
BSS + network	0.144	0.088	0.148	0.141	0.132	0.098	0.129	0.157	0.139	0.121

Table 7: Next-day occurrence skill by test year (temporal split, gradient boosting with all antecedent inputs, with and without network observations).

Precipitation Forecasting

Precipitation forecasting predicts where, when, and how much rain or snow will fall, from local radar nowcasts to global multi-day outlooks.

See all 56 papers

Can Machine Learning Forecast Rice Yields in Data-Constrained Settings? Satellite Climate Data, National Crop Statistics, and Lessons from Sierra Leone
Jun 11, 2026Ibrahim Denis FofanahTime Series Forecasting Spatiotemporal Forecasting
Seidenberg School of Computer Science & Information Systems, Pace University, New York, USA · RiseAfrica Foundation for STEM and Innovation, Sierra Leone, West Africa
Improvements to the post-processing of weather forecasts using machine learning and feature selection
Apr 21, 2026Kazuma Iwase, Tomoyuki TakenawaPrecipitation Forecasting Gradient-Boosted Decision Trees
Graduate School of Marine Science and Technology, Tokyo University of Marine Science and Technology, 2-1-6 Etchujima, Koto-ku, Tokyo 135-8533, Japan
MZ-Rain: Moisture-Budget-Guided Zero-Inflated Model for Station-Level Precipitation Nowcasting
Sep 4, 2026Yifang Zhang, Shengwu Xiong, Henan Wang +6Precipitation Nowcasting Physics-Informed ML
Sanya Science and Education Innovation Park, Wuhan University of Technology, Sanya, 572000, China · School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, China · School of Earth and Space Science and Technology, Wuhan University, Wuhan 430072, China +2

Jun 11, 2026J/K move · Enter open · S save

Can Machine Learning Forecast Rice Yields in Data-Constrained Settings? Satellite Climate Data, National Crop Statistics, and Lessons from Sierra Leone

Ibrahim Denis Fofanah

Seidenberg School of Computer Science & Information Systems, Pace University, New York, USA · RiseAfrica Foundation for STEM and Innovation, Sierra Leone, West Africa

Time Series Forecasting Spatiotemporal Forecasting Crop Yield Prediction

Open paper AskPDF arXiv

Sierra Leone's agriculture operates with almost no data-driven decision support, and no published machine learning study has examined the country's crop yields. We ask whether rice yield can be forecast from data Sierra Leone currently has. Using 25 years of FAOSTAT production data (2000-2024) for nine major crops, we train XGBoost, Gradient Boosting, and Random Forest under a strict anti-leakage protocol with expanding-window walk-forward evaluation across seven held-out years, benchmarked against naive persistence. No model trained on crop statistics alone outperforms persistence. Augmenting with free satellite climate data (CHIRPS rainfall, NASA POWER temperature) reverses this result: a climate-only XGBoost reduces forecast error by one third (RMSE 284 vs 428 kg/ha), a gain that holds for a linear model and is robust to excluding the anomalous 2018 season. Early-season (May-June) rainfall is the dominant predictor, implying seasonal yield risk is observable months before harvest. No model anticipated the 2018 collapse, whose origins were institutional rather than climatic. We translate the findings into policy recommendations for Sierra Leone's Feed Salone Strategy, with a fully open-source pipeline.

Abstract

Precipitation Forecasting

Explore similar work

Can Machine Learning Forecast Rice Yields in Data-Constrained Settings? Satellite Climate Data, National Crop Statistics, and Lessons from Sierra Leone

Improvements to the post-processing of weather forecasts using machine learning and feature selection

MZ-Rain: Moisture-Budget-Guided Zero-Inflated Model for Station-Level Precipitation Nowcasting