Ranked Right, Sized Wrong: Ensemble Spread Fails Silently at Streamflow Extremes

Justin Gallant; Tyler Wilson

stat.MLOct 14, 2026

Ranked Right, Sized Wrong: Ensemble Spread Fails Silently at Streamflow Extremes

Authors: Justin Gallant, Tyler Wilson

Organizations: University of British Columbia, Vancouver, BC, Canada

Abstract

Deep learning models, particularly Long Short-Term Memory networks, are increasingly used for operational streamflow forecasting, with ensemble spread providing a signal of predictive uncertainty. As climate change increases the frequency of hydroclimatic extremes, these models will increasingly operate under flow conditions that are rare in their historical records. We therefore evaluate whether ensemble spread remains informative under increasing flows, examining both its magnitude relative to error and its ability to rank errors. Prediction error is approximately $2\times$ the corrected spread at ordinary flows and up to $4\times$ at the highest flows, with the discrepancy increasing with flow (log-log slope $1.30$ , 95% CI: $[1.20,1.41]$ ). Under-dispersion is strongest in observed extreme flows beyond the training record, reaching an $18\times$ discrepancy. This is pervasive and not explainable by conditional mean bias. Area Under the Sparsification Error, a standard distribution-free metric of uncertainty ranking, describing how well a model knows when it is uncertain, indicates near-oracle ranking for ensemble spread ( $0.071$ ), yet ordering by predicted discharge alone nearly matches it. Informative uncertainty ranking can coexist with severe under-dispersion. In deployment, error ranking cannot be verified, yet the spread is available, and it is relied on as the uncertainty signal under increasingly extreme conditions, while systematically understating risk, most severely for the extreme events that matter most.

Figures & tables

Figure 1: UDF by decile of the conditioning variable. (a) Deciles of observed discharge (

y

). (b) Deciles of the ensemble mean discharge (

\hat{\mu}

), with the fitted log-log scaling relationship and 95% confidence interval. Error bars denote 95% bootstrap intervals clustered by catchment.

Figure 2: (a) Sparsification curves under ensemble spread and

\hat{\mu}

for the full test set and top observed-flow decile, compared with oracle and random ordering. (b) AUSE versus UDF across catchments.

	$>$ p90	$>$ p95	$>$ p99	$>$ p99.5	$>$ p99.9	$>$ max
UDF	$5.65$	$6.79$	$10.40$	$11.99$	$14.41$	$17.93$
observed / expected	$1.15$	$1.26$	$1.51$	$1.64$	$2.30$	N/A
catchment-days ( $n$ )	$100{,}974$	$55{,}212$	$13{,}295$	$7{,}185$	$2{,}023$	$603$

Table 1: UDF against each catchment’s fixed 1980–2005 training flow distribution.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Handling	Variable name	Units
Dynamic input	Maximum temperature	∘ C day -1
Dynamic input	Minimum temperature	∘ C day -1
Dynamic input	Total precipitation	mm day -1
Static input	Catchment area	km 2
Static input	Mean elevation	m above sea level
Static input	Elevation range	m

Appendix

Table 2: LSTM data used

Subset	Forcing data	Streamflow data	Number of observations
Training	1979/01/01–2005/12/30	1980/01/01–2005/12/31	2,278,178
Validation	2005/01/01–2012/12/30	2006/01/01–2012/12/31	607,967
Testing	2012/01/01–2022/12/30	2013/01/01–2022/12/31	877,729

Appendix

Table 3: Partitions of forcing and streamflow datasets. The forcing data subsets begin exactly one year prior to the streamflow subsets to provide the 365-day meteorological context window required for the first target prediction.

decile	$n$	UDF	mean $r$	$n$	UDF	mean $r$
	conditioned on $y$			conditioned on $\hat{\mu}$
0–10	82,317	2.00	$-0.096$	87,680	2.10	$-0.010$
10–20	88,420	1.97	$-0.086$	87,735	2.85	$-0.018$
20–30	90,156	2.16	$-0.101$	87,750	2.53	$-0.023$
30–40	96,425	2.16	$-0.117$	87,735	2.65	$-0.028$
40–50	84,593	2.23	$-0.160$	87,727	3.26	$-0.031$

Appendix

Table 4: The Conditional Bias, UDF, and mean residual

r=y-\hat{\mu}

behind Figure 1 , reported by decile of each conditioning variable. Deciles are within-catchment flow percentiles; ties in observed discharge leave the

y

-conditioned deciles slightly uneven. The null for the UDF is exactly 1 under

\hat{\mu}

-conditioning; the null for the mean residual is

0

. Top-decile catchment-clustered bootstrap 95% intervals (

B=1000

5.76

[

4.48

6.90

] conditioned on

y

4.27

[

3.50

4.99

] conditioned on

\hat{\mu}

. Bold outlines the out of sample test.

	all test				top flow decile
fraction removed	spread	flow	oracle	random	spread	flow	oracle	random
$0.0$	$1.000$	$1.000$	$1.000$	$1.000$	$1.000$	$1.000$	$1.000$	$1.000$
$0.1$	$0.294$	$0.351$	$0.139$	$1.000$	$0.610$	$0.670$	$0.208$	$1.002$
$0.2$	$0.183$	$0.217$	$0.080$	$1.003$	$0.465$	$0.575$	$0.144$	$0.997$
$0.3$	$0.129$	$0.157$	$0.052$	$1.002$	$0.389$	$0.483$	$0.107$	$0.998$
$0.4$	$0.097$	$0.117$	$0.036$	$0.999$	$0.302$	$0.422$	$0.081$	$0.997$

Appendix

Table 5: Sparsification curves behind Figure 2 a, on the plotted grid of

91

points, with removal in

1\%

steps to

90\%

removed, sampled here every tenth point. Each curve is normalized by its set’s full-sample RMSE, so every ordering starts at

1.000

and AUSE is unaffected by the normalization. Spread orders catchment-days by reported ensemble variance, flow by the ensemble mean prediction

\hat{\mu}

alone, oracle by realized squared error, and random by a seeded permutation averaged over ten draws, flat to within

0.016

. The flow column is the comparator for the ordering claim; it carries no uncertainty information, only predicted magnitude, and is computable at deployment. These are integrated over a

91

-point grid, not over the ten rows shown.

			UDF
AUSE decile	$n$	AUSE range	median	min	max
0–10	$27$	$[0.024,0.071]$	$3.45$	$1.68$	$10.40$
10–20	$27$	$[0.072,0.092]$	$3.14$	$1.17$	$5.94$
20–30	$27$	$[0.093,0.105]$	$2.44$	$1.38$	$5.25$
30–40	$27$	$[0.106,0.122]$	$2.16$	$1.39$	$4.68$
40–50	$26$	$[0.123,0.136]$	$2.44$	$1.03$	$10.78$

Appendix

Table 6: The 269 catchments of Figure 2 b, binned into deciles of per-catchment AUSE. Per-catchment rows are omitted for space, so the panel’s claim is given in its smallest form: sorting catchments by how well their spread ranks error does not sort them by how well it is scaled. UDF is computed within each catchment; Spearman

\rho=-0.23

(

p=0.00014

). Every catchment

(n=269)

exceeds the null of

1

, minimum

1.03

Streamflow Forecasting

See all 10 papers

Joint distribution of upstream runoff governs downstream river-discharge prediction uncertainty in distributed ML models
Jul 3, 2026Karan Ruparell, Tristan Hascoet, Takemasa Miyoshi +4Streamflow Forecasting Uncertainty Quantification
University of Reading, Department of Meteorology, Reading, UK · European Centre for Medium-Range Weather Forecasts (ECMWF), Reading, UK · RIKEN Center for Computational Sciences, Kobe, Japan +3
Distributed Hydrological Modeling in the Feature Space
Sep 26, 2026Mohamad Hakam Shams Eddin, Maria Luisa Taccari, Yikui Zhang +3Streamflow Forecasting Hydrology
Max Planck Institute for Biogeochemistry, Germany · European Centre for Medium-Range Weather Forecasts (ECMWF), United Kingdom · Institute of Bio- and Geosciences Agrosphere (IBG-3), Research Centre Jülich, Germany +2
Boundary-Aware Uncertainty Quantification for Wildfire Spread Prediction
May 4, 2026Jonas V. FunkUncertainty Quantification Wildfire Forecasting
Independent Research 76199 Karlsruhe, Germany

Jul 3, 2026J/K move · Enter open · S save

Joint distribution of upstream runoff governs downstream river-discharge prediction uncertainty in distributed ML models

Karan Ruparell, Tristan Hascoet, Takemasa Miyoshi, Kieran M. R. Hunt+3

University of Reading, Department of Meteorology, Reading, UK · European Centre for Medium-Range Weather Forecasts (ECMWF), Reading, UK · RIKEN Center for Computational Sciences, Kobe, Japan+3

Streamflow Forecasting Uncertainty Quantification Time Series Forecasting Probabilistic Forecasting Hydrology

Open paper AskPDF arXiv

Uncertainty quantification of hydrological predictions is necessary to inform operational decisions. Recent generative machine-learning methods have advanced probabilistic streamflow prediction, but have remained confined to lumped models that predict a basin outlet directly. At the same time, deterministic LSTM runoff models are increasingly applied at grid or catchment scale and routed through river networks to produce spatially continuous, physically consistent discharge fields. This technical note argues that moving probabilistic prediction from lumped to distributed models introduces a specific new requirement: the joint distribution of upstream runoff generation must be sampled jointly. In lumped inference, the model predicts the outlet distribution directly and can modulate spread from basin attributes. In distributed inference, downstream discharge is obtained by routing many upstream runoff predictions, so independent local sampling averages uncertainty away. Using Japan as a case study, we train two probabilistic basin-scale runoff LSTMs and route their runoff through a Hayami routing scheme. Randomly matching upstream ensemble members produces severely under-dispersed downstream ensembles, whereas a simple quantile matching strategy restores much of the spread of the direct basin-scale reference. The shift from lumped to distributed probabilistic hydrology therefore requires explicit attention to the spatial joint structure of runoff uncertainty.

Abstract

Streamflow Forecasting

Explore similar work

Joint distribution of upstream runoff governs downstream river-discharge prediction uncertainty in distributed ML models

Distributed Hydrological Modeling in the Feature Space

Boundary-Aware Uncertainty Quantification for Wildfire Spread Prediction