Calibrated uncertainty quantification for prosumer flexibility aggregation in ancillary service markets

Yogesh Pipada Sunil Kumar; S. Ali Pourmousavi; Jon A. R. Liisberg; Julian Lemos-Vinasco

eess.SYJan 21, 2026

Calibrated uncertainty quantification for prosumer flexibility aggregation in ancillary service markets

Authors: Yogesh Pipada Sunil Kumar, S. Ali Pourmousavi, Jon A. R. Liisberg, Julian Lemos-Vinasco

Organizations: School of Electrical and Mechanical Engineering, The University of Adelaide, Ingkarni Wardli Building, Adelaide, SA 5005, Australia · Watts A/S, Brogade 19D, Køge 4600, Denmark

Abstract

Reliable forecasting of prosumer flexibility is critical for demand response aggregators in frequency controlled ancillary service markets, where reliability requirements such as the P90 standard are enforced. Limited behavioural data and dependence on exogenous forecasts introduce uncertainty into flexibility estimates, which can cause reserve overestimation and financial penalties. This study systematically evaluates established uncertainty quantification approaches based on Monte Carlo dropout (MCD), conformal prediction (CP), and conformalised quantile regression (CQR) to construct prediction intervals for aggregated prosumer flexibility and support uncertainty-aware bidding. The approaches are evaluated for a behind-the-meter aggregator in the Danish manual frequency restoration reserve capacity market, using a large-scale synthetic dataset generated with an industry-grade home energy management system and publicly available load, solar, price, activation, and device-level data. Multiple multivariate CP strategies are applied to MCD predictions, alongside CQR using a separately trained quantile regression model. Under nominal test conditions, conformal methods substantially improve joint coverage over uncalibrated MCD baselines. More conservative methods satisfy the 90% lower-tail reliability requirement, while CQR and multivariate CP (MCP) yield narrower intervals and higher economic value, retaining approximately 69% and 68% of perfect-information profit, respectively, under the stricter penalty interpretation. A controlled robustness analysis shows that increasing perturbations of load, solar, price, and activation inputs increase overbidding and reduce coverage and economic performance. The results highlight the trade-off between reliability, economic value, and robustness to input uncertainty in aggregator bidding.

Figures & tables

\hat{\alpha}=\frac{\lceil(|\mathcal{D}_{\text{cal}}|+1)(1-\alpha)\rceil}{|\mathcal{D}_{\text{cal}}|+1}

Algorithm 1 Hybrid MCD and split CP for obtaining multivariate prediction intervals

Figure 2: Example conformal prediction region

\xi

for

\boldsymbol{y}\in\mathbb{R}^{2}

with bounding rectangle

\mathcal{B}_{\xi}

Panel of Figure 3 — Figure 3: Illustration of prediction regions $\xi$ and corresponding bounding hyperrectangles for $\boldsymbol{y}\in\mathbb{R}^{2}$ , across the evaluated CP methods.

Figure 4: Synthetic data generation framework for aggregated HEMS flexibility

Hyperparameter	MCD model	QR model
Batch size	32	16
Number of epochs	1000	1900
Optimiser	Adam	Adam
Learning rate	$3\times 10^{-4}$	$4.3\times 10^{-5}$
Loss metric	mean square error	Pinball loss
Network architecture	2 hidden layers	2 hidden layers

Table 1: Neural network training hyperparameters

Panel of Figure 5 — Figure 5: Hourly residuals between true and predicted aggregated up-regulation flexibility for the MCD and QR models

Panel of Figure 6 — Figure 6: Distribution of per-sample RMSE between true and predicted aggregated up-regulation flexibility for MCD and QR models

UQ Method	Expected	Joint	Joint MPIW	Joint IS
	Coverage	PICP	kW	kW
Individual quantiles	0.80	0.00	33.20	128.26
Naïve joint quantiles	0.80	0.00	86.56	122.58
CQR	0.80	0.80	144.18	147.99
MCP	0.80	0.80	163.90	167.86
MMCP	0.90	0.90	211.37	213.54

Table 2: Joint prediction interval quality comparison across various methods

Panel of Figure 7 — Figure 7: Distribution of daily overbid frequency for the evaluated prediction methods

Panel of Figure 8 — Figure 8: Distribution of daily average predicted up-regulation flexibility lower bounds as a percentage of aggregated maximum battery discharge power

Panel of Figure 9 — Figure 9: Hourly heatmap for different parameters across evaluated prediction methods

Figure 10: Mean daily unadjusted and adjusted profit for all UQ methods across all

\beta\in\mathcal{S}_{\beta}

UQ Method	Mean percentage of PI profit (%)
	$R(\boldsymbol{x^{\text{u}}},\beta)$	$R_{1}(\boldsymbol{x^{\text{u}}},\beta)$	$R_{2}(\boldsymbol{x^{\text{u}}},\beta)$
Sample mean $\boldsymbol{\mu}$	101	-56	-97
Individual quantiles	94	10	-43
Naïve joint quantiles	87	46	-10
CQR	73	70	69
MCP	71	69	68

Table 3: Mean percentage of PI profit for each UQ method

Panel of Figure 11 — Figure 11: Sensitivity of aggregator performance to the revenue-sharing factor $\beta$

Panel of Figure 12 — Figure 12: Sensitivity of interval evaluation metrics to simultaneous perturbations of all input sources.

Panel of Figure 13 — Figure 13: Sensitivity of mean daily overbid frequency to perturbations in individual input sources.

Panel of Figure 14 — Figure 14: Sensitivity of adjusted profit to input perturbations for $\beta=0.1$ .

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Block	Symbol	Quantity	Perturbation treatment
Purchase price	$\lambda^{P}$	Retail purchase price forecast	Coupled additive
Selling price	$\lambda^{S}$	Retail selling price forecast	Coupled additive
Incentive price	$\lambda^{I,u}$	Aggregator incentive price	Not perturbed
Activation signal	$a^{\uparrow}$	mFRR up-regulation activation forecast	Copula partial resampling
Load	$x_{l}$	Aggregated load forecast	Multiplicative log-normal
Solar	$x_{s}$	Aggregated PV forecast	Multiplicative log-normal

Appendix

Table 4: Input feature groups and corresponding perturbation treatment.

Input group	Perturbation	Distribution	Level parameter
Load, solar	Multiplicative $x\exp(\varepsilon)$	$\varepsilon\sim\mathcal{N}(0,s^{2})$	$s$ = log-SD
Retail prices	Shared additive shock	$\epsilon\sim\mathcal{N}(0,s^{2}\sigma_{S}^{2})$	$s\sigma_{S}$
Other price inputs	Additive $x+\eta$	$\eta\sim\mathcal{N}(0,s^{2}\sigma^{2})$	$s\sigma$
Activation	Copula partial resampling	Gaussian copula $(\boldsymbol{\Sigma},\boldsymbol{p})$	$s$ = fraction of hours
Settlement price	Additive $\lambda^{U,\mathrm{pre}}+\xi$	$\xi\sim\mathcal{N}(0,s^{2}\sigma^{2})$	$s\sigma$

Appendix

Table 5: Summary of perturbation models used in the robustness analysis.

Uncertainty Quantification

Uncertainty quantification estimates how reliable predictions and scientific inferences are, and how uncertainty propagates into decisions.

See all 647 papers

CoCopula: Conformal Calibration and Copula Aggregation for Reliable Renewable Energy Forecasting
Jan 31, 2026Alireza Moradi, Mathieu Tanneau, Reza Zandehshahvar +1Probabilistic Forecasting Solar Power Forecasting
NSF Artificial Intelligence Institute for Advances in Optimization H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology, Atlanta, GA, USA
Investigating Calibration Challenges in Probabilistic Electricity Price Forecasting
Jun 8, 2026Jan Niklas Lettner, Hadeer El Ashhab, Benjamin SchäferProbability Calibration Electricity Price Forecasting
Institute for Automation and Applied Informatics
Decision-Calibrated Conformal Uncertainty for Pacing Decisions in Streaming Advertising
Jun 8, 2026Prashant Shekhar, Caroline HowardSequential Decision Making Online Advertising
Department of Mathematics Embry-Riddle Aeronautical University Daytona Beach, FL, USA

Jan 31, 2026J/K move · Enter open · S save

CoCopula: Conformal Calibration and Copula Aggregation for Reliable Renewable Energy Forecasting

Alireza Moradi, Mathieu Tanneau, Reza Zandehshahvar, Pascal Van Hentenryck

NSF Artificial Intelligence Institute for Advances in Optimization H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology, Atlanta, GA, USA

Probabilistic Forecasting Solar Power Forecasting Conformal Prediction

Open paper AskPDF arXiv

The rapid growth of renewable energy penetration has intensified the need for reliable probabilistic forecasts to support grid operations at aggregated (fleet or system) levels. In practice, however, system operators often lack access to fleet-level probabilistic models and instead rely on site-level forecasts produced by heterogeneous third-party providers. Constructing coherent and calibrated fleet-level probabilistic forecasts from such inputs remains challenging due to complex cross-site dependencies and aggregation-induced miscalibration. Motivated by a real-world operational challenge identified through collaboration with an industry partner, this paper presents a probabilistic aggregation and calibration framework that directly converts site-level probabilistic forecasts into reliable fleet-level forecasts. The paper introduces \method~ (Conformal Calibration and Copula Aggregation), which integrates Copula-based dependence modeling to capture cross-site correlations followed by Context-Aware Conformal Prediction (CACP) to mitigate miscalibration at the aggregated level. This combination enables dependence-aware aggregation while improving empirical coverage and maintaining sharp prediction intervals. Experiments on large-scale solar generation datasets from three system operators in the United States including MISO, ERCOT, and SPP demonstrate that the proposed \method{} approach achieves near-nominal empirical coverage and consistently lower Winkler scores than competing aggregation-based baselines across different target coverage levels. Importantly, \method{} achieves performance comparable to directly calibrated system-level forecasts, demonstrating its practical value when dedicated fleet-level forecasting models are unavailable.

Abstract

Uncertainty Quantification

Explore similar work

CoCopula: Conformal Calibration and Copula Aggregation for Reliable Renewable Energy Forecasting

Investigating Calibration Challenges in Probabilistic Electricity Price Forecasting

Decision-Calibrated Conformal Uncertainty for Pacing Decisions in Streaming Advertising