A Generalized Synthetic Control Method for Baseline Estimation in Demand Response Services

Extreme weather and volatile wholesale electricity markets expose residential consumers to catastrophic financial risks, yet demand response at the distribution level remains an underutilized tool for grid flexibility and energy affordability. While a demand-response program can shield consumers by issuing financial credits during high-price periods, optimizing this sequential decision-making process presents a unique challenge for reinforcement learning despite the plentiful offline historical smart meter and wholesale pricing data available publicly. Offline historical data fails to capture the dynamic, interactive feedback loop between an electric utility's pricing signals and customer acceptance and adaptation to a demand-response program. To address this, we introduce DR-Gym, an open-source, online Gymnasium-compatible environment designed to train and evaluate demand-response from the electric utility's perspective. Unlike existing device-level energy simulators, our environment focuses on the market-level electric utility setting and provides a rich observational space relevant to the electric utility. The simulator additionally features a regime-switching wholesale price model calibrated to real-world extreme events, alongside physics-based building demand profiles. For our learning signal, we use a configurable, multi-objective reward function for specifying diverse learning objectives. We demonstrate through baseline strategies and data snapshots the capability of our simulator to create realistic and learnable environments.

With grid operators confronting rising uncertainty from renewable integration and a broader push toward electrification, Demand-Side Management (DSM) -- particularly Demand Response (DR) -- has attracted significant attention as a cost-effective mechanism for balancing modern electricity systems. Unprecedented volumes of consumption data from a continuing global deployment of smart meters enable consumer segmentation based on real usage behaviours, promising to inform the design of more effective DSM and DR programs. However, existing clustering-based segmentation methods insufficiently reflect the behavioural diversity of consumers, often relying on rigid temporal alignment, and faltering in the presence of anomalies, missing data, or large-scale deployments. To address these challenges, we propose a novel two-stage clustering framework -- Clustered Representations Optimising Consumer Segmentation (CROCS). In the first stage, each consumer's daily load profiles are clustered independently to form a Representative Load Set (RLS), providing a compact summary of their typical diurnal consumption behaviours. In the second stage, consumers are clustered using the Weighted Sum of Minimum Distances (WSMD), a novel set-to-set measure that compares RLSs by accounting for both the prevalence and similarity of those behaviours. Finally, community detection on the WSMD-induced graph reveals higher-order prototypes that embody the shared diurnal behaviours defining consumer groups, enhancing the interpretability of the resulting clusters. Extensive experiments on both synthetic and real Australian smart meter datasets demonstrate that CROCS captures intra-consumer variability, uncovers both synchronous and asynchronous behavioural similarities, and remains robust to anomalies and missing data, while scaling efficiently through natural parallelisation. These results...

Synthetic control (SC) matches a treated unit's pre-treatment trajectory to a weighted combination of donor units. We study Spectral SC, which instead matches the treated unit in coordinates defined by the leading temporal singular vectors of the donor panel, and a hybrid estimator that places separately tunable weight on retained and discarded directions, nesting raw-path SC and truncated Spectral SC as endpoints. We prove that the family reduces exactly to raw-path SC at full rank, that exact balance on $K$ retained dimensions with $N_0$ donors is underdetermined whenever $N_0>K+1$, with an affine solution set of dimension $N_0-K-1$, and that spectral imbalance maps to treatment-effect bias through a finite-sample best-linear-predictor decomposition. We evaluate the estimators across eleven data-generating regimes, using $400$ replications per regime and donor-only placebo validation to select regularization and the mixing weight. Truncated Spectral SC has significantly higher RMSE than tuned raw-path SC in every regime, with paired differences equal to $4$ to $11$ Monte Carlo standard errors. The hybrid estimator selects raw-path matching in most replications and is statistically indistinguishable from tuned SC in most regimes. The result is highly sensitive to preprocessing. With raw inputs, the performance gap is large; after removing unit and time fixed effects before spectral decomposition, as suggested by the assumptions behind our bound, the gap nearly disappears and placebo validation begins to favor truncation. We interpret these findings diagnostically rather than as evidence that Spectral SC should replace raw-path SC. Basis-estimation noise, balancing underdetermination, and fixed-effects contamination determine when spectral matching can help.