R$^2$Energy: A Large-Scale Benchmark for Robust Renewable Energy Forecasting under Diverse and Extreme Conditions

Zhi Sheng; Yuan Yuan; Guozhen Zhang; Yong Li

cs.LGFeb 17, 2026

R $^2$ Energy: A Large-Scale Benchmark for Robust Renewable Energy Forecasting under Diverse and Extreme Conditions

Authors: Zhi Sheng, Yuan Yuan, Guozhen Zhang, Yong Li

Organizations: Center for Urban Science and Computation, Tsinghua University · Courant Institute of Mathematical Sciences, New York University

Abstract

The rapid expansion of renewable energy, particularly wind and solar power, has made reliable forecasting critical for power system operations. While recent deep learning models have achieved strong average accuracy, the increasing frequency and intensity of climate-driven extreme weather events pose severe threats to grid stability and operational security. Consequently, developing robust forecasting models that can withstand volatile conditions has become a paramount challenge. In this paper, we present R $^2$ Energy, a large-scale benchmark for NWP-assisted renewable energy forecasting. It comprises over 10.7 million high-fidelity hourly records from 902 wind and solar stations across four provinces in China, providing the diverse meteorological conditions necessary to capture the wide-ranging variability of renewable generation. We further establish a standardized, leakage-free forecasting paradigm that grants all models identical access to future Numerical Weather Prediction (NWP) signals, enabling fair and reproducible comparison across state-of-the-art representative forecasting architectures. Beyond aggregate accuracy, we incorporate regime-wise evaluation with expert-aligned extreme weather annotations, uncovering a critical ``robustness gap'' typically obscured by average metrics. This gap reveals a stark robustness-complexity trade-off: under extreme conditions, a model's reliability is driven by its meteorological integration strategy rather than its architectural complexity. R $^2$ Energy provides a principled foundation for evaluating and developing forecasting models for safety-critical power system applications.

Figures & tables

Figure 1. Overall framework of R 2 Energy.

Type	Dataset	Province	# of Stations	Time Period	Total Records	Extreme (Lvl 1-3)		Variables & Settings
Type	Dataset	Province	# of Stations	Time Period	Total Records	Rainstorm	Heatwave	Variables & Settings
Wind	W1	Anhui	110	2023-01-01 – 2024-11-18	1,548,258	6,723	56,839	Physical Vars: $ws_{10},ssrd,u_{10},v_{10},t_{2m},tp,CF$ Quality Control: Masks for Missing & Outliers Extreme Labels: Rainstorm & Heatwave (Lvl 1-3)
	W2	Shanxi	233	2023-10-30 – 2025-03-25	2,324,213	1,882	3,953
	W3	Shandong	20	2024-12-01 – 2025-07-14	82,752	0	2,164
Solar	S1	Anhui	242	2023-01-01 – 2024-11-18	3,545,797	13,771	135,802
	S2	Shanxi	241	2023-10-30 – 2024-12-31	2,362,101	2,434	7,848

Table 1. Statistics of the six datasets. The rightmost column lists the variables and settings shared by all datasets.

Figure 2. Intra-day and seasonal variability across the wind and solar datasets. The top row (a) illustrates the average wind speed (m/s) variations between daytime (07:00–18:00, solid bars) and nighttime (dotted bars). The bottom row (b) compares solar radiation (W/m²) during peak midday hours (11:00–14:00, solid bars) versus shoulder hours (morning/evening, dotted bars). The numerical labels above the brackets denote the difference between the two periods, while the global mean and standard deviation are listed for each dataset. Missing bars in W3 indicate data unavailability for specific seasons.

Event Type	Level	Warning	Criteria
Rainstorm	Level 1	Blue	Rainfall $\geq 50$ mm within 12 h
	Level 2	Yellow	Rainfall $\geq 50$ mm within 6 h
	Level 3	Orange	Rainfall $\geq 50$ mm within 3 h
Heatwave	Level 1	Yellow	Max temperature $>35^{\circ}$ C for 3 consecutive days
	Level 2	Orange	Max temperature $>37^{\circ}$ C within 24 h
	Level 3	Red	Max temperature $>40^{\circ}$ C within 24 h

Table 2. Definition of extreme meteorological events and severity levels used in RexBench.

Figure 3. Impact of extreme weather on renewable power generation. (a) Solar (S2) and (b) Wind (W2) datasets. Shaded areas in (a) denote 95% confidence intervals; values in (b) indicate medians.

Forecast Type	Horizon	Operational Use Cases
Ultra-Short-Term (USTF)	$\leq$ 1 hour	Real-time dispatch,
		Automatic Generation Control,
		Power Smoothing.
Short-Term (STF)	$\leq$ 1 day	Economic Dispatch,
		Reserve Scheduling,
		Day-ahead Market Operations.

Table 3. Taxonomy of Renewable Energy Forecasting Horizons and Applications.

Figure 4. Power curve of the GE 2.5MW wind turbine. The curve illustrates the non-linear relationship between wind speed and power generation, highlighting three operational regions: cut-in (

v_{in}\approx 3

m/s), rated (

v_{rated}\approx 12.5

m/s), and cut-out (

v_{out}=25

m/s).

Model	S1			S2			S3
Model	MAE $\downarrow$	RMSE $\downarrow$	Q (%) $\uparrow$	MAE $\downarrow$	RMSE $\downarrow$	Q (%) $\uparrow$	MAE $\downarrow$	RMSE $\downarrow$	Q (%) $\uparrow$
Panel A: Ultra-short-term Forecast (Horizon = 1)
Baseline	0.1283/0.1318	0.1725/0.2151	86.41/80.89	0.1760/0.1545	0.2346/0.2490	72.33/76.39	0.1166/0.1274	0.1717/0.2054	86.75/82.40
RNN	0.0527 (0.001)	0.0866 (0.000)	97.70 (0.026)	0.0576 (0.003)	0.0964 (0.003)	96.78 (0.217)	0.0570 (0.001)	0.0888 (0.001)	97.57 (0.047)
LSTM	0.2170 (0.135)	0.2997 (0.174)	69.83 (22.70)	0.2034 (0.124)	0.2930 (0.164)	71.52 (20.78)	0.1744 (0.098)	0.2536 (0.137)	76.18 (17.56)
Bi-LSTM	0.1628 (0.134)	0.2289 (0.174)	79.12 (22.72)	0.1541 (0.123)	0.2268 (0.164)	80.01 (20.78)	0.1348 (0.098)	0.1981 (0.136)	83.35 (17.56)

Table 4. Prediction results on three Datasets (S1, S2, S3). The Baseline corresponds to a physics-informed photovoltaic formulation and the Seasonal-Naive method, respectively. The tasks are categorized into three Panels based on forecast horizons. Results are formatted as Mean (Std) . Darker background indicates better performance (Rank 1 to 4).

Model	W1			W2			W3
Model	MAE $\downarrow$	RMSE $\downarrow$	Q (%) $\uparrow$	MAE $\downarrow$	RMSE $\downarrow$	Q (%) $\uparrow$	MAE $\downarrow$	RMSE $\downarrow$	Q (%) $\uparrow$
Panel A: Ultra-short-term Forecast (Horizon = 1)
Baseline	0.2042/0.2106	0.2975/0.2934	66.86/-	0.2544/0.2546	0.3650/0.3451	61.07/59.58	0.1443/0.1747	0.2220/0.2480	80.01/74.68
RNN	0.0586 (0.001)	0.0889 (0.000)	97.85 (0.050)	0.0627 (0.001)	0.0957 (0.000)	97.29 (0.032)	0.0600 (0.001)	0.0939 (0.001)	97.31 (0.055)
LSTM	0.1761 (0.098)	0.2536 (0.137)	74.64 (13.70)	0.2019 (0.114)	0.2889 (0.159)	72.88 (20.00)	0.1398 (0.053)	0.2048 (0.080)	81.98 (11.93)
Bi-LSTM	0.1364 (0.097)	0.1979 (0.137)	80.21 (13.68)	0.1555 (0.114)	0.2241 (0.159)	81.06 (20.01)	0.1168 (0.054)	0.1693 (0.082)	87.15 (12.17)

Table 5. Prediction results on three Datasets (W1, W2, W3). The Baseline corresponds to a turbine power-curve-based formulation and the Seasonal-Naive method, respectively. The tasks are categorized into three Panels based on forecast horizons. Results are formatted as Mean (Std) . Darker background indicates better performance (Rank 1 to 4).

Panel A: Short-term Forecast (Horizon = 24) on S1
Model	Normal			Heatwave			Rainstorm
Model	MAE $\downarrow$	Q (%) $\uparrow$	S (%) $\uparrow$	MAE $\downarrow$	Q (%) $\uparrow$	S (%) $\uparrow$	MAE $\downarrow$	Q (%) $\uparrow$	S (%) $\uparrow$
Baseline	0.1269/0.1397	86.10/79.36	0.00 (–)	0.1309/0.0697	90.67/92.39	0.00 (–)	0.0687/0.2064	95.93/70.46	0.00 (–)
RNN	0.0885 (0.001)	92.97 (0.045)	30.26 (0.788)	0.0711 (0.001)	97.54 (0.105)	45.68 (0.764)	0.0615 (0.002)	97.29 (0.310)	10.48 (2.911)
LSTM	0.1350 (0.089)	84.66 (15.81)	-6.38 (70.13)	0.1437 (0.145)	85.36 (24.10)	-9.78 (110.7)	0.0730 (0.009)	96.13 (1.990)	-6.26 (13.10)
Bi-LSTM	0.1799 (0.108)	76.72 (19.33)	-41.77 (85.11)	0.2168 (0.177)	73.27 (29.49)	-65.62 (135.2)	0.0760 (0.012)	95.16 (2.454)	-10.63 (17.46)

Table 6. Robustness Evaluation under Extreme Weather Conditions (Task: Short-Term Forecasting, H=24). We report the MAE, Qualification Rate (Q), and Forecast Skill Score ( S ). S measures the relative improvement over the Baseline; higher S indicates better performance . The Baseline corresponds to a physics-informed photovoltaic formulation and the Seasonal-Naive method. Results are formatted as Mean (Std) . Darker background indicates better performance (Rank 1 to 4).

Figure 5. Efficiency-Accuracy Landscape (Solar S1, H=24). The scatter plot illustrates the trade-off between computational cost (Training Time per Epoch, Log Scale) and predictive performance (MAE). The dashed line represents the physical Baseline error (0.1272).

Forecasting Benchmarks

Forecasting Benchmarks are datasets and evaluation protocols for comparing predictions of future time-series values, events, and outcomes.

See all 224 papers

Sep 21, 2026

WPBench: A Comprehensive Benchmark for Wind Power Forecasting

Accurate, reliable, and deployable wind power forecasting is critical for power system dispatch, renewable energy integration, and electricity market operations. Progress in this field hinges on the ability to empirically and comprehensively benchmark forecasting methods. Yet existing benchmarks fall short of supporting systematic evaluation in four key aspects: 1) limited coverage of wind power scenarios across turbine scale, variable composition, and spatial structure; 2) incomplete coverage of forecasting model families; 3) evaluation metrics misaligned with wind power requirements; and 4) limited structure-aware diagnostics beyond individual temporal patterns. To address these limitations, we propose WPBench, a comprehensive, fair, and extensible benchmark for wind power forecasting. WPBench integrates 26 public datasets organized by turbine scale and variable composition, spanning single-turbine, multi-turbine, univariate, and multivariate settings. Under unified processing, training, and evaluation protocols, it benchmarks 19 representative models covering traditional methods, deep temporal models, spatio-temporal models, and foundation models. Beyond point-wise errors, WPBench assesses forecast-curve fidelity and computational efficiency, and delivers structure-aware diagnostics across temporal, variable-dependency, and spatial-dependency perspectives. Together, these capabilities enable systematic model comparison across diverse wind scenarios and provide a reusable platform for future research.

Yuhan Zhu, Jilin Hu, Xinying Cai +7

School of Data Science & Engineering, East China Normal University, Shanghai, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Hangzhou Hikvision Digital Technology Co., Ltd., Hangzhou, China

Multivariate Time Series Forecasting Benchmark Design Time Series Forecasting Forecasting Benchmarks Spatiotemporal Forecasting Wind Power Forecasting

Ask PDF

Apr 27, 2026

Energy-Arena: A Dynamic Benchmark for Operational Energy Forecasting

Energy forecasting research faces a persistent comparability gap that makes it difficult to measure consistent progress over time. Reported accuracy gains are often not directly comparable because models are evaluated under study-specific datasets, time periods, information sets, and scoring setups, while widely used benchmarks and competition datasets are typically tied to fixed historical windows. This paper introduces the Energy-Arena, a dynamic benchmarking platform for operational energy time series forecasting that provides a continuously updated reference point as energy systems evolve. The platform operates as an open, API-based submission system and standardizes challenge definitions and submission deadlines aligned with operational constraints. Performance is reported on rolling evaluation windows via persistent leaderboards. By moving from retrospective backtesting to forward-looking benchmarking, the Energy-Arena enforces standardized ex-ante submission and ex-post evaluation, thereby improving transparency by preventing information leakage and retroactive tuning. The platform is publicly available at Energy-Arena.org.

Max Kleinebrahm, Jonathan Berrisch, Philipp Eiser +11

Karlsruhe Institute of Technology, Institute for Industrial Production, Karlsruhe, Germany · University of Duisburg-Essen, House of Energy Markets and Finance, Essen, Germany · Karlsruhe Institute of Technology, Institute for Automation and Applied Informatics, Eggenstein-Leopoldshafen, Germany +5

Benchmark Design Time Series Forecasting Forecasting Benchmarks

Ask PDF

Feb 24, 2026

Benchmarking State Space Models, Transformers, and Recurrent Networks for US Grid Forecasting

Selecting the right deep learning model for power grid forecasting is challenging, as performance heavily depends on the data available to the operator. This paper presents a comprehensive benchmark of five modern neural architectures: two state space models (PowerMamba, S-Mamba), two Transformers (iTransformer, PatchTST), and a traditional LSTM. We evaluate these models on hourly electricity demand across six diverse US power grids for forecast windows between 24 and 168 hours. To ensure a fair comparison, we adapt each model with specialized temporal processing and a modular layer that cleanly integrates weather covariates. Our results reveal that there is no single best model for all situations. When forecasting using only historical load, PatchTST and the state space models provide the highest accuracy. However, when explicit weather data is added to the inputs, the rankings reverse: iTransformer improves its accuracy three times more efficiently than PatchTST. By controlling for model size, we confirm that this advantage stems from the architecture's inherent ability to mix information across different variables. Extending our evaluation to solar generation, wind power, and wholesale prices further demonstrates that model rankings depend on the forecast task: PatchTST excels on highly rhythmic signals like solar, while state space models are better suited for the chaotic fluctuations of wind and price. Ultimately, this benchmark provides grid operators with actionable guidelines for selecting the optimal forecasting architecture based on their specific data environments.

Sunki Hong, Jisoo Lee

Gramm AI

Forecasting Benchmarks Electricity Load Forecasting Multivariate Time Series Forecasting State Space Models Time Series Transformers Recurrent Neural Networks

Ask PDF

Abstract

Forecasting Benchmarks

Explore similar work

WPBench: A Comprehensive Benchmark for Wind Power Forecasting

Energy-Arena: A Dynamic Benchmark for Operational Energy Forecasting

Benchmarking State Space Models, Transformers, and Recurrent Networks for US Grid Forecasting