Benchmarking State Space Models, Transformers, and Recurrent Networks for US Grid Forecasting

Sunki Hong; Jisoo Lee

cs.LGFeb 24, 2026

Benchmarking State Space Models, Transformers, and Recurrent Networks for US Grid Forecasting

Authors: Sunki Hong, Jisoo Lee

Organizations: Gramm AI

Abstract

Selecting the right deep learning model for power grid forecasting is challenging, as performance heavily depends on the data available to the operator. This paper presents a comprehensive benchmark of five modern neural architectures: two state space models (PowerMamba, S-Mamba), two Transformers (iTransformer, PatchTST), and a traditional LSTM. We evaluate these models on hourly electricity demand across six diverse US power grids for forecast windows between 24 and 168 hours. To ensure a fair comparison, we adapt each model with specialized temporal processing and a modular layer that cleanly integrates weather covariates. Our results reveal that there is no single best model for all situations. When forecasting using only historical load, PatchTST and the state space models provide the highest accuracy. However, when explicit weather data is added to the inputs, the rankings reverse: iTransformer improves its accuracy three times more efficiently than PatchTST. By controlling for model size, we confirm that this advantage stems from the architecture's inherent ability to mix information across different variables. Extending our evaluation to solar generation, wind power, and wholesale prices further demonstrates that model rankings depend on the forecast task: PatchTST excels on highly rhythmic signals like solar, while state space models are better suited for the chaotic fluctuations of wind and price. Ultimately, this benchmark provides grid operators with actionable guidelines for selecting the optimal forecasting architecture based on their specific data environments.

Figures & tables

Panel of Figure 1 — Fig. 1 : Weather integration strategies. Each subfigure shows the baseline architecture (top) and its weather-integrated variant (bottom). (a) S-Mamba: early fusion by embedding-space summation. (b) PowerMamba: summation into decomposed streams. (c) PatchTST: channel fusion via independent patching. (d) iTransformer: tokenization of weather variables for cross-variate attention.

ISO	EIA ID	Peak (GW)	Characteristics
CAISO	EIA-CISO	47.1	High solar, duck curve
ISO-NE	EIA-ISNE	25.6	Weather-sensitive, winter peaks
MISO	EIA-MISO	127.1	Large, wind-rich
PJM	EIA-PJM	150.4	Largest US ISO
SWPP	EIA-SWPP	54.4	Wind-rich, central US †
ERCOT	EIA-ERCO	85.5	Isolated grid, extreme weather

TABLE I : US grid operators in the benchmark. Peak load and primary operating characteristics. † SWPP is used in weather experiments only (Section V-C ).

		SSM				Transformer				RNN
ISO	$W$	PowerMamba		S-Mamba		iTransformer		PatchTST		LSTM
		MSE	MAPE	MSE	MAPE	MSE	MAPE	MSE	MAPE	MSE	MAPE
		(%)	(%)	(%)	(%)	(%)	(%)	(%)	(%)	(%)	(%)
CAISO	24	5.12	3.38	4.74	3.17	7.30	5.16	4.70	3.12	5.91	4.08
	48	7.35	4.65	6.18	4.24	6.42	4.55	5.86	3.88	7.76	5.49
	72	8.70	5.45	7.47	5.14	7.21	5.12	6.73	4.49	8.83	6.08

TABLE II : US ISO benchmark on load forecasting. Results are reported per grid operator and prediction window (

W=24,48,72,96,168

) with fixed input length (

L=240

). For each model we report MSE (%) and MAPE (%) (lower is better). MSE (%) is normalized root mean squared error as a percentage of mean load. Best values per row are highlighted in bold . Full tail distribution metrics (

P_{0.5}/P_{99.5}

) for all models are reported separately in Table III .

Grid	$W$	PwrM.	S-Mamba	iTrnsf.	PTST	LSTM
		(%)	(%)	(%)	(%)	(%)
CAISO	24	-14.1/13.4	-12.2/12.5	-17.1/19.8	-12.6/11.6	-15.9/14.2
	48	-17.1/24.9	-14.3/18.1	-17.9/17.6	-15.3/15.8	-19.6/18.5
	72	-21.0/29.7	-16.3/21.2	-19.5/19.7	-17.7/16.4	-21.6/19.4
	96	-20.7/37.4	-18.5/25.1	-19.8/21.1	-18.4/17.8	-23.3/20.9
	168	-21.4/29.5	-22.4/51.2	-20.9/23.1	-22.1/19.8	-23.0/33.3

TABLE III : Tail distributions of forecast errors. Signed percentage-error tails

P_{0.5}/P_{99.5}

for each model across grid operators and prediction windows. Negative

P_{0.5}

indicates under-prediction; positive

P_{99.5}

indicates over-prediction. PwrM.: PowerMamba; iTrnsf.: iTransformer; PTST: PatchTST.

Model	Params	MAPE (%)	MSE (%)	MSE Wins	MAPE Wins
PowerMamba	4.0M	5.72	7.62	7/30	7/30
S-Mamba	2.0M	5.80	7.58	9/30	7/30
iTransformer	6.5M	6.14	7.95	2/30	1/30
PatchTST	2.0M	5.59	7.53	12/30	15/30
LSTM	2.6M	6.83	9.13	0/30	0/30

TABLE IV : Benchmark scorecard. MAPE (%) and MSE (%) averaged across the 30

(\mathrm{ISO},W)

rows. Row-level wins counted among the five main architectures.

	iTransformer		PatchTST		PowerMamba		S-Mamba		LSTM
Grid	Load	Weather	Load	Weather	Load	Weather	Load	Weather	Load	Weather
CAISO	5.16	3.19	3.12	3.20	3.38	3.51	3.17	3.62	4.08	3.71
ISO-NE	6.41	5.01	6.06	4.41	5.86	3.62	5.95	3.63	6.74	6.20
MISO	2.88	2.25	2.37	2.31	2.38	2.11	2.33	2.16	2.97	2.62
PJM	3.46	2.73	3.21	3.04	3.14	2.78	2.97	2.73	3.59	3.33
SWPP	5.94	3.32	3.47	3.21	3.50	3.18	3.43	3.11	4.37	3.70

TABLE V : Weather integration results: Load-only vs. Weather-integrated MAPE (%) across 7 US grids (

W=24

). Bold marks the best MAPE per grid within each condition. ‘Load’ uses historical load only; ‘Weather’ adds meteorological covariates. All models use architecture-specific weather fusion (Section IV-C ).

Grid	iTransformer	PatchTST	PowerMamba	S-Mamba	LSTM
CAISO	-1.97	+0.08	+0.13	+0.45	-0.37
ISO-NE	-1.40	-1.65	-2.24	-2.32	-0.54
MISO	-0.63	-0.06	-0.27	-0.17	-0.35
PJM	-0.73	-0.17	-0.36	-0.24	-0.26
SWPP	-2.62	-0.26	-0.32	-0.32	-0.67
ERCOT	-2.87	-0.80	-0.89	-1.21	-0.03

TABLE VI : Weather integration impact (

\Delta

MAPE, percentage points). Negative values indicate improvement with weather covariates. Bold marks the largest improvement per grid.

	Tier 1 ( $d=512$ )			Tier 2 ( $d=256$ )
Grid	iTransformer	PatchTST L	$\Delta$ Gap	iTransformer S	PatchTST	$\Delta$ Gap
	(6.44M)	(9.84M)		(2.44M)	(1.77M)
CAISO	-1.97	+0.20	-2.17	-0.75	+0.08	-0.83
ISO-NE	-1.40	-1.62	+0.22	-0.71	-1.65	+0.94
MISO	-0.63	+0.13	-0.76	-2.49	-0.06	-2.43
PJM	-0.73	-0.02	-0.71	-2.28	-0.17	-2.11

TABLE VII : Parameter-controlled weather comparison. MAPE (%) for baseline (None) and weather-integrated (Wx) variants at matched

d_{\text{model}}

sizes.

\Delta

is Weather

-

None (pp).

Fig. 2 : Weather integration improves accuracy and narrows the error range, with stronger effects on weather-sensitive grids. The plot illustrates the change in MAPE (left) and Error Range (

P_{99.5}-P_{0.5}

, right) for five neural architectures when transitioning from load-only to weather-integrated forecasting (

W=24

). Negative values indicate improvement. Grids are ordered by descending temperature variability (

\sigma_{T}

). High-variability grids (e.g., ISO-NE, ERCOT) exhibit substantially larger improvements in both mean accuracy and tail extremes compared to mild grids (e.g., CAISO).

Signal Type	PatchTST	S-Mamba	PowerMamba	iTransformer	LSTM
Solar	23.1	31.8	30.6	68.8	38.5
Wind	36.2	35.4	35.7	36.4	37.2
Price	30.9	27.5	34.1	60.4	37.5
Ancillary Svc.	79.3	119.9	139.1	597.2	131.5
nMAE Wins	11/18	5/18	2/18	0/18	0/18

TABLE VIII : Multitype forecasting scorecard: average nMAE (%) by signal type and model (no weather,

W=24

). Bold marks the best model per row. Win counts reflect per-grid nMAE victories.

\dagger

CAISO partial: 14 EIA-CISO logs lack nMAE (older metrics code).

Forecasting Benchmarks

Forecasting Benchmarks are datasets and evaluation protocols for comparing predictions of future time-series values, events, and outcomes.

See all 224 papers

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods
Jul 17, 2026Matthias Hertel, Sebastian Pütz, Jonathan Kolar +3Time Series Forecasting Forecasting Benchmarks
Karlsruhe Institute of Technology, Germany · Helmholtz AI, Germany
An Empirical Benchmark of Deep Time-Series Models for Smart Meter Energy Forecasting
Aug 19, 2026Behnaz Kavoosighafi, Maria Eidenskog, Wiktoria Glad +1Time Series Forecasting Forecasting Benchmarks
Linköping University, Department of Science and Technology, Norrköping, 60174, Östergötland, Sweden · Linköping University, Department of Thematic Studies, Linköping, 58183, Östergötland, Sweden
From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning
Feb 21, 2026Suhasnadh Reddy Veluru, Sai Teja Erukude, Viswa Chaitanya MarellaElectricity Load Forecasting Time Series Forecasting
Department of Computer Science and Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India · Department of Electronics and Communication Engineering, Bharat Institute of Engineering and Technology, Hyderabad, Telangana, India

Jul 17, 2026J/K move · Enter open · S save

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

Matthias Hertel, Sebastian Pütz, Jonathan Kolar, Benjamin Schäfer+2

Karlsruhe Institute of Technology, Germany · Helmholtz AI, Germany

Time Series Forecasting Forecasting Benchmarks Electricity Load Forecasting Time Series Transformers Time-Series Foundation Models Power Systems

Open paper AskPDF arXiv

Accurate load forecasting at multiple grid levels is essential for future smart grids, ranging from aggregated control area forecasts for balancing supply and demand to forecasts of individual end-consumer loads for demand-side management and energy management systems. We present a comprehensive benchmark for load forecasting across grid levels, comprising three datasets that represent a transmission system operator control area, low-voltage grid feeders, and individual end consumers. We evaluate ten methods for short-term load forecasting and find that Transformer-based approaches consistently outperform established methods, reducing forecast error by 6.6-10.7 %. To analyze the impact of architectural design, we introduce YAformer, a flexible Transformer architecture that integrates modifications from prior work and is optimized via hyperparameter optimization. However, the standard Transformer achieves superior performance, suggesting that these architectural modifications are not required for accurate load forecasting. We further evaluate the Transformer-based time-series foundation model Chronos-2, which demonstrates competitive zero-shot performance on two datasets but fails to accurately capture special events in the TSO data. Detailed analyses reveal model-specific strengths and weaknesses, and ablation studies highlight the importance of long input contexts, covariates and continuous retraining - aspects that are often overlooked in the time-series forecasting literature.

Abstract

Forecasting Benchmarks

Explore similar work

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

An Empirical Benchmark of Deep Time-Series Models for Smart Meter Energy Forecasting

From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning