An Empirical Benchmark of Deep Time-Series Models for Smart Meter Energy Forecasting

Behnaz Kavoosighafi; Maria Eidenskog; Wiktoria Glad; Katerina Vrotsou

cs.LGAug 19, 2026

An Empirical Benchmark of Deep Time-Series Models for Smart Meter Energy Forecasting

Authors: Behnaz Kavoosighafi, Maria Eidenskog, Wiktoria Glad, Katerina Vrotsou

Organizations: Linköping University, Department of Science and Technology, Norrköping, 60174, Östergötland, Sweden · Linköping University, Department of Thematic Studies, Linköping, 58183, Östergötland, Sweden

Abstract

Accurate forecasting of energy consumption is important for the efficient operation of power systems, with direct implications for operational costs, energy management, and system maintenance. Due to the availability of extensive high-resolution consumption data from smart meters, data-driven methods have been used for short-term and long-term forecasting. However, their comparative performance on real-world smart meter data is still not well studied. In this paper, we present an empirical benchmark of nine modern deep learning models for time-series forecasting, including linear, MLP-based, convolutional, and Transformer architectures. We evaluate these models on two publicly available smart meter datasets. Our analysis focuses on three factors that strongly affect forecasting performance: the length of historical input, the prediction horizon, and the choice of model architecture. We show that extending the historical context improves accuracy, but only up to a saturation point, after which additional input provides limited benefit. In contrast, accuracy decreases as the prediction horizon increases. We also investigate the trade-off between prediction accuracy and computational complexity, and assess the statistical significance and practical magnitude of performance differences across models. Our results show that deep learning models consistently outperform classical baselines, while lightweight architectures achieve relatively similar performance at significantly lower computational cost. Additionally, architectural differences only become meaningful at longer forecasting horizons and on more heterogeneous datasets. Finally, a subgroup analysis across geodemographic and household categories shows that model choice has limited impact for most population segments.

Figures & tables

Panel of Figure 1 — Figure 1: Hourly electricity consumption patterns in the LCL and Pecan Street datasets. (a) Population-level hourly envelope for the LCL dataset, showing the median and 25–75% and 5–95% quantile bands across households. (b) Population-level hourly envelope for the Pecan Street dataset, showing the median and 25–75% and 5–95% quantile bands across households over the period 2014–2017. (c) Median intraday profiles by ACORN socioeconomic group for the LCL dataset. (d) Median intraday profiles by building type for the Pecan Street dataset.

Model	Architecture	Core Mechanism	Complexity
DLinear [ 40 ]	Linear model	Series decomposition	Very low
LightTS [ 41 ]	MLP-based	Interval and continuous sampling	Low
TSMixer [ 15 ]	MLP-based	Time-channel mixing	Low
FiLM [ 43 ]	Memory-based	Frequency-domain denoising	Medium
SCINet [ 27 ]	CNN-based	Recursive downsampling	Medium
TiDE [ 12 ]	MLP-based	Residual encoder-decoder	Medium

Table 1: Comparison of forecasting models used in our benchmark.

Model	MSE $\downarrow$	MAE $\downarrow$	MAPE $\downarrow$	$R^{2}$ $\uparrow$	$r$ $\uparrow$	$\cos$ $\uparrow$	FD $\downarrow$
Classical baselines
Naïve seasonal	0.240	0.241	0.721	0.391	0.697	0.792	0.712
SARIMA	0.196	0.246	0.991	0.504	0.739	0.820	0.631
Deep learning models
DLinear	0.177	0.227	1.310	0.553	0.744	0.857	0.960
LightTS	0.166	0.216	1.226	0.579	0.761	0.863	0.982

Table 2: Forecast accuracy of the deep learning models and classical baselines on the LCL dataset at

\texttt{seq\_len}=24

and

\texttt{pred\_len}=24

. The best value in each column is bold .

Panel of Table 3 — Table 3: Best-performing forecasting model (lowest composite score) for each combination of input sequence length ( seq_len ) and prediction horizon ( pred_len ) for the LCL dataset.

Panel of Table 4 — Table 4: Best-performing forecasting model (lowest composite score) for each combination of input sequence length ( seq_len ) and prediction horizon ( pred_len ) for the Pecan Street dataset.

$s$	Model	MSE $\downarrow$	MAE $\downarrow$	MAPE $\downarrow$	$R^{2}$ $\uparrow$	$r$ $\uparrow$	$\cos$ $\uparrow$	FD $\downarrow$	$tt$ $\downarrow$	$it$ $\downarrow$
24	DLinear	0.177	0.227	1.310	0.553	0.744	0.857	0.960	21.82	0.24
	LightTS	0.166	0.216	1.226	0.579	0.761	0.863	0.982	36.51	0.80
	TSMixer	0.168	0.217	1.203	0.576	0.760	0.861	0.988	29.75	0.98
	FiLM	0.177	0.221	0.863	0.551	0.743	0.857	0.969	198.96	7.46
	SCINet	0.174	0.218	0.854	0.560	0.749	0.858	0.974	307.76	12.11
	TiDE	0.177	0.221	0.863	0.551	0.743	0.857	0.968	43.35	1.32

Table 5: Forecasting performance on the LCL dataset across different historical context lengths (

s

, in hours) for pred_len = 24. MAPE values are reported in units of

\times 10^{3}

. Here,

r

denotes the Pearson correlation coefficient,

\cos

the cosine similarity,

FD

the Fréchet distance, and

tt

and

it

the training and inference time in minutes and milliseconds, respectively. The best result per context length is highlighted in bold , and the overall best result is emphasized in bold and underlined .

$s$	Model	MSE $\downarrow$	MAE $\downarrow$	MAPE $\downarrow$	$R^{2}$ $\uparrow$	$r$ $\uparrow$	$\cos$ $\uparrow$	FD $\downarrow$	$tt$ $\downarrow$	$it$ $\downarrow$
24	DLinear	0.192	0.241	1.603	0.520	0.722	0.827	1.254	21.88	0.24
	LightTS	0.180	0.231	1.733	0.550	0.742	0.835	1.253	37.09	0.80
	TSMixer	0.181	0.230	1.530	0.546	0.740	0.835	1.265	31.23	0.97
	FiLM	0.193	0.233	0.961	0.516	0.720	0.826	1.262	196.28	7.47
	SCINet	0.189	0.229	0.968	0.526	0.727	0.833	1.251	310.05	12.19
	TiDE	0.193	0.233	0.963	0.517	0.720	0.826	1.260	43.24	1.33

Table 6: Forecasting performance on the LCL dataset across different historical context lengths (

s

, in hours) for pred_len = 96. MAPE values are reported in units of

\times 10^{3}

. Here,

r

denotes the Pearson correlation coefficient,

\cos

the cosine similarity,

FD

the Fréchet distance, and

tt

and

it

$s$	Model	MSE $\downarrow$	MAE $\downarrow$	MAPE $\downarrow$	$R^{2}$ $\uparrow$	$r$ $\uparrow$	$\cos$ $\uparrow$	FD $\downarrow$	$tt$ $\downarrow$	$it$ $\downarrow$
24	DLinear	0.195	0.244	1.687	0.517	0.719	0.824	1.443	22.06	0.24
	LightTS	0.183	0.232	1.650	0.546	0.739	0.832	1.453	36.81	0.79
	TSMixer	0.185	0.233	1.627	0.540	0.736	0.832	1.480	30.84	0.98
	FiLM	0.196	0.235	0.982	0.513	0.718	0.825	1.448	195.83	7.47
	SCINet	0.192	0.232	1.006	0.523	0.725	0.829	1.456	309.90	12.11
	TiDE	0.196	0.235	0.982	0.513	0.718	0.825	1.450	44.40	1.31

Table 7: Forecasting performance on the LCL dataset across different historical context lengths (

s

, in hours) for pred_len = 168. MAPE values are reported in units of

\times 10^{3}

. Here,

r

denotes the Pearson correlation coefficient,

\cos

the cosine similarity,

FD

the Fréchet distance, and

tt

and

it

Explore similar work

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods
Jul 17, 2026Matthias Hertel, Sebastian Pütz, Jonathan Kolar +3Time Series Forecasting Forecasting Benchmarks
Karlsruhe Institute of Technology, Germany · Helmholtz AI, Germany
Deep Learning for Cross-Border Electricity Price Forecasting: A Comparative Study
Aug 17, 2026Hadeer Elashhab, Sai Srijan Papineni, Marvin Dorn +2Electricity Price Forecasting Time Series Forecasting
Institute for Automation and Applied Informatics, Karlsruhe Institute of Technology, Karlsruhe, Germany · Department of Computer Science, Karlsruhe Institute of Technology, Karlsruhe, Germany
FETS Benchmark: Foundation Models Outperform Dataset-specific Machine Learning in Energy Time Series Forecasting
Apr 24, 2026Marco Obermeier, Marco Pruckner, Florian Haselbeck +1Time Series Forecasting Forecasting Benchmarks
Julius-Maximilians-Universität Würzburg, Modeling and Simulation Lab, Am Hubland, Würzburg, 97074, Germany · Weihenstephan-Triesdorf University of Applied Sciences, Smart Farming, Am Staudengarten 1, Freising, 85354, Germany · Weihenstephan-Triesdorf University of Applied Sciences, Digital Energy Transition, Hans-Carl-von-Carlowitz-Platz 3, Freising, 85354, Germany

Jul 17, 2026J/K move · Enter open · S save

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

Matthias Hertel, Sebastian Pütz, Jonathan Kolar, Benjamin Schäfer+2

Karlsruhe Institute of Technology, Germany · Helmholtz AI, Germany

Time Series Forecasting Forecasting Benchmarks Electricity Load Forecasting Time Series Transformers Time-Series Foundation Models Power Systems

Open paper AskPDF arXiv

Accurate load forecasting at multiple grid levels is essential for future smart grids, ranging from aggregated control area forecasts for balancing supply and demand to forecasts of individual end-consumer loads for demand-side management and energy management systems. We present a comprehensive benchmark for load forecasting across grid levels, comprising three datasets that represent a transmission system operator control area, low-voltage grid feeders, and individual end consumers. We evaluate ten methods for short-term load forecasting and find that Transformer-based approaches consistently outperform established methods, reducing forecast error by 6.6-10.7 %. To analyze the impact of architectural design, we introduce YAformer, a flexible Transformer architecture that integrates modifications from prior work and is optimized via hyperparameter optimization. However, the standard Transformer achieves superior performance, suggesting that these architectural modifications are not required for accurate load forecasting. We further evaluate the Transformer-based time-series foundation model Chronos-2, which demonstrates competitive zero-shot performance on two datasets but fails to accurately capture special events in the TSO data. Detailed analyses reveal model-specific strengths and weaknesses, and ablation studies highlight the importance of long input contexts, covariates and continuous retraining - aspects that are often overlooked in the time-series forecasting literature.

Abstract

Explore similar work

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

Deep Learning for Cross-Border Electricity Price Forecasting: A Comparative Study

FETS Benchmark: Foundation Models Outperform Dataset-specific Machine Learning in Energy Time Series Forecasting