A Lightweight Multi-View Approach to Short-Term Load Forecasting

Julien Guité-Vinet; Alexandre Blondin Massé; Éric Beaudry

cs.LGFeb 9, 2026

A Lightweight Multi-View Approach to Short-Term Load Forecasting

Authors: Julien Guité-Vinet, Alexandre Blondin Massé, Éric Beaudry

Organizations: Université du Québec à Montréal, Département d’informatique, Montréal, Canada

Abstract

Time series forecasting is a critical task across domains such as energy, finance, and meteorology, where accurate predictions enable informed decision-making. While transformer-based and large-parameter models have recently achieved state-of-the-art results, their complexity can lead to overfitting and unstable forecasts, especially when older data points become less relevant. In this paper, we propose a lightweight multi-view approach to short-term load forecasting that leverages single-value embeddings and a scaled time-range input to capture temporally relevant features efficiently. We introduce an embedding dropout mechanism to prevent over-reliance on specific features and enhance interpretability. Our method achieves competitive performance with significantly fewer parameters, demonstrating robustness across multiple datasets, including scenarios with noisy or sparse data, and provides insights into the contributions of individual features to the forecast.

Figures & tables

Problem formulation		Embedding
$\mathcal{Y}$	target load series	$d$	embedding dimension of a single value
$\mathcal{X}$	matrix stacking the $F$ views	$d_{\mathrm{model}}$	width of the aggregated representation
$\mathcal{M}_{\theta}$	forecasting model of parameters $\theta$	$W_{f}$ , $b_{f}$	projection and bias of the view $f$
$\mathcal{J}$	loss function	$w_{f}$	scalar weight of the view $f$ under SVE
$\tau$	forecasting horizon	$z_{l,f}$	embedding of the entry at lag $l$ for the view $f$
$m$	context window length	$z_{f}$	embedding of the whole view $f$

Table 1: Notation used throughout the paper.

Figure 1: Single-value embedding. (a) The scaled time-range input gathers the rows of

\mathcal{X}

at the lags of

\mathcal{L}

. (b) The SVE module applies Equation 9 to every row, so that

d_{\mathrm{model}}=F

; the hatched column is a view withheld by the embedding dropout of Equation 7 , that is

r_{f}=0

. (c) The decoder derives both its keys and its values from the encoded states, whereas its queries follow Equations 11 and 12 , the loop below the decoder returning

\tilde{Y}^{\ell-1}

as the query of the next layer; the output entry attached to the null lag predicts

t+1

Dataset	Load series	Calendar	Weather source	Start	End
IESO	ieso.ca/Power-Data	CA-ON	Environment and Climate Change Canada, Toronto	2002-05-01	2024-09-30
CND	doi.org/10.17632/byx7sztj59.1	provided	provided	2015-01-03	2020-06-27
AEP	https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption	US-OH	Open-Meteo, Columbus	2004-10-01	2018-08-03

Table 2: Origin and temporal coverage of the series and covariates. Calendar features are derived from the Python holidays package, whose region code is reported here, and yield a binary holiday indicator together with a categorical holiday identifier. IESO weather features are retrieved from climate.weather.gc.ca, whereas AEP weather features are retrieved from Open-Meteo, using the location reported here.

	SVE	N-BEATS	TiDE	DLinear	PatchTST
Epochs	300
Loss	MAPE
Timestep	1 hour
Covariates	time and exogenous features
Optimiser	AdamW
Scheduler	CosineAnnealingLR

Table 3: Hyperparameters of the deep models. The upper block gathers the settings shared by every model, the lower one those specific to an architecture. A dash indicates a hyperparameter that does not apply to the architecture. The lag set

\mathcal{L}

is defined in Section 5 .

Panel of Figure 2 — Figure 2: Embeddings effects of long term, short term, temperature, weather and holidays related views on the IESO dataset for a 48 hours horizon using the SVE method. Each panel reports the forecast obtained when a single view is retained at inference, in colour, against the observed load, in black. The combined effect is shown at the bottom right. The horizontal axis is the forecast horizon in hours and the vertical axis is the load in MW.

Figure 3: Embedded encoder representations. Each row corresponds to a view and each cell to one of its discrete slots, continuous values being rounded to the nearest integer: the hour view, for instance, is discretised into twenty-four slots, the first of which carries the weight assigned to midnight. Leftmost cells represent the lowest categorical values (e.g., Spring to Winter for season). Cell intensity encodes the learned embedding weight on a diverging scale centred on zero. The weighting vector $W$ gives, for each view, the weight it carries in the forecast.

	Embeddings methods
		additive			concatenative			SVE
num params		93k			143k			33,605
horizon		1d	2d	7d	1d	2d	7d	1d	2d	7d
IESO	full	4.41	4.96	5.77	5.35	5.81	6.54	4.64	5.17	6.24
	holidays	5.16	6.10	6.68	4.77	5.49	6.32	5.55	6.33	7.39
	corrupted	6.08	6.92	8.20	5.25	5.98	7.04	5.84	6.64	7.78

Table 4: MAPE scores for different embeddings combination methods. Parameter counts include every trainable weight of the network, embedding layers comprised.

Models		SVE	N-BEATS	TiDE	DLinear	PatchTST
num params		33,605	200,635	287,046	346,818	104,408
IESO	1d	4.64	4.62	3.49	2.92	4.11
	2d	5.17	5.50	4.36	3.62	4.83
	7d	6.24	7.33	5.61	4.92	5.71
CND	1d	6.65	5.04	3.94	3.69	4.53
	2d	7.37	5.48	4.58	4.32	4.89

Table 5: Multivariate time series forecasting results on three public datasets. Parameter counts follow the convention of Table 4 .

Panel of Figure 4 — Figure 4: Forecasts from N-BEATS, TiDE, DLinear, SARIMAX, PatchTST and the proposed model on the IESO test set, over a two-day horizon spanning January 1st and 2nd 2023. Each panel reports the forecast of one model, in colour, against the observed load, in black. The proposed model is shown at the bottom right. The horizontal axis is the forecast horizon in hours and the vertical axis is the load in MW. All models are given the same forecast origin and the same exogenous features, and the period covers a public holiday, when the daily profile departs from the usual weekday pattern. SARIMAX is shown for reference only and is excluded from the comparison of Table 5 .

Time Series Modeling

See all 56 papers

Feb 21, 2026

From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning

Accurate short-term power load forecasting is important to effectively manage, optimize, and ensure the robustness of modern power systems. This paper performs an empirical evaluation of a traditional statistical model and deep learning approaches for predicting short-term energy load. Four models, namely ARIMA, LSTM, BiLSTM, and Transformer, were leveraged on the PJM Hourly Energy Consumption data. The data processing involved interpolation, normalization, and a sliding-window sequence method. Each model's forecasting performance was evaluated for the 24-hour horizon using MAE, RMSE, and MAPE. Of the models tested, the Transformer model, which relies on self-attention algorithms, produced the best results with 3.8 percent of MAPE, with performance above any model in both accuracy and robustness. These findings underscore the growing potential of attention-based architectures in accurately capturing complex temporal patterns in power consumption data.

Suhasnadh Reddy Veluru, Sai Teja Erukude, Viswa Chaitanya Marella

Department of Computer Science and Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, India · Department of Electronics and Communication Engineering, Bharat Institute of Engineering and Technology, Hyderabad, Telangana, India

Electricity Load Forecasting Time Series Forecasting Time Series Transformers

Ask PDF

Jul 17, 2026

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

Accurate load forecasting at multiple grid levels is essential for future smart grids, ranging from aggregated control area forecasts for balancing supply and demand to forecasts of individual end-consumer loads for demand-side management and energy management systems. We present a comprehensive benchmark for load forecasting across grid levels, comprising three datasets that represent a transmission system operator control area, low-voltage grid feeders, and individual end consumers. We evaluate ten methods for short-term load forecasting and find that Transformer-based approaches consistently outperform established methods, reducing forecast error by 6.6-10.7 %. To analyze the impact of architectural design, we introduce YAformer, a flexible Transformer architecture that integrates modifications from prior work and is optimized via hyperparameter optimization. However, the standard Transformer achieves superior performance, suggesting that these architectural modifications are not required for accurate load forecasting. We further evaluate the Transformer-based time-series foundation model Chronos-2, which demonstrates competitive zero-shot performance on two datasets but fails to accurately capture special events in the TSO data. Detailed analyses reveal model-specific strengths and weaknesses, and ablation studies highlight the importance of long input contexts, covariates and continuous retraining - aspects that are often overlooked in the time-series forecasting literature.

Matthias Hertel, Sebastian Pütz, Jonathan Kolar +3

Karlsruhe Institute of Technology, Germany · Helmholtz AI, Germany

Time Series Forecasting Forecasting Benchmarks Electricity Load Forecasting Time Series Transformers Time-Series Foundation Models Power Systems

Ask PDF

Jun 22, 2026

Interpretable Kolmogorov-Arnold Network with Feature-Isolated Temporal Attention Mechanism for Electricity Load Forecasting

Accurate electricity load forecasting is a crucial prerequisite for stable power system operations. While prevalent deep learning models present competitive performance, they often operate as black boxes and lack interpretability. While the Kolmogorov-Arnold network (KAN) has emerged as a promising alternative because of its learnable activation function design, its direct application to time-series forecasting faces challenges in modeling complex temporal data patterns. Also, simple integration into existing architectures, such as serving as replacement of neural modules, cannot fully leverage KAN's interpretability strengths. To address these gaps, this study develops LoadKAN, a novel hybrid and interpretable framework for load forecasting that synergistically combines a specifically-designed feature-isolated temporal attention mechanism with a KAN module. The attention stage aims to extract temporal dynamics from each input feature independently, such as historical load and human mobility, providing distilled feature representations to the KAN module for interpretable predictions. When evaluated on datasets from three representative U.S. electricity markets, our LoadKAN remains highly competitive when compared to extensively-tuned, state-of-the-art, black-box deep learning benchmarks. More importantly, LoadKAN's interpretability enables a granular analysis of the learned non-linear relationships between six distinct mobility patterns and electricity load. Through KAN-learned activation functions, our quantitative sensitivity analyses on mobility features reveal complex and market-specific dependencies. These findings further demonstrate the ability of our LoadKAN to generate insights often obscured by opaque black-box neural forecasting models.

Jinhao Li, Hao Wang

Department of Data Science and AI, Faculty of Information Technology, Monash University, Melbourne, Victoria, Australia · Monash Energy Institute, Monash University, Melbourne, Victoria, Australia

Multivariate Time Series Forecasting Time Series Forecasting Neural Network Interpretability Kolmogorov-Arnold Networks Temporal Attention Electricity Load Forecasting

Ask PDF

Abstract

Time Series Modeling

Explore similar work

From ARIMA to Attention: Power Load Forecasting Using Temporal Deep Learning

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

Interpretable Kolmogorov-Arnold Network with Feature-Isolated Temporal Attention Mechanism for Electricity Load Forecasting