A Diagnostic Protocol for GNNs in Electricity Load Forecasting: Separating Parameter Sharing from Neighborhood Access
Organizations: CB · LMO
Abstract
A graph neural network (GNN) forecasting a collection of load series combines two mechanisms, sharing parameters across series and reading neighboring series, and a gain over a non-graph model may come from either; telling them apart decides whether a graph is worth maintaining. We propose a model diagnostic protocol that compares the GNN with three reference models: a per-node network, a network shared across nodes without neighbors, and the same GNN without edges. Degree-preserving rewiring tests whether neighbor identity matters. After a tuned benchmark on regional load in France and on low-voltage feeders in the UK, we apply the protocol to deliberately short node histories, as when non-stationary demand leaves only recent observations relevant: the feeders, within one week, and a public dataset of client consumption series. On the feeders, sharing parameters recovers most of the error gap between node-specific models and GNNs, about two thirds against a regularized one. Adding the edges to an otherwise identical backbone brings a smaller gain, half or more of which disappears under rewiring. On the client series, the neighbor mean degrades the per-node model on the shorter window and the edges' gain is not consistently distinguishable from zero; on the regions of France, where each node's own history is already informative, the ranking between per-node models and GNNs changes with tuning. A synthetic study points to noisy local measurements as one regime where neighbors help. In an exploratory online aggregation, graph and non-graph forecasters appear complementary, and the weight assigned to GNNs is not proportional to their contribution. In none of the three datasets do the edges account for most of a GNN's advantage over node-specific models, so a forecasting gain should be credited to the graph only after comparison with per-node and shared references.
Figures & tables
| Block | Datasets | Training window | Models | Fixed or distinctive settings |
| Architecture benchmark (§ 6.1 ; Q1) | regions-fr ; weave-uk | d; d | tuned GNNs, non-graph baselines, GNN and shared-MLP seed ensembles, and GNN mixtures | Split and horizon within each dataset; inputs as specified |
| Per-node reference (§ 6.2 ; Q1, Q4) | regions-fr | – d | Per-node MLP ( neighbor mean) and all-node MLP; identity-graph and SAGE GNN | SAGE and hidden width |
| Synthetic regimes (§ 6.3 ; Q4) | Synthetic denoising data | Training fraction varied | Same 4 node-wise models | Base generator, one factor varied |
| Larger datasets (§ 6.4 ; Q2–Q4) | electricity ; weave-uk | / d; h/ d | Per-node, shared, all-node MLP; per-node and shared MLP neighbors; SAGE/ChebConv GNN, empty-graph GNN, rewired graphs | Hidden width |
| Complementarity (§ 6.5 ; exploratory) | regions-fr ; synthetic data | Jan –Jun | GNNs, feedforward network, additive model, ML-Poly mixture | Own split, seed per expert |
| electricity , clients (RMSE in MW) | weave-uk , feeders (RMSE in kWh) | |||||
| Model | RMSE | MAPE (%) | nRMSE | RMSE | MAPE (%) | nRMSE |
| Short window: days ( electricity ), hours ( weave-uk ) | ||||||
| Per-node MLP | ( ) | ( ) | ( ) | ( ) | ( ) | ( ) |
| Per-node MLP neighbors | ( ) | ( ) | ( ) | ( ) | ( ) | ( ) |
| Shared MLP | ( ) | ( ) | ( ) | ( ) | ( ) | ( ) |
| All-node MLP | ( ) | ( ) | ( ) | ( ) | ( ) | ( ) |
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Setting | Best expert | Uniform | ML-Poly | Without the GNN experts | Without the two others |
| regions-fr | single-step | ( ) | ( ) | |||
| seq-to-seq | ( ) | ( ) | ||||
| Synthetic structured | single-step | ( ) | ( ) | |||
| seq-to-seq | ( ) | ( ) | ||||
| Synthetic independent | single-step | ( ) | ( ) | |||
| seq-to-seq | ( ) | ( ) |
| Features | Best | Best to worst |
| Calendar and weather | ||
| temperatures of the target slots | ||
| 48 load lags | ||
| Both blocks |
| Training days | Per-node MLP | All-node MLP | Identity-graph GNN | SAGE GNN |
| ( ) | ( ) | ( ) | ( ) | ( ) |
| ( ) | ( ) | ( ) | ( ) | ( ) |
| ( ) | ( ) | ( ) | ( ) | ( ) |
| ( ) | ( ) | ( ) | ( ) | ( ) |
| ( ) | ( ) | ( ) | ( ) | ( ) |
| Window | Nodes | MLP | GNN | |||
| Per-node | Shared | All-node | SAGE | ChebConv | ||
| Per-node error, nRMSE | ||||||
| h | ( ) | ( ) | ( ) | ( ) | ( ) | |
| h | ( ) | ( ) | ( ) | ( ) | ( ) | |
| h | ( ) | ( ) | ( ) | ( ) | ( ) | |
| h | ( ) | ( ) | ( ) | ( ) | ( ) | |
| Dataset and window | Forecast change: rewired true | Edge gain | Graph diagnostic: true rewired | |||
| RMSE | MAPE points | nRMSE | nRMSE | Two-hop nodes | ||
| electricity , d | MW | |||||
| electricity , d | MW | |||||
| weave-uk , h | kWh | |||||
| weave-uk , d | kWh | |||||
| electricity | weave-uk | |||
| Model | d | d | h | d |
| Persistence | ||||
| Per-node MLP | ||||
| Per-node ridge | ||||
| Shared ridge | ||||
| Shared MLP | ||||
| electricity | weave-uk | |||
| Model | d | d | h | d |
| Per-node MLP | ||||
| Per-node MLP neighbors | ||||
| Shared MLP | ||||
| Shared MLP neighbors | ||||
| GNN | ||||
| Window end | Total gap | Sharing (share) | Neighbors, unshared | Edges, same backbone |
| Jul 2019 | ( ) | |||
| Jun 2019 | ( ) | |||
| May 2019 | ( ) | |||
| Apr 2019 | ( ) | |||
| Mar 2019 | ( ) |
| Edges kept | Count | Highest attention | Random | Lowest attention |
| All edges | ||||
| One half | ( ) | ( ) | ( ) | |
| One quarter | ( ) | ( ) | ( ) | |
| One tenth | ( ) | ( ) | ( ) | |
| No neighbors | ( ) | ( ) | ( ) |
| Dataset | Model | Graph | Batch size | Layers | Width | Learning rate |
| regions-fr | GCN | Space | ||||
| SAGE | DTW | |||||
| GAT | Space | |||||
| GATv2 | Space | |||||
| Transformer | Correlation | |||||
| TAG | Space |
Graph Neural Networks
Graph Neural Networks learn from graph-structured data by combining information across connected nodes, edges, and larger structures.
See all 735 papers