Extreme weather events and cyberattacks can cause component failures and disrupt the operation of power distribution networks (DNs), during which reconfiguration and load shedding are often adopted for resilience enhancement. This study introduces a topology-aware graph reinforcement learning (RL) framework for outage management that embeds higher-order topological features of the DN into a graph-based RL model, enabling reconfiguration and load shedding to maximize energy supply while maintaining operational stability. Results on the modified IEEE 123-bus feeder across 300 diverse outage scenarios demonstrate that incorporating the topological data analysis (TDA) tool, persistence homology (PH), yields 9-18% higher cumulative rewards, up to 6% increase in power delivery, and 6-8% fewer voltage violations compared to a baseline graph-RL model. These findings highlight the potential of integrating RL with TDA to enable self-healing in DNs, facilitating fast, adaptive, and automated restoration.
Figures & tables
Fig. 1 : Architecture of the policy network (PH-GCAPCN).
Fig. 2 : Training performance for the proposed RL model based on PH-GCAPCN and the baseline RL model using GCAPCN.
Metric
Test Set 1
Test Set 2
Test Set 3
PH-GCAPCN
GCAPCN
PH-GCAPCN
GCAPCN
PH-GCAPCN
GCAPCN
Reward ( R )
0.4626 ± 0.5757
0.3923 ± 0.5692
0.5132 ± 0.5540
0.4710 ± 0.5407
0.5538 ± 0.4842
0.5010 ± 0.4659
Energy Supplied ( Esupp )
0.6760 ± 0.3433
0.6374 ± 0.3265
0.7119 ± 0.3234
0.6821 ± 0.3129
0.7302 ± 0.2913
0.6921 ± 0.2769
Voltage Violation ( Vviol )
0.2134 ± 0.2330
0.2282 ± 0.2276
0.1932 ± 0.2189
0.2057 ± 0.2166
0.1763 ± 0.1942
0.1911 ± 0.1909
TABLE I : Performance of PH-GCAPCN and GCAPCN RL models across three test samples. Values are reported as mean ± std over 100 distinct out-of-sample scenarios.
Test Set
Reward
Energy Supplied
Voltage Violation
1
6.83×10−4∗∗∗
1.26×10−8∗∗∗
9.42×10−4∗∗∗
2
3.82×10−5∗∗∗
3.75×10−9∗∗∗
2.73×10−5∗∗∗
3
2.15×10−7∗∗∗
7.39×10−8∗∗∗
9.04×10−5∗∗∗
TABLE II : Paired t-test p -values comparing PH-GCAPCN and GCAPCN over 100 matched outage scenarios per test sample.
Fig. 3 : Test performance comparison of PH-GCAPCN and GCAPCN models
Self-healing smart grids can quickly adjust their network configuration during outages to minimize power disruptions. During an outage, several actions can be taken, such as network reconfiguration through switching operations and emergency load shedding. However, traditional machine learning methods for outage mitigation are not well suited for smart grids due to their slow response time and high computational cost. To address these challenges, recent studies have explored reinforcement learning to automatically perform network reconfiguration. In these approaches, the control policy is typically modeled using a graph neural network (GNN). However, conventional GNNs operate in the spatial domain and may fail to capture important relationships in the frequency domain. Frequency-domain information is particularly useful for modeling global structural patterns and system-wide interactions in power networks. In this paper, we propose a spectral graph reinforcement learning framework for outage management in distribution networks to enhance system resilience. Our model learns the optimal power restoration policy using a spectral graph neural network. We evaluate the proposed method on three modified IEEE test systems: the 13-bus, 34-bus, and 123-bus networks. Experimental results show that our approach achieves near-optimal performance in real time and generalizes well across a wide range of outage scenarios.
Optimal dispatch of energy storage systems (ESSs) in distribution networks involves jointly improving operating economy and voltage security under time-varying conditions and possible topology changes. To support fast online decision making, we develop a topology-aware Reinforcement Learning architecture based on Twin Delayed Deep Deterministic Policy Gradient (TD3), which integrates graph neural networks (GNNs) as graph feature encoders for ESS dispatch. We conduct a systematic investigation of three GNN variants: graph convolutional networks (GCNs), topology adaptive graph convolutional networks (TAGConv), and graph attention networks (GATs) on the 34-bus and 69-bus systems, and evaluate robustness under multiple topology reconfiguration cases as well as cross-system transfer between networks with different system sizes. Results show that GNN-based controllers consistently reduce the number and magnitude of voltage violations, with clearer benefits on the 69-bus system and under reconfiguration; on the 69-bus system, TD3-GCN and TD3-TAGConv also achieve lower saved cost relative to the NLP benchmark than the NN baseline. We also highlight that transfer gains are case-dependent, and zero-shot transfer between fundamentally different systems results in notable performance degradation and increased voltage magnitude violations. This work is available at: https://github.com/ShuyiGao/GNNs_RL_ESSs and https://github.com/distributionnetworksTUDelft/GNNs_RL_ESSs.
Shuyi Gao, Stavros Orfanoudakis, Shengren Hou +2
Department of Electrical Sustainable Energy, Delft University of Technology, Mekelweg 4, Delft, 2628 CD, The Netherlands · Beijing Energy Quant Ltd., Haidian, Beijing, 53100, China
As the integration of volatile renewable energy sources increases the strain on modern power grids, the use of Reinforcement Learning (RL) for autonomous topological reconfiguration has emerged as a promising research field to keep strained grids stable and operational. Compared to traditional redispatching measures, topological actions offer a cheaper and more cost-effective way to manage grid congestion. However, their implementation is hindered by a vast combinatorial action space and strict operational constraints. This paper investigates the effectiveness of model-based AlphaZero-inspired approaches that utilize Monte Carlo Tree Search (MCTS) for proactive grid management. We systematically evaluate how reward functions, observation density, and search guidance influence an agent's survivability. Our results demonstrate that the optimized AlphaZero approach achieves a peak survivability of 98.43%, significantly outperforming the proximal policy optimization (PPO) variant. We find that conducting the MCTS without guidance from a prior learned policy or value function can enhance training efficiency, and that a straightforward binary survival reward provides more effective search guidance than complex, multi-objective functions. Our findings demonstrate that while AlphaZero is a powerful framework for topological control, pure reinforcement learning is not sufficient; rather, an effective and reliable system requires a 'minimalist' integration of domain-specific heuristics, binary rewards, and a restricted observation space of line loads.
Lukas Zetto, Benjamin Schäfer, Qiong Huang
Karlsruhe Institute of Technology, Karlsruhe, Germany