Reinforcement Learning for Power-Flow Network Analysis

Alperen Ergur; Julia Lindberg; Vinny Miller

cs.LGMar 5, 2026

Reinforcement Learning for Power-Flow Network Analysis

Authors: Alperen Ergur, Julia Lindberg, Vinny Miller

Organizations: The University of Texas at San Antonio, Mathematics and Computer Science Departments · Georgia Tech, Mathematics Department · The University of Texas at San Antonio, Computer Science Department

Abstract

The power flow equations are non-linear multivariate equations that describe the relationship between power injections and bus voltages of electric power networks. Given a network topology, we are interested in finding network parameters with many equilibrium points. This corresponds to finding instances of the power flow equations with many real solutions. Current state-of-the art algorithms in computational algebra are not capable of answering this question for networks involving more than a small number of variables. To remedy this, we design a probabilistic reward function that gives a good approximation to this root count, and a state-space that mimics the space of power flow equations. We derive the average root count for a Gaussian model, and use this as a baseline for our RL agents. The agents discover instances of the power flow equations with many more solutions than the average baseline. This demonstrates the potential of RL for power-flow network design and analysis as well as the potential for RL to contribute meaningfully to problems that involve complex non-linear algebra or geometry. \footnote{Author order alphabetic, all authors contributed equally.

Figures & tables

	Time Taken (s)		Trace Distance		Summation Distance
System Size	BFGS	Newton-CG	BFGS	Newton-CG	BFGS	Newton-CG	Success Rate
50	0.0079	0.11	1.3e-5	8.9e-6	5.0e-14	5.0e-14	0.3
150	0.37	10	7.6e-7	5.8e-7	1.6e-13	1.6e-13	0.08
250	2.3	82	8.8e-6	7.5e-6	2.5e-13	2.5e-13	0.2

Table 1: Accuracy improvements made by using a Newton-conjugate corrector for 5 steps. The same systems used in Figure 1(c) were used here. Only those systems that were successfully corrected were averaged over.

Panel of Figure 2 — Figure 2: $L=10,15,20$ agent test runs

Table 4

Solution Count	Random	$L=10$	$L=15$	$L=20$
80	12.0	7.0	6.0	1.0
90	15.5	3.5	9.0	2.0
100	N/A	14.0	11.0	3.5

Table 4: Median number of steps in test runs the agents took to exceed solution counts of interest

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Reward function comparison with $N=100000$ , $M=2500$

Network	Architecture
Actor $\pi_{\phi}(s)$	Fully Connected: $D\rightarrow 500\rightarrow 400\rightarrow 300\rightarrow D$ , ReLU activations.
Critics $Q_{\theta}(s,a)$	Fully Connected: $D\rightarrow 500\rightarrow 400\rightarrow 300\rightarrow 1$ , ReLU activations.

Appendix

Table 5: Neural network architectures used where

D=10\times 10\times 10

Panel of Figure 4 — Figure 4: Training plots for the $L=10,15,20$ matrix formulation agents.

Power Systems

Power systems research studies the planning, operation, control, and protection of electrical grids and their integration with energy resources.

See all 231 papers

Reinforcement Learning to Initialize Newton-Raphson for AC Power Flow with Quantum Annealing-Based Environment Updates
Nov 25, 2025Zeynab Kaseb, Matthias Moller, Lindsay Spoor +4Reinforcement Learning Quantum Annealing
Electrical Sustainable Energy, Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands. · Applied Mathematics, Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands. · Leiden Institute of Advanced Computer Science, Leiden University, P.O. Box 9512, 2300 RA Leiden, The Netherlands. +3
Newton's Lantern: A Reinforcement Learning Framework for Finetuning AC Power Flow Warm Start Models
May 11, 2026Shourya Bose, Helgi Hilmarsson, Dhruv SuriRL Fine-Tuning Power Systems
Pravah
PowerModelsGAT-AI: Physics-Informed Graph Attention for Multi-System Power Flow with Continual Learning
Feb 24, 2026Chidozie Ezeakunne, Jose E. Tabarez, Reeju Pokharel +1Physics-Informed GNNs Power Systems
Los Alamos National Laboratory, Los Alamos, NM, USA · Department of Physics, University of Central Florida, Orlando, FL 32816 USA

Nov 25, 2025J/K move · Enter open · S save

Reinforcement Learning to Initialize Newton-Raphson for AC Power Flow with Quantum Annealing-Based Environment Updates

Zeynab Kaseb, Matthias Moller, Lindsay Spoor, Jerry J. Guo+3

Electrical Sustainable Energy, Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands. · Applied Mathematics, Delft University of Technology, P.O. Box 5031, 2600 GA Delft, The Netherlands. · Leiden Institute of Advanced Computer Science, Leiden University, P.O. Box 9512, 2300 RA Leiden, The Netherlands.+3

Reinforcement Learning Quantum Annealing Learning to Optimize Power Systems

Open paper AskPDF arXiv

The Newton-Raphson (NR) method is widely used for solving power flow (PF) equations due to its quadratic convergence. However, its performance deteriorates under poor initialization or extreme operating scenarios, e.g., high levels of renewable energy penetration. We propose the use of reinforcement learning (RL) to optimize the initialization of NR, and introduce a quantum-enhanced RL environment update mechanism that addresses the combinatorially large action space at each RL timestep by formulating the voltage adjustment task as a Quadratic Unconstrained Binary Optimization (QUBO) problem, solved with an Ising machine. RL initialization is benchmarked against flat start and start from the DC (linearized) PF solution on a standard 4-bus system, Iwamoto's ill-conditioned 11-bus system, and the IEEE 118-bus system under normal and stressed loading and reactive power limits, with verified operational solutions. On all systems, a supervised initializer refined by RL requires fewer NR iterations than flat and DC starts and than the same initializer without RL, for all seeds. For example, on the 118-bus system under normal and stressed loading, it reached 2.04 and 2.86 NR iterations, compared with 3.02 and 5.13 from DC start and 2.61 and 3.09 without RL. In wall-clock time, this pays off only for an initializer integrated into the solver and reused for many solves on a fixed topology. On the 4-bus system, a quantum-enhanced RL agent with a quantum-inspired annealer moved challenging initial states that required 29 and 44 NR iterations to initializations that required three NR iterations within one RL timestep.

Abstract

Power Systems

Explore similar work

Reinforcement Learning to Initialize Newton-Raphson for AC Power Flow with Quantum Annealing-Based Environment Updates

Newton's Lantern: A Reinforcement Learning Framework for Finetuning AC Power Flow Warm Start Models

PowerModelsGAT-AI: Physics-Informed Graph Attention for Multi-System Power Flow with Continual Learning