Safe Decentralized Operation of EV Virtual Power Plant with Limited Network Visibility via Multi-Agent Reinforcement Learning

Chenghao Huang; Jiarong Fan; Weiqing Wang; Hao Wang

eess.SYMar 24, 2026

Safe Decentralized Operation of EV Virtual Power Plant with Limited Network Visibility via Multi-Agent Reinforcement Learning

Authors: Chenghao Huang, Jiarong Fan, Weiqing Wang, Hao Wang

Organizations: Department of Data Science and AI, Faculty of IT, Monash University, Australia · Monash Energy Institute, Monash University, Australia

Abstract

As power systems advance toward net-zero targets, behind-the-meter renewables are driving rapid growth in distributed energy resources (DERs). Virtual power plants (VPPs) increasingly coordinate these resources to support power distribution network (PDN) operation, with EV charging stations (EVCSs) emerging as a key asset due to their strong impact on local voltages. However, in practice, VPPs must make operational decisions with only partial visibility of PDN states, relying on limited, aggregated information shared by the distribution system operator. This work proposes a safety-enhanced VPP framework for coordinating multiple EVCSs under such realistic information constraints to ensure voltage security while maintaining economic operation. We develop Transformer-assisted Lagrangian Multi-Agent Proximal Policy Optimization (TL-MAPPO), in which EVCS agents learn decentralized charging policies via centralized training with Lagrangian regularization to enforce voltage and demand-satisfaction constraints. A transformer-based embedding layer deployed on each EVCS agent captures temporal correlations among prices, loads, and charging demand to improve decision quality. Experiments on a realistic 33-bus PDN show that the proposed framework reduces voltage violations by approximately 45% and operational costs by approximately 10% compared to representative multi-agent DRL baselines, highlighting its potential for practical VPP deployment.

Figures & tables

Fig. 1: System model of decentralized EVCS coordination under a VPP’s centralized training in a PDN.

Fig. 2: The developed TL-MAPPO for safe EVCS coordination under partial PDN visibility, primarily built on MARL and Transformer.

Method	Energy cost ($)	Cycling overhead	Avg. volt. vio. (p.u./5 min)	Avg. demand dissat. (kWh/EV)
MATD3	144.2 $\pm$ 3.8	118.7 $\pm$ 2.5%	7.8 $\pm$ 0.6 $\times 10^{-3}$	0.88 $\pm$ 0.07
MAPPO	140.6 $\pm$ 4.1	122.4 $\pm$ 3.2%	6.1 $\pm$ 0.5 $\times 10^{-3}$	0.76 $\pm$ 0.06
MASAC	148.9 $\pm$ 3.9	126.5 $\pm$ 2.9%	9.3 $\pm$ 0.8 $\times 10^{-3}$	1.12 $\pm$ 0.09
TL-MAPPO	133.5 $\pm$ 3.4	110.2 $\pm$ 2.1%	4.2 $\pm$ 0.4 $\times 10^{-3}$	0.58 $\pm$ 0.05

TABLE I: Test performance over 100 independent episodes.

Fig. 3: Learning curves of the developed TL-MAPPO and three baselines.

Panel of Figure 5 — Fig. 4: Voltage statistical distributions of buses in one day of the compared methods.

Panel of Figure 6 — Fig. 5: EVCS-level comparison of (a) charging power rate and (b) voltage.

Hyperparameter	Value
Number of layers	6
Hidden dimension	256
Attention heads	4
Dropout	0.1
Optimizer	Adam
Learning rate	$1\times 10^{-4}$

TABLE II: Transformer Hyperparameters

\lambda\leftarrow\Big[\lambda+\eta_{\lambda}\big(\hat{C}-\bar{C}\big)\Big]_{+},\quad\hat{C}=\mathbb{E}_{t\in\mathcal{D}}[c_{t}].

Algorithm 1 TL-MAPPO: Transformer-assisted Lagrangian MAPPO for Safe EVCS Coordination

Method	Energy cost ($)	Cycling overhead (%)	Avg. volt. vio. (p.u./5 min)	Avg. demand dissat. (kWh/EV)
Lag-MAPPO (no temporal encoding)	136.1 $\pm$ 3.6	112.8 $\pm$ 2.3	4.9 $\pm$ 0.4 $\times 10^{-3}$	0.63 $\pm$ 0.05
Lag-MAPPO + LSTM	135.2 $\pm$ 3.7	111.9 $\pm$ 2.4	4.6 $\pm$ 0.4 $\times 10^{-3}$	0.61 $\pm$ 0.05
TL-MAPPO (Transformer)	133.5 $\pm$ 3.4	110.2 $\pm$ 2.1	4.2 $\pm$ 0.4 $\times 10^{-3}$	0.58 $\pm$ 0.05

TABLE III: Ablation Study on Temporal Encoding in Lag-MAPPO

Power Systems

Power systems research studies the planning, operation, control, and protection of electrical grids and their integration with energy resources.

See all 229 papers

Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches
Jun 30, 2026Xavier Rate, Eloann Le Guern, Raphaël Féraud +8Multi-Agent Coordination Electric Vehicle Charging
Orange · ENS Rennes · IMT Atlantique +1
LOOP-MAC: Learning-Based Multi-Agent Coordination with Guaranteed Local Feasibility in a Virtual Power Plant
Oct 27, 2023Meiyi Li, Lucas Ruda, Javad MohammadiMulti-Agent Coordination Power Systems
Emission-Aware Reinforcement Learning for Sustainable Electric Vehicle Charging and Carbon Dioxide Reduction Under Varying Renewable Penetration
May 23, 2026Ninglin Ou, Mohammad A. Razzaque, Iftekher Islam Shovon +5RL Control Multi-Objective Reinforcement Learning
anasc Research, School of Computer Science & IT, University College Cork, IE · School of Computing, Engineering and Digital Technologies, Teesside University, UK · International Energy Research Centre, Tyndall National Institute, Cork, IE +1

Jun 30, 2026J/K move · Enter open · S save

Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches

Xavier Rate, Eloann Le Guern, Raphaël Féraud, Fatma Salem+7

Orange · ENS Rennes · IMT Atlantique+1

Multi-Agent Coordination Electric Vehicle Charging Policy Gradient Methods Contextual Bandits Decentralized MARL Power Systems

Open paper AskPDF arXiv

The electrification of transportation through electric vehicles introduces new challenges for power grid management, such as increased peak demand, voltage fluctuations, line overloads, and the integration of variable renewable energy sources. To enable efficient integration of EVs while minimizing costs for users and avoiding network overloads, implicit coordination between EVs is required. This work compares two independent multi-agent reinforcement learning approaches for optimizing such decentralized EV charging: contextual combinatorial bandits and policy gradient algorithms. Using a realistic simulation environment with autonomous agents making decisions based on local environmental information (including price signals, state-of-charge, and temporal constraints), we evaluate their performance across varying congestion levels, and mixed-strategy configurations with heterogeneous agent groups under dynamic electricity pricing derived from real photovoltaic production data.

Abstract

Power Systems

Explore similar work

Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches

LOOP-MAC: Learning-Based Multi-Agent Coordination with Guaranteed Local Feasibility in a Virtual Power Plant

Emission-Aware Reinforcement Learning for Sustainable Electric Vehicle Charging and Carbon Dioxide Reduction Under Varying Renewable Penetration