LOOP-MAC: Learning-Based Multi-Agent Coordination with Guaranteed Local Feasibility in a Virtual Power Plant

Meiyi Li; Lucas Ruda; Javad Mohammadi

cs.LGOct 27, 2023

LOOP-MAC: Learning-Based Multi-Agent Coordination with Guaranteed Local Feasibility in a Virtual Power Plant

Authors: Meiyi Li, Lucas Ruda, Javad Mohammadi

Abstract

By aggregating distributed energy resources (DERs) into a single dispatchable resource, virtual power plants (VPPs) enable the grid to use the flexibility of numerous small-scale DERs. Delivering an aggregate VPP commitment requires multi-agent coordination, where agents manage local DER models and operating constraints while jointly meeting a power commitment within limited communication rounds. Classical distributed optimization is computationally intensive, requiring iterative local optimization and repeated communication to reach consensus. Learning-based methods can accelerate coordination but often lack guarantees of local feasibility and accurate commitment tracking. This paper proposes LOOP-MAC (Learning to Optimize the Optimization Process for Multi-Agent Coordination), a fully decentralized learning-based framework that ensures local feasibility by construction while efficiently reducing the aggregate commitment gap. LOOP-MAC employs a two-stage decision process. Learned coordination uses neural optimizers equipped with a gauge layer to produce locally feasible schedules through a few rounds of message exchange. Local adjustment then rebalances each schedule toward feasible flexibility extremes using a commitment-gap estimate inferred from messages already received. The adjusted schedule remains locally feasible as a convex combination of feasible schedules, and the adjustment requires no additional communication round. On a 100-agent VPP, five communication rounds followed by local adjustment reduce the mean commitment gap from 5.1% to 0.75% of the agents' aggregate absolute power and the optimality gap from 3.93% to 3.07% relative to learned coordination alone. Per-agent online computation takes approximately 1.1 ms, including the adjustment.

Figures & tables

Fig. 1: LOOP-MAC: a fully decentralized learning-based framework for VPP commitment delivery. Agents coordinate through a few rounds of message exchange and then decide locally, so every schedule is locally feasible by construction and the aggregate commitment gap stays small.

Fig. 2: Gauge layer: the gauge map pulls any network output into the inequality-feasible set, and equality completion reconstructs the full schedule, so local feasibility never depends on how well the network is trained.

Example	Detected gap	Favored adjustment
	Excess in both windows $[\hat{r}_{i}]_{1},[\hat{r}_{i}]_{2}>0$ Larger excess later $[\hat{r}_{i}]_{2}>[\hat{r}_{i}]_{1}$	Absorption (consume more) Deferral (consume later)
	Shortfall in both windows $[\hat{r}_{i}]_{1},[\hat{r}_{i}]_{2}<0$ Larger shortfall later $[\hat{r}_{i}]_{2}<[\hat{r}_{i}]_{1}$	Curtailment (consume less) Advance (consume earlier)

TABLE I: Illustrative local adjustment preferences for two windows

Method	$E$ , kW (%)	Max viol.	Loc. feas.	Opt. gap
ADMM [ 5 ]	262.7 (79)	0	yes	$-8.45\%^{\dagger}$
Penalty [ 13 ]	12.02 (3.6)	7.37	no	$-16.1\%^{\ddagger}$
Coord. only [ 10 ]	16.88 (5.1)	0	yes	3.93%
LOOP-MAC	2.48 (0.75)	0	yes	3.07%
LOOP-MAC, indep. set	3.77 (1.13)	0	yes	5.23%

TABLE II: Main results (

E

: commitment gap)

Rounds $K$	ADMM	Coord. only	LOOP-MAC
5	262.7 (79)	16.88 (5.1)	2.48 (0.75)
10	146.8 (44)	16.99 (5.1)	2.21 (0.66)

TABLE III: Commitment gap

E

, kW (%), versus communication rounds, original data

Variant	Original	Indep.
Coord. only	16.88 (5.1)	18.08 (5.4)
+ adjustment, initial gain	6.95 (2.1)	7.60 (2.3)
+ adjustment, trained gain ( LOOP-MAC )	2.48 (0.75)	3.77 (1.13)

TABLE IV: Ablation of the local adjustment,

E

, kW (%)

Component	LOOP-MAC	ADMM
One coordination round	158 $\mu$ s	0.33 s
Local adjustment	293 $\mu$ s	—
Total, five rounds and adjustment	1.1 ms	1.65 s

TABLE V: Per-agent online computation time

Multi-Agent Coordination

Multi-agent coordination studies how autonomous agents align actions, share information, and divide work to achieve individual or collective goals.

See all 502 papers

Safe Decentralized Operation of EV Virtual Power Plant with Limited Network Visibility via Multi-Agent Reinforcement Learning
Mar 24, 2026Chenghao Huang, Jiarong Fan, Weiqing Wang +1Power Systems Decentralized MARL
Department of Data Science and AI, Faculty of IT, Monash University, Australia · Monash Energy Institute, Monash University, Australia
Scalable Constrained Multi-Agent Reinforcement Learning via State Augmentation and Consensus for Separable Dynamics
May 28, 2026Santiago Amaya-Corredor, Miguel Calvo-Fullana, Anders JonssonMulti-Agent System Optimization Multi-Agent Coordination
Department of Engineering, University Pompeu Fabra
GradMAP: Gradient-Based Multi-Agent Proximal Learning for Grid-Edge Flexibility
Apr 27, 2026Yihong Zhou, Hongtai Zeng, Thomas MorstynMulti-Agent Control Constrained RL
Department of Engineering Science, University of Oxford

Mar 24, 2026J/K move · Enter open · S save

Safe Decentralized Operation of EV Virtual Power Plant with Limited Network Visibility via Multi-Agent Reinforcement Learning

Chenghao Huang, Jiarong Fan, Weiqing Wang, Hao Wang

Department of Data Science and AI, Faculty of IT, Monash University, Australia · Monash Energy Institute, Monash University, Australia

Power Systems Decentralized MARL Electric Vehicle Charging Safe RL Partially Observable RL

Open paper AskPDF arXiv

As power systems advance toward net-zero targets, behind-the-meter renewables are driving rapid growth in distributed energy resources (DERs). Virtual power plants (VPPs) increasingly coordinate these resources to support power distribution network (PDN) operation, with EV charging stations (EVCSs) emerging as a key asset due to their strong impact on local voltages. However, in practice, VPPs must make operational decisions with only partial visibility of PDN states, relying on limited, aggregated information shared by the distribution system operator. This work proposes a safety-enhanced VPP framework for coordinating multiple EVCSs under such realistic information constraints to ensure voltage security while maintaining economic operation. We develop Transformer-assisted Lagrangian Multi-Agent Proximal Policy Optimization (TL-MAPPO), in which EVCS agents learn decentralized charging policies via centralized training with Lagrangian regularization to enforce voltage and demand-satisfaction constraints. A transformer-based embedding layer deployed on each EVCS agent captures temporal correlations among prices, loads, and charging demand to improve decision quality. Experiments on a realistic 33-bus PDN show that the proposed framework reduces voltage violations by approximately 45% and operational costs by approximately 10% compared to representative multi-agent DRL baselines, highlighting its potential for practical VPP deployment.

Abstract

Multi-Agent Coordination

Explore similar work

Safe Decentralized Operation of EV Virtual Power Plant with Limited Network Visibility via Multi-Agent Reinforcement Learning

Scalable Constrained Multi-Agent Reinforcement Learning via State Augmentation and Consensus for Separable Dynamics

GradMAP: Gradient-Based Multi-Agent Proximal Learning for Grid-Edge Flexibility