The rapid proliferation of distributed energy resources (DERs) aggregated by virtual power plants (VPPs) elevates cybersecurity risk by expanding the attack surface. A coordinated cyberattack has the potential to disrupt distribution-system, bulk-system, and market operations. Existing cyberphysical defense strategies largely assume access to grid network models and operating conditions, which may not be available to third-party aggregators. This paper proposes a novel hybrid, perturbation-based cyber-physical defense for VPP-operated DERs that blends game-theoretic planning with model-free reinforcement learning (RL) under a Bayesian belief-update framework. Topology agnosticism is defined from the VPP information and implementation perspective: the VPP does not require access to the feeder topology, line parameters, switching configuration, or full system state. This work advances the state of the art through three contributions: (1) a multi-output Gaussian process with Hamiltonian Monte Carlo for discrete attacker-type inference, probability of detection, and confidence of detection for modeling the belief distribution; (2) moment-matched Gaussian propagation of posterior beliefs across episodes; and (3) an RL policy that adaptively integrates Stackelberg leader (VPP) actions with exploration. Furthermore, we establish equilibrium existence and derive an episode-level closed-loop utility bound connecting attacker-belief uncertainty with GT-RL action alignment. Evaluations on IEEE 13- and 123-Bus feeders with DER portfolios demonstrate effective detection under attacker-model uncertainty and evaluated feeder re-configuration conditions, while requiring no explicit feeder-topology knowledge. Overall, the results indicate that combining bi-level game-theoretic planning, nonparametric Bayesian inference, and adaptive RL is a promising direction for VPP cybersecurity.
Figures & tables
Fig. 1: The proposed GT-RL method optimizes the perturbation magnitudes applied to the physical DER environment, while the environment returns voltage observations Vo and the prescribed false-alarm setting PFA to the perturbation detector for watermark-response evaluation.
Fig. 2: Defender’s detection probability on the IEEE 13-Bus system.
Fig. 3: Defender’s evolving belief distribution on the IEEE 13-Bus system. The dashed lines show the ±2σ∗(θdet) region around the mean of posterior belief distribution, μ∗(θdet) .
Fig. 4: Convergence of reinforcement learning agent behavior across IEEE 13-Bus and 123-Bus systems. Learned perturbation magnitude decreases by an order of magnitude with no impact to detection performance.
Fig. 5: IEEE-13 single-attack baseline comparisons for MiTM, FDI, and DoS attacks. The proposed GT-RL methods are compared with RL-only, fixed-watermarking, machine-learning-based, and residual/statistical detection baselines.
Fig. 6: Ablation study contrasting attack detection rates with and without game-theoretic parameter tuning of the RL agent, alongside conventional statistical approaches: (a) MiTM; (b) FDI; (c) DoS attack detection for IEEE 123-Bus system.
Fig. 7: Impact of game-theoretic belief update on 1) attacker-profile parameter distribution on defender end; 2) the resulting perturbation magnitude obtained by RL agent; 3) detector SNR; and 4) mixed-attack macro-F1 across evaluation episodes for the IEEE-123 feeder under adaptive mixed attacks.
Method
Adaptive
Topology
Adaptive+Topology
GT-PPO
0.572 ± 0.203
0.591 ± 0.138
0.591 ± 0.138
GT-TD3
0.650 ± 0.136
0.699 ± 0.182
0.714 ± 0.192
RL-PPO
0.625 ± 0.169
0.652 ± 0.149
0.670 ± 0.134
RL-TD3
0.571 ± 0.158
0.608 ± 0.136
0.620 ± 0.154
RL-SAC
0.616 ± 0.132
0.604 ± 0.165
0.604 ± 0.165
RandWM
0.606 ± 0.081
0.670 ± 0.112
0.660 ± 0.110
TABLE I: Mixed-attack macro-F1 for the IEEE-13 test system. Entries are mean ± standard deviation over 10 evaluation runs. Bold marks the best method for each regime.
Method
Adaptive
Topology
Adaptive+Topology
GT-PPO
0.649 ± 0.236
0.586 ± 0.182
0.587 ± 0.192
GT-TD3
0.696 ± 0.067
0.717 ± 0.097
0.717 ± 0.097
RL-PPO
0.664 ± 0.230
0.630 ± 0.241
0.632 ± 0.255
RL-TD3
0.513 ± 0.184
0.520 ± 0.170
0.520 ± 0.170
RL-SAC
0.585 ± 0.191
0.577 ± 0.203
0.561 ± 0.178
RandWM
0.502 ± 0.241
0.498 ± 0.232
0.517 ± 0.203
TABLE II: Mixed-attack macro-F1 for the IEEE-123 test system. Entries are mean ± standard deviation over 10 evaluation runs. Bold marks the best method for each regime.
As power systems advance toward net-zero targets, behind-the-meter renewables are driving rapid growth in distributed energy resources (DERs). Virtual power plants (VPPs) increasingly coordinate these resources to support power distribution network (PDN) operation, with EV charging stations (EVCSs) emerging as a key asset due to their strong impact on local voltages. However, in practice, VPPs must make operational decisions with only partial visibility of PDN states, relying on limited, aggregated information shared by the distribution system operator. This work proposes a safety-enhanced VPP framework for coordinating multiple EVCSs under such realistic information constraints to ensure voltage security while maintaining economic operation. We develop Transformer-assisted Lagrangian Multi-Agent Proximal Policy Optimization (TL-MAPPO), in which EVCS agents learn decentralized charging policies via centralized training with Lagrangian regularization to enforce voltage and demand-satisfaction constraints. A transformer-based embedding layer deployed on each EVCS agent captures temporal correlations among prices, loads, and charging demand to improve decision quality. Experiments on a realistic 33-bus PDN show that the proposed framework reduces voltage violations by approximately 45% and operational costs by approximately 10% compared to representative multi-agent DRL baselines, highlighting its potential for practical VPP deployment.
Chenghao Huang, Jiarong Fan, Weiqing Wang +1
Department of Data Science and AI, Faculty of IT, Monash University, Australia · Monash Energy Institute, Monash University, Australia
Distributed Volt--VAR control relies on voltage measurements and is therefore vulnerable to false-data injection. Neighbor corroboration can distinguish an isolated corrupted measurement from a physical disturbance, but coordinated agents can falsely corroborate one another. This paper proposes DynaTrust-VVC, a cyber-resilient multi-agent Volt--VAR framework that assigns each incoming neighbor message a directional trust value based on the consistency between its reported voltage increment and a sensitivity-based prediction from active- and reactive-power changes. Trust falls immediately after a physics mismatch and recovers gradually after sustained consistency. DynaTrust-VVC further uses a physics-adaptive evidence threshold and counterfactual safe-voltage recovery-based mitigation. The proposed method was validated using a nonlinear IEEE 123-bus feeder with minute-resolution residential profiles and 27 matched replicates. DynaTrust-VVC identifies a coordinated three-bus attack one minute after onset in all 27 matched replicates, whereas an otherwise identical configuration with fixed neighbor trust identifies none. The composite resilience index also increases from 0.227 to 0.315, with no statistically significant increase in the benign false-alarm rate.
Md Fazley Rafy, Kamrul Hasan, Anurag K. Srivastava
Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV 26505, USA
This paper compares the performance of model-free controllers on a nonlinear system under cyberattacks, including false data injection and denial-of-service attacks. Four RL reward types are analyzed for accuracy, cost, and resilience. Results show that the Lyapunov reward offers the best resilience with low tracking error. Exponential mode also provides good trade-offs with acceptable resilience under moderate training conditions. Progressive and linear rewards converge faster but are less robust. RL-MPCs show strong steady-state resilience but require longer training times; RL-PID controllers are faster with significantly less training time. Proximal Policy Optimization outperforms Deep Deterministic Policy Gradient with a significant reduction in KPI variance. This study serves to highlight how well-designed RL rewards can improve performance and resilience against cyber threats.
Hugo O. Garcés, Alejandro J. Rojas, Bernardo A. Hernández +5
Departamento de Ingeniería Informática y Ciencias de la Computación, Universidad de Concepción, Concepción, Chile · Electrical Engineering Departament, Universidad de Talca · Department of Electrical & Computer Engineering, University of Alberta, Edmonton, T6G 1H9, Alberta, AB, Canada +2