Load constrained wind farm flow control through multi-objective multi-agent reinforcement learning

Teodor Åstrand; Marcus Binder Nilsen; Iasonas Tsaklis; Tuhfe Göçmen; Pierre-Elouan Réthoré; Nikolay Dimitrov

eess.SYApr 13, 2026

Load constrained wind farm flow control through multi-objective multi-agent reinforcement learning

Authors: Teodor Åstrand, Marcus Binder Nilsen, Iasonas Tsaklis, Tuhfe Göçmen, Pierre-Elouan Réthoré, Nikolay Dimitrov

Organizations: Department of Wind and Energy Systems, Technical University of Denmark, Roskilde, Denmark

Abstract

This study presents a multi-agent reinforcement learning (MARL) framework for load-constrained wind farm flow control (WFFC). While wake steering can enhance total wind farm power, it often introduces increased structural loads on downstream turbines. To address this, we integrate an Independent Soft Actor-Critic (I-SAC) architecture with a data-driven, local inflow sector-averaged surrogate model to provide real-time estimates of Damage Equivalent Loads (DELs). By incorporating these estimates into a shaped reward function, turbine-specific agents are trained to maximize power production while adhering to specific load-increase thresholds ( $Δ_{max}$ ) of 10%, 20%, and 30% relative to a baseline controller. The framework is implemented within the WindGym environment using the DYNAMIKS flow solver with Dynamic Wake Meandering (DWM) model to capture non-stationary wake physics. Results indicate that the MARL agents successfully learn collaborative policies that prioritise power gain while actively retreating from high-DEL control strategies.

Figures & tables

Figure 1: Overview of the utilised MARL setup using WindGym.

\br Category	Description / Value
\mr Numerical Tools	DYNAMIKS (DWM-based solver), Mann turbulence model
Inflow Conditions	$ws=10$ m/s, $wd=270^{\circ}$ , $TI=5\%$ (Uniform profile)
Mann Turbulence Box	Length scale $L=33.6$ m, Flow length $L_{x}=12,276$ m
Farm Layout	3 turbines, inline configuration, $6D$ spacing
Turbine Model	Siemens SWT-2.3 (2.3 MW)
Constraint Driving Load	Blade-root flapwise bending moment (DEL)

Table 1: Numerical environment and inflow parameters.

Figure 2: Agent training progress for six random seeds, each running in $N_{env}=15$ parallel environments for a cumulative 150,000 environment steps.

Figure 3: Time series of farm-level power production for the MARL-controlled case compared with the baseline greedy controller during a 3000-second simulation using a turbulence box unseen during training, for different levels of DEL constraint.

Panel of Figure 4 — Figure 4: a) Time series of turbine-specific DEL for the blade root flap-wise moment, computed using a sliding-window method, for the MARL-controlled over a 3000-second simulation using a common unseen turbulence box. The DEL limit is based on the max DEL for all turbines from a baseline controller, where the analysis region represents the period after the flow is advected through the wind farm. b) Distribution of turbine-level DEL for the blade root flap-wise moment. The time-dependent DEL constraint is represented by its mean (dotted vertical line) and the $5^{th}$ and $95^{th}$ percentiles (shaded region).

Figure 5: Yaw angles commanded by learned agent policies during evaluation.

Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning studies how interacting agents learn coordinated or competitive policies through shared experience and rewards.

See all 373 papers

Hierarchical RL-MPC Control for Dynamic Wake Steering in Wind Farms
Apr 13, 2026Marcus Binder Nilsen, Teodor Olof Benedict Åstrand, Tuhfe Göçmen +1RL Control Model Predictive Control
Technical University of Denmark, Department of Wind and Energy Systems. Roskilde, Frederiksborgvej 399, Denmark
Accelerating Reinforcement Learning for Wind Farm Control via Expert Demonstrations
Apr 13, 2026Marcus Binder Nilsen, Julian Quick, Tuhfe Göçmen +2RL Control Offline-To-Online RL
Department of Wind and Energy Systems, Technical University of Denmark, Roskilde, Denmark
Noise Steering: Exploiting Wake Steering and Acoustic Directivity for Noise-Constrained Wind Farm Control via Reinforcement Learning
Oct 27, 2026Martín de Frutos, Baptiste Corban, Claire Bizon Monroc +4Optimal Control RL Control
ETSIAE-UPM - School of Aeronautics, Universidad Politécnica de Madrid, Plaza Cardenal Cisneros 3, Madrid, E-28040, Spain · Inria, 48 rue Barrault, Paris, 75013, France · DI ENS, Ecole Normale Supérieure, PSL Research University, Paris, France +3

Apr 13, 2026J/K move · Enter open · S save

Hierarchical RL-MPC Control for Dynamic Wake Steering in Wind Farms

Marcus Binder Nilsen, Teodor Olof Benedict Åstrand, Tuhfe Göçmen, Pierre-Elouan Réthoré

Technical University of Denmark, Department of Wind and Energy Systems. Roskilde, Frederiksborgvej 399, Denmark

RL Control Model Predictive Control Hierarchical RL

Open paper AskPDF arXiv

Wind farm wake steering optimization is challenging due to complex flow physics and changing conditions. This paper presents a hierarchical framework that combines reinforcement learning with model predictive control, where an RL agent learns compensatory state estimates for an MPC controller, rather than directly controlling turbines. Evaluated on a three-turbine case, the approach achieves a 23% power gain over the baseline control and surpasses the idealized MPC with perfect state knowledge. Compared to direct RL control, the hybrid architecture maintains superior safety characteristics during training while achieving comparable performance with more stable control actions.

Abstract

Multi-Agent Reinforcement Learning

Explore similar work

Hierarchical RL-MPC Control for Dynamic Wake Steering in Wind Farms

Accelerating Reinforcement Learning for Wind Farm Control via Expert Demonstrations

Noise Steering: Exploiting Wake Steering and Acoustic Directivity for Noise-Constrained Wind Farm Control via Reinforcement Learning