Noise Steering: Exploiting Wake Steering and Acoustic Directivity for Noise-Constrained Wind Farm Control via Reinforcement Learning
Authors: Martín de Frutos, Baptiste Corban, Claire Bizon Monroc, Ana Bušić, Donatien Dubuc, Jiamin Zhud, Esteban Ferrer
Organizations: ETSIAE-UPM - School of Aeronautics, Universidad Politécnica de Madrid, Plaza Cardenal Cisneros 3, Madrid, E-28040, Spain · Inria, 48 rue Barrault, Paris, 75013, France · DI ENS, Ecole Normale Supérieure, PSL Research University, Paris, France · IFP Energies nouvelles, 1 et 4 avenue de Bois-Préau, Rueil-Malmaison, 92852, France · Center for Processes, Renewable Energy and Energy Systems (PERSEE), Mines Paris PSL, 1 Rue Claude Daunesse, Sophia Antipolis, 06560, France · Center for Computational Simulation, Universidad Politécnica de Madrid, Campus de Boadilla del Monte, Madrid, E-28660, Spain
Existing noise mitigation strategies for wind farms usually consist in reducing turbines power output, without accounting for the wake interactions between them and the noise emission directivity. This leads to over-conservative policies and unexploited wind power. Yet, turbine yaw can simultaneously affect wake interactions and the direction of noise emission, offering an additional degree of freedom for farm-level noise control. We therefore propose Noise Steering, a model-free reinforcement learning (RL) strategy that jointly controls turbine yaw angles and power setpoints to maximise farm power while satisfying a local noise constraint in a defined region. To enable training at a reduced computational cost, a low-fidelity acoustic model is coupled with the steady-state FLORIS simulator. The RL agent effectively learns how to redirect wakes and noise directivity away from the protected region while redistributing power across the farm. On a six-turbine test farm, the policy satisfies the 40 dB noise constraint throughout the full range of turbulent winds, with an average power deficit of only 2% relative to greedy unimpeded control, and up to 7.7% at high wind speeds. In comparison, the industry-standard noise-reduced operation (NRO) baseline here generates an average power loss of 17%. When deployed in the higher-fidelity FAST.Farm simulator, the policy largely retains its performance under steady and turbulent inflow, despite differences in simulation fidelity. These results demonstrate the potential of combining yaw control and power derating through RL to improve the power-noise trade-off in wind farm operation.
Figures & tables
Velocity deficit
Deflection
ka
kb
αg
βg
dm
bd
FLORIS default
0.38
0.004
0.58
0.077
1.0
0.0
Calibrated
0.31
0.0024
0.99
0.149
0.893
0.00699
Table 1: Parameters of the Gaussian wake model used in FLORIS.
Figure 2: Average episode reward and penalisation terms during training. An episode corresponds to 500 steps. Bold curves are filtered over using an exponential smoothing of 0.8.
Figure 3: Power and noise emissions reached by the wind farm at the end of one episode for the complete wind speed range. Comparison of different control strategies.
Figure 4: Final commands of the Noise Steering policy for the complete wind speed range. The wind turbine naming follows Figure 1 convention, where ”1” stands for the row closer to the acoustic region and ”2” for the other row of wind turbines.
Figure 5: Wind farm final state reached by the RL policy for U∞=15\mathrm{m}\text{,}{\mathrm{s}}^{-1}$$ on FLORIS. Top: Wind speed field. Bottom: Acoustic field, control region denoted for reference.
Figure 6: Comparison of the three different control strategies on FAST.Farm and FLORIS for the complete wind speed range.
Figure 7: Turbulent and unsteady wind speed profile. Ambient wind displayed at hub height for the three wind turbine positions of the first row (front, mid and last). Bold curves are filtered using a 60s moving average.
Figure 8: Comparison of the three different control strategies on FAST.Farm for an unsteady inflow.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Value
Reward
α
0.15
β1
0.002
β2
0.1
β3
0.0025
PPO
Learning rate
0.0003
Critic learning rate
0.0006
Appendix
Table 2: Reward and PPO hyperparameters used for RL training.
Case
U∞(\mathrm{m}\text{,}{\mathrm{s}}^{-1})
γt(\mathrm{\SIUnitSymbolDegree})
PS(\mathrm{MW})
ΔL^p(\mathrm{dB})
1
10.0
[30,30,0]
[5,5,5]
0.74
2
11.4
[0,0,0]
[5,5,5]
-0.61
3
10.0
[0,−10,20]
[3,3,5]
0.57
Appendix
Table 3: Wind farm operating conditions for the noise validation cases shown in Figure 9 . The difference between the higher fidelity model (FAST.Farm + BPM) and the lower fidelity model (FLORIS +Hagg) noise levels in the control region, ΔL^p , is also reported.
Figure 9: Comparison of three wind farm noise modelling approaches for three operational conditions. The control region is denoted for reference. The OASPL is averaged in time to reduce the amplitude modulation oscillations.
Figure 10: Turbine-scale episode comparison for the Noise Steering policy evaluated in FAST.Farm and FLORIS for U∞=13.2\mathrm{m}\text{,}{\mathrm{s}}^{-1}$$ . Only the first row of turbines is displayed.
Figure 11: Farm-scale episode comparison for the Noise Steering policy evaluated in FAST.Farm and FLORIS. Steady free wind speed of U∞=13.2\mathrm{m}\text{,}{\mathrm{s}}^{-1}.
Wind farm wake steering optimization is challenging due to complex flow physics and changing conditions. This paper presents a hierarchical framework that combines reinforcement learning with model predictive control, where an RL agent learns compensatory state estimates for an MPC controller, rather than directly controlling turbines. Evaluated on a three-turbine case, the approach achieves a 23% power gain over the baseline control and surpasses the idealized MPC with perfect state knowledge. Compared to direct RL control, the hybrid architecture maintains superior safety characteristics during training while achieving comparable performance with more stable control actions.
Marcus Binder Nilsen, Teodor Olof Benedict Åstrand, Tuhfe Göçmen +1
Technical University of Denmark, Department of Wind and Energy Systems. Roskilde, Frederiksborgvej 399, Denmark
This paper addresses the wind farm power maximization problem in the presence of wind direction changes. Specifically, a model-free Modified Twin Delayed Deep Deterministic Policy Gradient with Behavior Cloning (MTD3-BC) algorithm is proposed to tackle this task through yaw control under varying wind direction conditions. MTD3-BC is an offline reinforcement learning (RL) algorithm that aims to infer good behavior from only a precollected offline dataset. Additionally, to ensure smooth and moderate yaw adjustments, a new action consistency term is introduced into the policy optimization objective. Unlike online RL methods, MTD3-BC does not require extensive interactions with a wind farm simulator during training, significantly reducing computational costs and training time. A wind tunnel experiment is conducted to validate the effectiveness of the algorithm under varying wind directions. The results demonstrate that MTD3-BC successfully mitigates wake effects, delivering farm-level power gains of approximately 10% over the baseline greedy strategy and performance on par with a data-calibrated model-based wake-steering benchmark, while requiring no wake model and only a small fraction of the training cost of online RL. To our knowledge, this is the first time an offline RL wind farm control policy has been validated and demonstrated experimentally.
Yuhan Su, Hongyang Dong, Simone Tamaro +3
Intelligent Control and Smart Energy (ICSE) Research Group, School of Engineering, University of Warwick, Coventry CV4 7AL, U.K. · Wind Energy Institute, Technical University of Munich, 85748 Garching bei München, Germany
Reinforcement learning (RL) offers a promising approach for adaptive wind farm flow control, yet its practical deployment is hindered by slow training convergence and poor initial performance, factors that could translate to years of reduced power output if an untrained agent were deployed directly. This work investigates whether domain knowledge from steady-state wake models can accelerate RL training and improve initial controller performance. We propose a pretraining methodology in which expert demonstrations are generated by deploying a PyWake-based steady-state optimizer within a dynamic wake simulation (WindGym), then used to initialize both the actor and critic networks of a Soft Actor-Critic agent via behavior cloning. Experiments on a 2x2 wind farm show that pretraining eliminates the costly initial learning phase: while an untrained agent underperforms the greedy zero-yaw baseline by approximately 12%, pretraining raises initial performance to near-baseline levels. During online fine-tuning, all configurations converge within 250,000 environment steps to achieve similar performance, ultimately exceeding that of a lookup-table controller, which reaches approximately 7% power gain after 500,000 steps.
Marcus Binder Nilsen, Julian Quick, Tuhfe Göçmen +2
Department of Wind and Energy Systems, Technical University of Denmark, Roskilde, Denmark