Physics-guided surrogate learning enables zero-shot control of turbulent wings

Yuning Wang; Pol Suarez; Mathis Bode; Ricardo Vinuesa

physics.flu-dynApr 10, 2026

Physics-guided surrogate learning enables zero-shot control of turbulent wings

Authors: Yuning Wang, Pol Suarez, Mathis Bode, Ricardo Vinuesa

Organizations: Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI 48109, USA. · FLOW, Engineering Mechanics, KTH Royal Institute of Technology, SE-100 44 Stockholm, Sweden. · Jülich Supercomputing Centre, Forschungszentrum Jülich GmbH, 52425 Jülich, Germany

Abstract

Turbulent boundary layers over aerodynamic surfaces are a major source of aircraft drag, yet their control remains challenging due to multiscale dynamics and spatial variability, particularly under adverse pressure gradients. Reinforcement learning has outperformed state-of-the-art strategies in canonical flows, but its application to realistic geometries is limited by computational cost and transferability. Here we show that these limitations can be overcome by exploiting local structures of wall-bounded turbulence. Policies are trained in turbulent channel flows matched to wing boundary-layer statistics and deployed directly onto a NACA4412 wing section at $Re_c=2\times10^5$ without additional training on the wing, constituting zero-shot deployment. The transferred policies reduce total drag by 11.1%, while state-of-the-art opposition control reduces it by 8.9%. With the actuation cost taken into account, the corresponding net-energy savings are 10.9% and 8.8%, respectively. Surrogate training provides an estimated improvement of four orders of magnitude in exploration efficiency over direct on-wing training, enabling scalable flow control.

Figures & tables

Figure 1: Physics-guided zero-shot MARL framework for turbulent wing control. (A) The suction side of a NACA4412 wing is decomposed into chordwise control blocks $\Omega_{i}$ . (B) Each $\Omega_{i}$ is matched to a turbulent channel-flow surrogate by reproducing the local shape factor ( $H_{12}$ ) and momentum-thickness Reynolds number ( $Re_{\theta}$ ), capturing both flow conditions and pressure-gradient history of the local state. (C) MARL agents are trained exclusively within these surrogates using a shared neural network, with wall-parallel and wall-normal velocity fluctuations $\{u_{t},v_{n}\}$ at sensing height $y_{s}$ as state, reward function based on the wall-shear stress $\tau_{w}$ , $p^{\prime}$ , $v_{n,(i,j)}|_{{wall}}$ , and local blowing/suction wall-normal velocity $v_{n,(i,j)}|_{{wall}}$ as action. (D) The trained policy is deployed directly to the corresponding wing blocks without additional on-wing training.

Figure 2: Computational setup. NACA4412 wing at $Re_{c}=2\times 10^{5}$ and $\alpha=5^{\circ}$ with the suction-side control region partitioned into four chordwise blocks $\Omega_{1}$ – $\Omega_{4}$ , together with representative turbulent structures from the wing and a matched turbulent channel-flow (TCF) surrogate.

Figure 3: Surrogate-matching strategy. (A) Chordwise evolution of $Re_{\theta}$ , $H_{12}$ , and $\beta$ on the suction side of the base flow; shaded regions indicate the blocks where statistics are averaged. (B) Wing-to-TCF matching tree for DRL-US. Controllers trained on each surrogate are deployed directly to the corresponding wing blocks with uniform suction of $0.2\%\,U_{\infty}$ .

Figure 4: Zero-shot control performance summary on the NACA4412 wing section. (A) Left: chordwise distribution of the local net-energy saving $\mathrm{NES}_{\mathrm{Wing},\tau}$ (top) and ZNMF power-input ratio $\mathrm{PIR}^{\mathrm{ZNMF}}_{\mathrm{Wing},\tau}$ (bottom) for the DRL-US and OC-US, together with the block-averaged values over $\Omega_{1}$ – $\Omega_{4}$ as bar charts. Right: block-averaged $\mathrm{NES}_{\mathrm{TCF},\tau}$ (top) and $\mathrm{PIR}^{\mathrm{ZNMF}}_{\mathrm{TCF},\tau}$ (bottom) obtained during MARL training in each TCF surrogate $\Omega_{1}$ – $\Omega_{4}$ . (B) Total drag-coefficient budget decomposed into friction ( $C_{d,f}$ ) and pressure ( $C_{d,p}$ ) contributions relative to the uncontrolled base flow, for the uncontrolled case with and without uniform suction (sketched on the left), and for OC-US and DRL-US. Labels on the right indicate the relative change in total drag with respect to BASE, $\Delta C_{d}/C_{d}^{\rm BASE}=-\mathrm{PR}_{\mathrm{Wing},D}$ , and the dashed boxes mark the corresponding drag reduction. The bars below each case show the power-input ratios of the uniform suction, $\mathrm{PIR}^{\mathrm{BLC}}_{\mathrm{Wing},D}$ , and of the ZNMF actuation, $\mathrm{PIR}^{(p)}_{\mathrm{Wing},D}$ and $\mathrm{PIR}^{(v)}_{\mathrm{Wing},D}$ , with their sum annotated, on the lower axis in percent of the reference drag power.

Figure 5: Zero-shot transfer of the actuation structure from the channel surrogates to the NACA4412 wing. First row (TCF): instantaneous wall-normal actuation velocity $v_{n}|_{\mathrm{wall}}$ in the turbulent-channel-flow surrogate environments $\Omega_{1}$ – $\Omega_{4}$ . Second row: spanwise two-point correlation of the wall actuation, $R_{v_{n}v_{n}}|_{\mathrm{wall}}$ , as a function of the spanwise separation $\Delta z^{+}$ , comparing each TCF surrogate, dashed, with the corresponding location on the wing ( $x_{ss}/c=0.30$ , $0.42$ , $0.65$ and $0.75$ for $\Omega_{1}$ – $\Omega_{4}$ ), solid line. Third and fourth rows (DRL): zero-shot on the suction side of the NACA4412 wing for DRL with uniform suction at $0.2\%\,U_{\infty}$ (DRL-US), showing close-up views of the windows marked by dashed boxes in the full suction-side field below. Wing quantities in wall units are scaled with the block-averaged viscous length $\langle l^{*}\rangle_{\Omega_{i}}$ . Bottom row (OC): corresponding deployment for opposition control with uniform suction (OC-US).

Figure 6: Computational and operational cost of zero-shot DRL training and deployment. (A) Computational savings for training DRL controllers, expressed as ratios between the NACA4412 wing and TCF configurations for the factors in equation 4 . Single-controller cost ratios are shown below the dashed line, and the policy-count ratio

N_{\rm policy}

required to cover the full control region is shown above; data correspond to control block

\Omega_{2}

and its surrogate channel flow. (B) Operational cost in terms of core-hours and average memory (GB) for one episode of zero-shot wing deployment, TCF exploitation, and TCF training using Intel Xeon Platinum 8168 CPUs. Percentages in parentheses denote the proportion relative to the cost of wing deployment.

Case	Block	$x_{ss}/c$	$\langle Re_{\theta}\rangle_{x}$	$\langle H_{12}\rangle_{x}$	$\langle\beta\rangle_{x}$	Target	Value	Deviation ( $\leq 5\%$ )	Name
Base Flow	$\Omega_{1}$	$0.25$ – $0.40$	365	1.70	0.39	$Re_{\theta}$	359	1.6%	TCF-1
–	$\Omega_{2}$	$0.40$ – $0.50$	522	1.69	0.73	–	518	0.6%	TCF-2
–	$\Omega_{3}$	$0.50$ – $0.70$	739	1.72	1.42	–	732	0.9%	TCF-3
–	$\Omega_{4}$	$0.70$ – $0.86$	1074	1.87	5.27	–	–	–	–
BASE-US $0.2\%\,U_{\infty}$	$\Omega_{1}$	$0.25$ – $0.40$	352	1.65	0.30	$H_{12}$	1.65	0.0%	TCF-BDF-1
–	$\Omega_{2}$	$0.40$ – $0.50$	461	1.63	0.51	–	1.65	1.0%	TCF-BDF-2

Table 1: Flow characteristics of the surrogate channel flows. Block-averaged statistics of the wing base flows and matching parameters of the surrogate channel flows that reproduce the target metrics within 5% discrepancy.

Name	$Re_{b}$	$Re_{\tau}$	$N_{x}$	$N_{y}$	$N_{z}$	$\Delta x^{+}$	$\Delta y^{+}_{w}$	$\Delta z^{+}$	$\gamma$	$y^{+}_{b}$
TCF-1	3,300	206	80	64	72	18.8	0.5	10.1	–	–
TCF-2	4,600	280	80	64	72	25.6	0.7	15.4	–	–
TCF-3	7,000	406	120	96	108	20.6	0.5	14.4	–	–
TCF-BDF-1	3,300	185	80	64	72	16.8	0.4	9.0	5.0	20.0
TCF-BDF-2	4,300	204	80	64	72	18.7	0.5	11.2	15.0	30.0
TCF-BDF-3	6,000	286	120	96	108	14.5	0.4	10.1	15.0	60.0

Table 2: Channel flow configurations. The domain size is

(L_{x},\,L_{y},\,L_{z})=(2\pi h,\,h,\,\pi h)

for all configurations.

\Delta y^{+}_{w}

denotes the wall-normal distance of the first grid point from the wall.

Figure 7: Sketch of the energy budget underlying the net-energy saving (NES). (A) Driving-power density at the wall,

\mathcal{P}=U\tau_{w}

, with

U\in\{U_{b},U_{\infty}\}

, together with the pressure-work and kinetic-energy contributions to ZNMF actuation power,

\mathrm{PI}^{(p)}

and

\mathrm{PI}^{(v)}

, and the pumping power associated with uniform suction,

\mathrm{PI}^{\mathrm{BLC}}

, on the wing section. (B) Power budget normalized by the reference driving power

\mathcal{P}^{\mathrm{ref}}

. Subtracting the applicable actuation-power ratios,

\mathrm{PIR}^{\mathrm{ZNMF}}

and

\mathrm{PIR}^{\mathrm{BLC}}

, from the power-reduction ratio PR gives NES. Bar lengths are illustrative.

Case	Control configuration	Background flow	NES reference
Case	Control configuration	Background flow	Local	Integral
BASE	Uncontrolled flow	–	–	–
OC-noUS	OC with $\alpha=1.0$ and $y_{n}^{+}=15$	BASE	BASE	BASE
DRL-noUS	DRL on $\Omega_{1}$ – $\Omega_{3}$	BASE	BASE	BASE
DRL-noUS-UB	DRL on $\Omega_{1}$ – $\Omega_{3}$ , with UB at $0.2\%\,U_{\infty}$ on $\Omega_{4}$	BASE	BASE	BASE
BASE-US	US at $0.2\%\,U_{\infty}$	BASE	BASE	BASE

Table 3: Wing control configurations and reference states for net-energy saving. The listed control is applied to the indicated background flow. US and UB denote uniform suction and uniform blowing, respectively. OC and US act over

0.25\leq x_{ss}/c\leq 0.86

. The last two columns identify the reference states used to evaluate the local wall-shear-based NES,

\mathrm{NES}_{\mathrm{Wing},\tau}

, and the integral drag-based NES,

\mathrm{NES}_{\mathrm{Wing},D}

, respectively.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Learning type	Evaluation type	Configuration	$\mathrm{PR}_{\mathrm{TCF},\tau}$ $[\%]$	${\rm PIR}_{\mathrm{TCF},\tau}^{(p)}$ $[\%]$	${\rm PIR}_{\mathrm{TCF},\tau}^{(v)}$ $[\%]$	$\mathrm{NES}_{\mathrm{TCF},\tau}$ $[\%]$
-	I	OC	22.75	0.40	0.22	22.12
I	I	DRL	30.19	1.55	0.22	28.42
II	I	DRL	26.61	0.96	0.10	25.55
-	II	OC	22.75	0.40	0.22	22.12
I	II	DRL	30.19	4.93	0.04	25.20
II	II	DRL	26.61	3.19	0.05	23.37

Appendix

Supplementary Table 1 : Effect of the power-input convention on training and evaluation in TCF-2. PR, pressure-work and kinetic-energy input ratios, and NES are reported for OC and the selected DRL policies. Learning type identifies the reward convention. Evaluation type identifies the convention used to report the costs and NES. Type-I follows Methods and Type-II is given in equation 2 . Means use six initial conditions after excluding

t^{+}<500

. Bold entries identify the highest reported NES under each evaluation convention.

Supplementary Figure 1 : Pressure distributions used to estimate the uniform-suction power input. Left: suction-side

-c_{p}

for the uncontrolled Base Flow (grey dashed) and uniform suction at

0.2\%\,U_{\infty}

(black solid). Right: the absolute difference

|c_{p}^{\mathrm{US}}-c_{p}^{\mathrm{Base}}|

. Shading identifies the four control blocks within

0.25\leq x_{\mathrm{ss}}/c\leq 0.86

. The pressure difference enters the BLC power estimate defined in Methods.

Case	Configuration	$\mathrm{PR}_{\mathrm{TCF},\tau}$ $[\%]$	$\mathrm{PIR}^{(p)}_{\mathrm{TCF},\tau}$ $[\%]$	$\mathrm{PIR}^{(v)}_{\mathrm{TCF},\tau}$ $[\%]$	$\mathrm{NES}_{\mathrm{TCF},\tau}$ $[\%]$
TCF-1	OC	25.00	0.29	0.10	24.61
	DRL	31.40	0.49	0.07	30.85
TCF-2	OC	22.75	0.40	0.22	22.12
	DRL	30.19	1.55	0.22	28.42
TCF-3	OC	22.12	0.44	0.20	21.48
	DRL	24.81	0.96	0.13	23.72

Appendix

Supplementary Table 2 : Deterministic energy performance in the seven channel surrogates. PR, pressure-work and kinetic-energy input ratios, and NES use the Type-I convention defined in Methods. Means use six initial conditions after excluding

t^{+}<500

. Bold entries identify the higher NES in each channel.

Supplementary Figure 2 : Training, deterministic performance, and wall actuation in the channel surrogates. (A) Episode-mean NES (top) and its wall-shear, pressure-work, and kinetic-energy contributions (bottom). Curves are smoothed using sliding-window averaging with a window size equal to one episode. Stars mark maximum rewards. Component axes are symmetric-logarithmic, with costs negative. (B) Realization-mean deterministic NES for OC and DRL, with coloured bands of $\pm$ one standard deviation. Grey shading marks $t^{+}<500$ . Legend values are means over the retained data. (C) Instantaneous OC (top) and DRL (bottom) wall actuation against $x/h$ and $z/h$ . Colour limits are $\pm 0.04$ for $v_{n}|_{\rm wall}$ . Positive and negative values denote blowing and suction, respectively.

Supplementary Figure 3 : Zero-shot wing control without background suction. (A) Uncontrolled suction-side $Re_{\theta}$ , $H_{12}$ , and $\beta$ , with the control blocks shaded. (B) Local wing NES and the negative ZNMF power-input ratio, referenced to the uncontrolled Base Flow. Bars compare block means for OC and DRL without downstream blowing over $\Omega_{1}$ – $\Omega_{3}$ . The lower bar chart gives the corresponding deterministic channel results. Additional curves show hybrid DRL with downstream blowing at $0.2\%\,U_{\infty}$ and uniform blowing alone at $0.1\%\,U_{\infty}$ . The local metric excludes BLC pump power. (C) Instantaneous hybrid DRL (top) and OC (bottom) wall actuation, with common colour limits of $\pm 0.04$ in the exported simulation velocity units. Positive and negative values denote blowing and suction. The hybrid’s downstream blowing block $\Omega_{4}$ is masked.

Case	$\mathrm{PR}_{\mathrm{Wing},D}$	$\mathrm{PIR}^{(\mathrm{ZNMF})}_{\mathrm{Wing},D}$	$\mathrm{PIR}^{(\mathrm{BLC})}_{\mathrm{Wing},D}$	$\mathrm{NES}_{\mathrm{Wing},D}$
OC-noUS	$3.85\%$	$0.12\%$	$0.00\%$	$3.73\%$
DRL-noUS	$5.18\%$	$0.14\%$	$0.00\%$	$5.04\%$
DRL-noUS-UB	$4.95\%$	$0.14\%$	$0.01\%$	$4.80\%$
BASE-US	$3.53\%$	$0.00\%$	$0.14\%$	$3.39\%$
OC-US	$8.93\%$	$0.04\%$	$0.14\%$	$8.75\%$
DRL-US	$11.11\%$	$0.10\%$	$0.14\%$	$10.87\%$

Appendix

Supplementary Table 3 : Integral wing-drag power budget. All entries are percentages of the uncontrolled BASE driving power. ZNMF costs are integrated over the active interval using the Type-I convention in Methods. The BLC estimate uses the uniform-suction pressure profile for all US cases. For the hybrid DRL-noUS-UB case, ZNMF acts over

0.25\leq x_{\mathrm{ss}}/c\leq 0.70

and the downstream-blowing cost uses its DRL pressure profile over

0.70\leq x_{\mathrm{ss}}/c\leq 0.86

. NES subtracts both applicable costs from the total-drag PR.

Case	$C_{l}$	$C_{d,f}$	$C_{d,p}$	$C_{d}=C_{d,f}+C_{d,p}$	$L/D$
Base Flow	$0.8611$	$0.0127$	$0.0087$	$0.0214$	$40.3$
OC-noUS	$0.8734\;(+1.4\%)$	$0.0119\;(-6.4\%)$	$0.0087\;(0.0\%)$	$0.0206\;(-3.8\%)$	$42.5\;(+5.5\%)$
DRL-noUB	$0.8775\;(+1.9\%)$	$0.0118\;(-6.9\%)$	$0.0085\;(-2.6\%)$	$0.0203\;(-5.2\%)$	$43.3\;(+7.5\%)$
DRL-noUS-UB	$0.8568\;(-0.5\%)$	$0.0117\;(-8.3\%)$	$0.0087\;(-0.1\%)$	$0.0203\;(-5.0\%)$	$42.1\;(+4.7\%)$
BASE-US	$0.9180\;(+6.6\%)$	$0.0139\;(+9.7\%)$	$0.0067\;(-22.9\%)$	$0.0206\;(-3.5\%)$	$44.5\;(+10.5\%)$
OC-US	$0.9349\;(+8.6\%)$	$0.0132\;(+3.8\%)$	$0.0063\;(-27.5\%)$	$0.0195\;(-8.9\%)$	$48.0\;(+19.2\%)$

Appendix

Supplementary Table 4 : Integral aerodynamic coefficients and lift-to-drag ratio. Values are arithmetic means of complete, non-overlapping averaging segments starting at elapsed

t\geq 6

flow-over time units. Parentheses give signed percentage changes relative to the uncontrolled Base Flow.

L/D

is the mean of the segmentwise ratios. DRL-noUB denotes the no-suction DRL configuration without downstream blowing, called DRL-noUS elsewhere.

Supplementary Figure 4 : Total-drag convergence and finite-time sampling uncertainty. (A) Cases without background suction. (B) Cases with uniform suction at

0.2\%\,U_{\infty}

. Columns show (left) segment-averaged

C_{d}

, (middle) duration-weighted backward means, and (right) retained arithmetic segment means. In the middle column, dashed and solid curves identify excluded and retained regions. Vertical bars show the retained weighted mean plus and minus one weighted population standard deviation. In the right column, open markers show arithmetic means, capped dashed bars give approximate 95% confidence intervals allowing for estimated positive serial correlation, and shaded rectangles show the observed segment ranges. Only complete segments starting at elapsed

t\geq 6

are retained. Time is measured from each case’s first available segment.

Supplementary Figure 5 : Aerodynamic changes relative to the common uncontrolled Base Flow. Panels show signed changes in lift (

C_{l}

, labelled

C_{L}

in the plot), friction drag, pressure drag, total drag, and lift-to-drag ratio, using

\Delta(X)

defined in the text. Circles and diamonds denote cases without and with background suction at

0.2\%\,U_{\infty}

, respectively. Open markers give the estimated changes. Horizontal bars give approximate individual 95% confidence intervals incorporating sampling uncertainty in both case and reference means. Dashed vertical lines mark zero change. The analysis retains complete segments starting at elapsed

t\geq 6

flow-over time units.

Supplementary Figure 6 : OC parameter sensitivity with background suction at

0.2\%\,U_{\infty}

. Local NES (top) and negative ZNMF power-input ratio (bottom), referenced to BASE-US. The five tested pairs

(y_{n}^{+},\alpha)

are

(15,0.5)

(15,0.25)

(15,1)

(7.5,1)

, and

(22.5,1)

. Shading identifies

0.25\leq x_{\mathrm{ss}}/c\leq 0.86

. Legend values are arithmetic station means of NES over this interval. The power-input panel uses a symmetric-logarithmic scale.

Configuration	$y^{+}_{n}$	$\mathrm{PR}_{\mathrm{TCF},\tau}$ $[\%]$	$\mathrm{PIR}^{(p)}_{\mathrm{TCF},\tau}$ $[\%]$	$\mathrm{PIR}^{(v)}_{\mathrm{TCF},\tau}$ $[\%]$	$\mathrm{NES}_{\mathrm{TCF},\tau}$ $[\%]$
OC	5	10.76	0.14	$\sim 0.0$	10.62
DRL	5	19.32	0.29	$\sim 0.0$	19.03
OC	15	25.00	0.29	0.10	24.61
DRL	15	31.40	0.49	0.07	30.85
OC	25	-38.57	5.09	6.11	-49.78
DRL	25	11.31	0.09	$\sim 0.0$	11.22

Appendix

Supplementary Table 5 : Sensing-height sensitivity in TCF-1. PR, pressure-work and kinetic-energy input ratios, and NES for OC and DRL trained and evaluated at the indicated

y_{n}^{+}

. Means use six initial conditions after excluding

t^{+}<500

. Bold entries identify the higher NES at each height. Kinetic-energy input ratios below

10^{-4}\%

are shown as

\sim 0

Supplementary Figure 7 : Single-component feedback in TCF-1 at $y_{n}^{+}=15$ . (A) Trajectory-conditioned mean wall actuation after the GLL-weighted ZNMF correction, plotted against the observed velocity component: tangential-only DRL (red), normal-only DRL (blue), and OC (black dashed). (B) Direct outputs of the same controllers on a synthetic $(u_{t}^{+},v_{n}^{+})$ input mesh, with common wall-action limits $[-1,1]$ . These maps evaluate the feedback laws before the instantaneous spatial ZNMF correction. All velocities use the common reference friction-velocity scale. The DRL policies in this ablation were trained with PR as the reward.

Evaluation	Learning	Configuration	$\mathrm{PR}_{\mathrm{TCF},\tau}$ $[\%]$	$\mathrm{PIR}^{(p)}_{\mathrm{TCF},\tau}$ $[\%]$	$\mathrm{PIR}^{(v)}_{\mathrm{TCF},\tau}$ $[\%]$	$\mathrm{NES}_{\mathrm{TCF},\tau}$ $[\%]$
TCF-BDF-3	–	OC	22.72	0.18	0.06	22.48
TCF-BDF-3	TCF-BDF-3	DRL	28.80	0.41	0.06	28.33
TCF-BDF-3	TCF-BDF-1	DRL	24.38	0.40	0.05	23.92

Appendix

Supplementary Table 6 : Policy transfer between two channel surrogates. All policies are evaluated in TCF-BDF-3. Learning identifies the training environment. PR, pressure-work and kinetic-energy input ratios, and NES are deterministic means over six initial conditions after excluding

t^{+}<500

. Bold identifies the highest NES. The TCF-BDF-1 policy is transferred without retraining.

Case	Block	$x_{\rm ss}/c$	$\langle Re_{\theta}\rangle_{x}$	$\langle H_{12}\rangle_{x}$	$\langle\beta\rangle_{x}$	Target	Value	Deviation ( $\leq 5\%$ )	Name
BASE-US $0.2\%\,U_{\infty}$	$\Omega_{1}$	$0.25$ – $0.65$	467	1.64	0.51	$H_{12}$	1.65	0.6%	TCF-BDF-1
–	$\Omega_{2}$	$0.65$ – $0.86$	828	1.69	2.08	–	1.64	3.0%	TCF-BDF-4

Appendix

Supplementary Table 7 : Targets for the two-block wing-control configuration. Wing properties are averaged over each listed interval. The Target and Value columns identify the matched quantity,

H_{12}

, and its value in the selected channel surrogate. The reported deviations compare the surrogate and wing shape factors. The

5\%

criterion does not refer to all listed boundary-layer properties.

Supplementary Figure 8 : Sensitivity to the chordwise partition with background suction at

0.2\%\,U_{\infty}

. Local NES (top) and negative ZNMF power-input ratio (bottom), referenced to BASE-US. Solid red, dashed red, and black curves denote four-block DRL, two-block DRL, and OC, respectively. Shading and vertical lines mark the two-block intervals,

0.25

–

0.65

and

0.65

–

0.86

. Four-block DRL is the configuration used in the main results. The lower panel uses a symmetric-logarithmic scale.

Case	Method	$C_{l}$	$C_{d,f}$	$C_{d,p}$	$C_{d}$	$L/D$
Base	LES	0.861	0.0127	0.0087	0.0214	40.2
	RANS	0.848 (-1.5%)	0.0113 (-11.0%)	0.0079 (-9.2%)	0.0192 (-10.3%)	44.1 (+9.7%)
US $0.2\%\,U_{\infty}$	LES	0.925	0.0140	0.0066	0.0206	44.9
	RANS	0.906 (-2.1%)	0.0128 (-8.6%)	0.0056 (-15.2%)	0.0183 (-11.2%)	49.5 (-11.2%)

Appendix

Supplementary Table 8 : RANS and LES integral aerodynamic quantities. Lift, friction drag, pressure drag, total drag, and lift-to-drag ratio are compared for the uncontrolled and uniform-suction cases. Parentheses report signed RANS deviations from the corresponding LES reference, using

100(\mathrm{RANS}-\mathrm{LES})/\mathrm{LES}

Supplementary Figure 9 : RANS and LES suction-side boundary-layer properties. Skin-friction coefficient, shape factor, and momentum-thickness Reynolds number for NACA4412 at

Re_{c}=200{,}000

and an angle of attack of

5^{\circ}

. Blue and red denote LES and RANS. Solid and dashed curves denote the uncontrolled and uniform-suction (

0.2\%\,U_{\infty}

) cases. The wall-shear stress is normalized by the free-stream dynamic pressure. Shading marks

\Omega_{1}

–

\Omega_{4}

Case	Block	$x_{\rm ss}/c$	$\langle Re_{\theta}\rangle_{x}$			$\langle H_{12}\rangle_{x}$
			LES	RANS	Deviation [%]	LES	RANS	Deviation [%]
Base flow	$\Omega_{1}$	$0.25$ – $0.40$	373	334	-10.4%	1.699	1.843	+8.5%
	$\Omega_{2}$	$0.40$ – $0.50$	525	478	-8.9%	1.688	1.820	+7.8%
	$\Omega_{3}$	$0.50$ – $0.70$	736	666	-9.4%	1.717	1.827	+6.4%
US $0.2\%\,U_{\infty}$	$\Omega_{1}$	$0.25$ – $0.40$	353	320	-9.4%	1.648	1.779	+7.9%
	$\Omega_{2}$	$0.40$ – $0.50$	462	428	-7.3%	1.632	1.726	+5.8%

Appendix

Supplementary Table 9 : Block-mean boundary-layer properties from RANS and LES.

Re_{\theta}

and

H_{12}

are averaged over the listed suction-side intervals. Signed deviations from the corresponding LES values are computed as

100(\mathrm{RANS}-\mathrm{LES})/\mathrm{LES}

and expressed in percent.

Supplementary Figure 10 : Mean tangential velocity in local viscous units. (A) Wing-section cases without background suction (noUS). (B) Wing-section cases with uniform suction at

0.2\%\,U_{\infty}

(US). Rows show the background without feedback, OC, and DRL, from top to bottom. Columns compare the indicated nominal wing stations with their channel surrogates, covering

\Omega_{1}

–

\Omega_{3}

in (A) and

\Omega_{1}

–

\Omega_{4}

in (B). Dashed and solid curves denote channel and wing-section results, respectively. Velocity and wall-normal distance are expressed in viscous units based on the local friction velocity of each case. DRL in (A) is applied without downstream blowing.

Supplementary Figure 11 : Reynolds stresses in local viscous units. (A) Wing-section cases without background suction (noUS). (B) Wing-section cases with uniform suction at

0.2\%\,U_{\infty}

(US). Rows show the background without feedback, OC, and DRL, from top to bottom. Columns compare the indicated nominal wing stations with their channel surrogates, covering

\Omega_{1}

–

\Omega_{3}

in (A) and

\Omega_{1}

–

\Omega_{4}

in (B). Dashed and solid curves denote channel and wing-section results, respectively. Blue, orange, green, and red denote

\overline{u_{t}^{2}}^{+}

\overline{v_{n}^{2}}^{+}

\overline{w^{2}}^{+}

, and

\overline{u_{t}v_{n}}^{+}

, respectively. Stresses and wall-normal distance use the local viscous scales of each case. DRL in (A) is applied without downstream blowing.

Supplementary Figure 12 : Premultiplied spanwise spectra of tangential-velocity fluctuations. (A) Wing-section cases without background suction (noUS). (B) Wing-section cases with uniform suction at

0.2\%\,U_{\infty}

(US). Rows show the background without feedback, OC, and DRL, from top to bottom. Columns compare the indicated nominal wing stations with their channel surrogates, covering

\Omega_{1}

–

\Omega_{3}

in (A) and

\Omega_{1}

–

\Omega_{4}

in (B). Spanwise wavelength and wall-normal distance are expressed in local viscous units for each case.

Supplementary Figure 13 : Normalized spanwise two-point correlations of tangential-velocity fluctuations. (A) Wing-section cases without background suction (noUS). (B) Wing-section cases with uniform suction at

0.2\%\,U_{\infty}

(US). Rows show the background without feedback, OC, and DRL, from top to bottom. Columns compare the indicated nominal wing stations with their channel surrogates, covering

\Omega_{1}

–

\Omega_{3}

in (A) and

\Omega_{1}

–

\Omega_{4}

in (B).

Supplementary Figure 14 : Temporal cross-correlations between wall-shear and tangential-velocity fluctuations. (A) Wing-section cases without background suction (noUS). (B) Wing-section cases with uniform suction at

0.2\%\,U_{\infty}

(US). Rows show the background without feedback, OC, and DRL, from top to bottom. Columns compare the indicated nominal wing stations with their channel surrogates, covering

\Omega_{1}

–

\Omega_{3}

in (A) and

\Omega_{1}

–

\Omega_{4}

in (B).

Parameter	Value
Learning algorithm	TD3
Actor network architecture	Fully connected (8 neurons)
Critic network architecture	Fully connected (16–64–64 neurons)
Number of input features ( $N_{\rm feat}$ )	2 (i.e. $u^{\prime+},\,v^{\prime+}$ )
Replay buffer size ( $\mathcal{D}$ )	Scaled with grid resolution
Mini-batch size ( $\mathcal{B}$ )	Scaled with grid resolution

Appendix

Supplementary Table 10 : Summary of the main DRL hyperparameters used in the present MARL framework. See Table 11 for the batch size and replay-buffer size employed in each configuration.

Name	$N_{\rm penv}$	$\mathcal{B}$	$\mathcal{D}$
TCF-1	4,410	2,048	$2\times 10^{7}$
TCF-2	4,410	2,048	$2\times 10^{7}$
TCF-3	10,890	4,096	$1\times 10^{8}$
TCF-BDF-1	4,410	2,048	$2\times 10^{7}$
TCF-BDF-2	4,410	2,048	$2\times 10^{7}$
TCF-BDF-3	10,890	4,096	$1\times 10^{8}$

Appendix

Supplementary Table 11 : Training configurations in the channel flows.

\mathcal{B}

and

\mathcal{D}

denote the mini-batch size and replay-buffer capacity, respectively.

RL Control

Reinforcement learning control uses reward-driven learning to make decisions in dynamic systems, from robots and vehicles to energy and infrastructure.

See all 272 papers

Jul 14, 2026

Gradient-free learning of a closed-loop wall controller for turbulent drag reduction

Closed-loop wall control learnt by multi-agent reinforcement learning can lower skin-friction drag in turbulent channels, but these gradient-based policies are trained on small periodic boxes and exhibit reduced performance when carried over to a larger domain. We recently showed that such policies are also prone to saturated bang-bang actuations that collapse into standing streamwise waves whose scale is set by the computational box rather than by the near-wall cycle, and proposed architectural fixes that avoid these degeneracies. Here, we employ Evolution Strategy (ES) to optimise a recurrent closed-loop controller directly on a large turbulent channel at

\mathit{Re}_τ\simeq180

, evaluating policy performance over full flow episodes using an energy-aware criterion and processing candidate policies in parallel. To our knowledge, this is the first application of an evolution strategy to the control of a turbulent flow. The ES controller reduces the skin friction by about

26\%

, exceeding the gradient-based multi-agent controller of Cavallazzi et al. (2026), GRU-MARL, trained on a minimal box (

17\%

), and marginally exceeding classic opposition control (OC,

22\%

). A wall-normal decomposition of the friction, Reynolds-stress profiles and anisotropy invariants show that the ES and opposition-controlled flows follow separate trajectories through the buffer layer, reaching comparable drag reduction by different reorganisations of the near-wall turbulence. In particular, the ES actuation correlates predominantly with the streamwise velocity fluctuations rather than with the wall-normal velocity that classical OC targets.

Giorgio Maria Cavallazzi, Miguel Pérez Cuadrado, Alfredo Pinelli

Department of Engineering, City St. George’s, University of London, Northampton Square, EC1V 0HB, London, UK

Computational Fluid Dynamics

Ask PDF

May 31, 2026

Explainable deep reinforcement learning reveals energy-efficient control strategies for turbulent drag reduction

We propose a method combining Multi-Agent Deep Reinforcement Learning (MARL) and eXplainable Deep Learning (XDL) to reduce drag in wall-bounded turbulent flows. Taking as a baseline the results of training agents directly targeting wall-shear stress and opposition control, three SHAP-guided approaches are compared. In the first, the reward is computed from SHAP attributions of a U-net predicting the future velocity field; in the second, from SHAP attributions of a U-net predicting the skin-friction coefficient; in the third, from a combination of SHAP attributions of two U-nets predicting the skin-friction coefficient and the wall pressure fluctuations, respectively. The combined SHAP strategy based on skin-friction coefficient and wall-pressure fluctuations achieves the best overall performance, achieving a DR of 34.44% and a NES of 34.01% with only 0.43% normalized input power. Relative to opposition control, drag reduction and net energy saving increase by 49.41% and 48.52%, respectively. Compared with the direct wall-shear-stress baseline, the proposed strategy simultaneously improves performance while reducing the normalized actuation cost from 5.90% to 0.43%. Analysis of the results reveals that the energetically efficient policy is consistent with pressure-gated actuation, activating predominantly at near-zero wall pressure, and operates on a temporal timescale comparable to the lifetime of the near-wall turbulent structures.

Federica Tonti, Ricardo Vinuesa

Department of Aerospace Engineering University of Michigan Ann Arbor, MI 48109, United States

RL Control Reinforcement Learning Interpretability in RL Computational Fluid Dynamics

Ask PDF

Dec 19, 2025

The HydroGym Reinforcement Learning Platform for Fluid Dynamics

Modeling and controlling fluids is critical across science and engineering. Effective flow control can increase lift, reduce drag, enhance mixing, and attenuate noise, potentially unlocking new technologies. Yet controlling fluids is hard: the dynamics are high-dimensional, nonlinear, and multiscale. While reinforcement learning (RL) has recently succeeded in robotics and protein folding through shared benchmarks, fluid dynamics has resisted such progress: each controller is typically tuned to a single geometry and operating point, making results hard to accumulate, transfer, and compare. We introduce HydroGym, a solver-independent RL platform for flow control, and show that standardized infrastructure unlocks transferable control intelligence across flow regimes. HydroGym provides 61+ validated environments spanning laminar to turbulent flows, with systematic Reynolds number progressions up to Re=400,000 and Mach number variations in 2D and 3D. It supports diverse backends, including finite-volume, spectral-element, finite-element, lattice-Boltzmann, and fully differentiable solvers for gradient-enhanced optimization. Across environments, RL agents consistently discover robust control principles, such as boundary-layer manipulation, acoustic-feedback disruption, and wake reorganization, yielding drag reductions exceeding 90% in canonical configurations. Critically, we demonstrate zero-shot transfer: agents trained only on a simplified channel flow achieve 38% friction-drag reduction on an unseen 3D wing section at chord Reynolds number Re=200,000 reducing exploration costs by four orders of magnitude versus direct on-wing optimization. This suggests RL agents uncover essential physics rather than configuration-specific patterns, pointing toward generalizable control. HydroGym offers extensible, scalable community infrastructure for fluid dynamics, machine learning, and control research.

Christian Lagemann, Sajeda Mokbel, Miro Gondrum +18

Department of Mechanical Engineering, University of Washington, Seattle, WA 98195, United States · AI Institute in Dynamic Systems, University of Washington, Seattle, WA 98195, United States · Chair of Fluid Mechanics and Institute of Aerodynamics, RWTH Aachen University, Aachen, Germany +9

RL Control RL Benchmarks Reinforcement Learning Computational Fluid Dynamics

Ask PDF

Abstract

RL Control

Explore similar work

Gradient-free learning of a closed-loop wall controller for turbulent drag reduction

Explainable deep reinforcement learning reveals energy-efficient control strategies for turbulent drag reduction

The HydroGym Reinforcement Learning Platform for Fluid Dynamics