physics.flu-dynApr 14, 2026
SaveLearning step-level dynamic soaring in shear flow
Organizations: J.C.Wu Center for Aerodynamics, School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai 200240, PR China
Abstract
Dynamic soaring enables sustained flight by extracting energy from wind shear, yet it is commonly understood as a cycle-level maneuver that assumes stable flow conditions. In realistic unsteady environments, however, such assumptions are often violated, raising the question of whether explicit cycle-level planning is necessary. Here, we show that dynamic soaring can emerge from step-level, state-feedback control using only local sensing, without explicit trajectory planning. Using deep reinforcement learning as a tool, we obtain policies that achieve robust omnidirectional navigation across diverse shear-flow conditions. The learned behavior organizes into a structured control law that coordinates turning and vertical motion, giving rise to a two-phase strategy governed by a trade-off between energy extraction and directional progress. The resulting policy generalizes across varying conditions and reproduces key features observed in biological flight and optimal-control solutions. These findings identify a feedback-based control structure underlying dynamic soaring, demonstrating that efficient energy-harvesting flight can emerge from local interactions with the flow without explicit planning, and providing insights for biological flight and autonomous systems in complex, flow-coupled environments.
Figures & tables
Figure 1: Problem formulation and deep reinforcement learning framework for autonomous dynamic soaring. (A) Three-dimensional trajectory of the navigation task. (B) The point-mass glider model [ 14 ] . The egocentric frame denotes heading, left-wing, and up directions. , , and represent airspeed, ground velocity, and wind velocity. The aerodynamic states are defined by pitch , heading , and bank angle [ 41 ] . , , and denote lift, drag, and weight. and denote the wind direction and the target direction relative to the wind in the inertial (Earth-fixed) frame. (C) Horizontal projections of representative tailwind, crosswind, and headwind trajectories, illustrating the characteristic zig-zag motion [ 11 ] . The circular regions indicate the randomized initial positions and the target success regions. (D) Deep reinforcement learning (DRL) framework. The agent interacts with the wind environment through observations and outputs continuous control actions . (E) Parameter space of wind conditions. Grey shading indicates available energy ( , subsection 4.1 ). Shaded regions denote prior studies: blue [ 16 ] , yellow [ 38 ] , and green this work. The red line marks the minimum wind-speed boundary [ 18 ] (left region infeasible), and the magenta dotted line indicates the theoretical perceptual resolution limit . (F) Logistic wind profiles. (G, H) Training success-rate curves under different observation settings ( Table 1 ) and reward formulations ( Table 2 ). Curves are averaged over five random seeds, with shaded regions indicating standard deviation. (I, J) Success rate as a function of , and , where white dashed contours denote the SR boundary. Statistical definitions of success are provided in subsection 4.3 .
| No. | Setting | Obs. | Training SR | Test SR |
|---|---|---|---|---|
| E1 | full | |||
| E2 | No shear | |||
| E3 | ground speed | |||
| E4 | polar wind | |||
| G1 | geocentric | |||
| E0 | No wind |
Table 1: Observation ablation. This table summarizes the observation configurations used in the sensory ablation study. All policies are trained with the same reward formulation (Rwd.1 in Table 2 ). Success-rate definitions are provided in subsection 4.3 . E1 defines the full egocentric observation set. Removing shear information (E2) or replacing airspeed with groundspeed (E3) leads to moderate performance changes, while the polar representation (E4) significantly degrades learning. The geocentric formulation (G1) also shows reduced performance compared to the egocentric baseline. Configurations without wind (E0) or without airspeed information (E0’) fail to learn meaningful policies. For some settings, curriculum learning does not cover the full – range ( subsection 4.2 ): E0 converges within – , E0’ fails beyond – , and E4 converges within – . Reported success rates are computed over these respective ranges.
| No. | Training SR | Test SR | |
|---|---|---|---|
| 1 | |||
| 2 | |||
| 3 | |||
| 4 |
Table 2: Process reward ablation. Training and test success rates under different reward formulations. All policies use the same observation space (Obs.E1 in Table 1 ). The combined reward (Rwd.1) achieves the best performance, while the directional-progress term alone (Rwd.3) yields comparable results. In contrast, the energy-based term alone (Rwd.2) fails to produce a robust policy. State-based rewards (Rwd.4) achieve moderate performance but remain inferior to process-based formulations.
Figure 2: Emergence of a two-phase dynamic-soaring navigation strategy governed by kinetic-energy management. (A–D) Time evolution of key variables along a representative crosswind trajectory ( Figure 1 A): (A) airspeed , ground-directed velocity , and altitude ; (B) total energy , kinetic energy , and potential energy ; (C) pitch angle and heading angle ; (D) control actions and . The grey line indicates the transition time ( subsection 4.3 ), separating the dynamic-soaring (DS) and targeted-gliding (TG) phases. (E–H) Statistical characterization across : (E) two-phase significance ratio (SiR, subsection 4.3 ); (F) transition time ( subsection 4.3 ); (G) airspeed at transition ; (H) altitude at transition . (I–P) Energy and navigation performance in the two phases. (I–J) net change in total energy ; (K–L) net change in kinetic energy ; (M–N) net change in potential energy ; (O–P) mean directional velocity . Panels (I, K, M, O) correspond to the DS phase, and (J, L, N, P) to the TG phase. Energy changes are computed relative to the transition time : and , with analogous definitions for and . is averaged over each phase. These results show that kinetic energy is accumulated during the DS phase and expended during the TG phase to enable goal-directed navigation.
Figure 3: Structured policy representation in observation space under a fixed condition ( , , ). Columns 1–2 show a representative successful trajectory colored by and , with the start, target, and DS–TG transition marked by a green circle, red circle, and red cross. Columns 3–4 show occupancy-filtered heatmaps from successful DS-phase trajectories (TG phase in Figure S3 ), retaining states with more than samples. Rows 1–3 correspond to relative position , velocity , and relative wind . Trajectories show temporal evolution, while heatmaps reveal consistent observation–action mappings. In , the state encodes distance and target direction , organizing navigation and the DS–TG transition. In , the auxiliary curve indicates the load-factor limit for . In , concentric circles at and indicate characteristic shear-layer and free-stream wind magnitudes. The final row shows coordinate mappings: (M, N) and (O, P) .
Figure 4: Robustness and generalization under out-of-distribution conditions. (A, C) Representative trajectory in a spatially varying wind field with coupled speed and shear variations. (B) Normalized spatial distribution of the harmonic disturbance field (defined in subsection 4.4 ). (D–F) Success-rate heatmaps under perturbed wind conditions, showing robust performance across variations in wind-direction scale , reference wind speed , and shear thickness . G Representative trajectories for navigation tasks. (H–I) Success-rate heatmaps for static targets and dynamic targets . (J) Robustness to observation noise with increasing intensity . The policy maintains high performance across environmental variability, task complexity, and sensing uncertainty.
Figure 5: Comparison of ground-speed envelopes and energy-direction trade-offs across learned, biological, and optimal strategies. (A–C) Ground-speed envelopes under different wind conditions. (A) RL policy predictions for in polar coordinates. (B) Experimental envelopes derived from biological flight data [ 10 ] , fitted using a generalized additive model [ 11 ] , with background shading indicating data density. (C) Numerically optimal envelopes obtained via IPOPT-based trajectory optimization [ 14 ] . (D–F) Joint distributions of energy-harvesting coefficient and directional-flight coefficient (defined in [ 14 ] ) for . Panels show (D) RL policy samples, (E) filtered experimental data, and (F) IPOPT solutions. Color maps indicate log-scaled density with overlaid contours. All three approaches exhibit a consistent trade-off structure between energy acquisition and directional progress, despite variability in experimental data.
| No. | Training SR | Test SR | ||
|---|---|---|---|---|
| 1 | ||||
| 2 | ||||
| 3 | ||||
| 4 |
Table S1: Training and test success rates for different neural network architectures. All configurations use the same observation space (Obs.E1, Table 1 ) and reward formulation (Rwd.1, Table 2 ). Each architecture is defined by the layer widths of the actor and critic networks.
Figure S1: Flight trajectories and state variables under representative wind conditions (corresponding to Figure 1 C, Figure 2 A–D). The figure presents agent performance across five scenarios: (row 1) tailwind target ( ); (row 2) headwind target ( ); (row 3) low wind speed ( ); (row 4) high wind speed ( ); and (row 5) thick shear layer ( ). Column 1 shows top-down trajectories in the – plane, color-coded by altitude . Column 2 presents kinematic variables. Column 3 shows flight angles. Column 4 displays control inputs.
Figure S2: Phase-wise performance metrics across wind-shear conditions ( and ; corresponding to Figure 2 E–P). (A) Two-phase significance ratio (SiR); (B) transition time ; (C) airspeed at transition ; (D) altitude at transition . (E–L) Energy and navigation metrics for the two phases. (E, F) net change in total energy ; (G, H) net change in kinetic energy ; (I, J) net change in potential energy ; (K, L) mean directional velocity . Panels (E, G, I, K) correspond to the DS phase, and (F, H, J, L) to the TG phase.
Figure S3: Policy distribution during the targeted gliding (TG) phase (corresponding to Figure 3 ). The left column ( A , C , E ) shows the bank angle , while the right column ( B , D , F ) shows the lift coefficient .
Figure S4: Evolution of learned state–action mappings across training. Heatmaps illustrate the policy-induced state–action distributions in multiple kinematic domains at four training checkpoints ( , , , and timesteps), arranged from left to right.
Figure S5: Trajectory tracking and policy heatmaps under tailwind and headwind conditions. Results are shown for the environmental setting and . The left two columns (A, B, E, F, I, J) correspond to a target direction , while the right two columns (C, D, G, H, K, L) correspond to . Columns 1 and 3 show representative successful trajectories, whereas columns 2 and 4 present aggregated state–action visitation heatmaps over successful episodes.
Figure S6: Trajectory tracking and policy heatmaps under high-wind and thick-shear conditions. Results are shown for a fixed target direction . The left two columns (A, B, E, F, I, J) correspond to with , while the right two columns (C, D, G, H, K, L) correspond to with . Columns 1 and 3 show representative successful trajectories, whereas columns 2 and 4 present aggregated state–action visitation heatmaps over successful episodes.
Figure S7: Training dynamics and observation-space-dependent performance across task and environmental conditions. (A–D) Training curves over timesteps (corresponding to Figure 1 G), including cumulative reward (A), timeout rate (B), crash rate (C), and steps per episode (D). Curves correspond to different observation-space designs ( Table 1 ); solid lines denote means and shaded regions indicate standard deviations. (E–L) Success-rate distributions evaluated within the training domain (corresponding to Figure 1 I, J). Columns correspond to Obs.E2 (E,I), Obs.E3 (F,J), Obs.E4 (G,K), and Obs.G1 (H,L). Colormaps represent success rate (SR, %), with dashed contours indicating regions where SR exceeds 90%.
Figure S8: Success-rate distributions and trajectory patterns under varying wind directions. (A, B) Contour maps of navigation success rate as a function of target direction ( ) and wind direction ( ) for egocentric (relative) observations (Obs.E1) (A) and geocentric (absolute) observations (Obs.G1) (B). Dashed contours denote the 90% success-rate boundary. (C–F) Representative trajectories projected onto the horizontal (East–North) plane under different wind directions with ( C ), ( D ), ( E ), and ( F ), all shown for Obs.E1.
Figure S9: Training dynamics and reward-dependent performance across task and environmental conditions. (A–D) Training curves over timesteps (corresponding to Figure 1 H), including cumulative reward (A), timeout rate (B), crash rate (C), and steps per episode (D). Curves correspond to different reward designs ( Table 2 ); solid lines denote means and shaded regions indicate standard deviations. (E–L) Success-rate distributions evaluated within the training domain. Columns correspond to Rwd.1 (E,I), Rwd.2 (F,J), Rwd.3 (G,K), and Rwd.4 (H,L). Colormaps represent success rate (SR, %), with dashed contours indicating regions where SR exceeds 90%.
Figure S10: Evolution and decomposition of the process reward during training. Results are shown for Obs.E1 with reward formulation Rwd.1 ( Table 2 ). A detailed definition of the process reward and its components is provided in subsection 4.2 .
Figure S11: Mission feasibility boundaries and limiting behaviors in static target-navigation tasks. (A–D) Performance landscapes over target direction and distance (corresponding to Figure 4 H), including success rate (A) , timeout rate (B) , terminal distance to the target at episode end (C) , and average episode length (D) . (E–H) Flight analysis for a successful boundary case ( ). (I–L) Analysis of a timeout case beyond the feasibility limit ( ). In (I–L), although sustained flight is maintained through repeated energy-harvesting cycles, the agent fails to progress toward the target due to limited exposure to such relative configurations during training (K), resulting in a quasi-stationary loitering pattern until termination.
Figure S12: Performances of dynamic target interception across velocity and heading regimes. (A–D) Performance landscapes over target velocity and heading (corresponding to Figure 4 I), including success rate (A) , timeout rate (B) , terminal distance to the target at episode end (C) , and average episode length (D) . (E–H) Successful interception of a fast downwind-moving target ( , ). (I–L) Interception failure for an identical target moving upwind ( , ). Although the agent is unable to close the distance, resulting in a timeout-driven chase, it continuously pursues the target. In (E,I) , the red arrow indicates the direction of target motion, with endpoints marking the initial and final positions.
Figure S13: Flight trajectories, wind estimation, and energy/direction-related coefficients from empirical seabird data. Representative samples are from the BiP_WA_CROZET2019_F0957_N18035_G dataset [ 10 ] . (A, D) Spatial trajectories of the flights. (B, E) Relationship between heading angle ( ) and horizontal ground speed ( ). Gray markers denote empirical data, while the solid red curve shows the fitted trigonometric model used to estimate wind speed ( ) and wind direction ( , indicated by vertical dashed blue lines), following the method of [ 50 ] . (C, F) Temporal evolution of the energy-harvesting rate ( , green) and the directional efficiency ( , purple) over the selected flight segment. Horizontal dashed lines indicate the corresponding time-averaged values, computed following [ 14 ] .
More in this topic
Feedback Control
Feedback control uses ongoing measurements to adjust actions, stabilize dynamical systems, track goals, and respond to disturbances and constraints.
See all 255 papers