Authors: Fei Han, Xinyu Cui, Zhipeng Wang, Ning Yang, Hang Xu, Haifeng Zhang, Zhongming Hu, Jun Wang, Junfeng Du, Dixia Fan
Organizations: College of Computer Science and Technology, Zhejiang University, Hangzhou, China · School of Engineering, Westlake University, Hangzhou, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Mechanical and Material Engineering, Queen’s University, Kingston, ON K7L 3N6, Canada · Department of Computer Science, University College London, London WC1E 6BT, United Kingdom · College of Engineering, Ocean University of China, Qingdao, China
Biological swimmers and flyers exploit unsteady vortices for propulsion, whereas engineered vehicles usually suppress them as disturbances. Learning such flow exploitation in machines is difficult because real-fluid interaction data are scarce and unstructured exploration is unstable in high-dimensional, history-dependent flows. Here we present REEF, a co-designed physical-learning framework that integrates SHOAL, an eight-channel high-throughput array for real fluid--structure interaction, with V-STAR, a staged algorithm that converts these interactions into policies through imitation, offline internalization, and online adaptation. Across lift-based, drag-based, and momentum-jet propulsors, REEF expands the attainable force envelope to more than twice that of parameterized search. Particle image velocimetry shows that these gains arise from coordinated vortex formation, growth, and force projection, rather than refinement of a fixed motion-to-force mapping. Force-trained policies transfer zero-shot to free-moving robots whose body motion changes the surrounding flow, suggesting that REEF learns transferable wake-coupling principles for embodied propulsion in unsteady fluids.
Figures & tables
Figure 1: Overview of REEF, a co-designed physical-learning ecosystem integrating the SHOAL experimental array with the V-STAR learning algorithm. (A) Conventional engineering suppresses vortices to reduce disturbances, whereas biological systems exploit unsteady flows for propulsion; REEF advances toward deliberate vortex generation and manipulation. (B) Three representative propulsors, including a 3-DOF flapper, 2-DOF webbed foot, and 1-DOF caudal fin, span distinct locomotion modes and force-generation regimes. (C) V-STAR converts parameterized trajectories into flow-compliant policies through three-stage algorithm (imitation, offline internalization, and online adaptation). (D) SHOAL enables parallel real-flow learning with an eight-channel towing array, rapid flow reset, synchronized actuation, and real-time six-axis force sensing.
Variant
Modification (baseline)
Reward
Std
t80% (h)
REEF
— (full pipeline)
113.5
±11.9
2.62
REEF-1channel
Single channel
105.4
±28.4
4.46
REEF-w/o-Seq
Transformer → MLP
099.9
±09.9
4.26
REEF-w/o-Phys
Physics-grounded → statistical (Uni-O4 [ 30 ] )
084.1
±24.3
—
REEF-w/o-Offline
No offline stage (imitation-init PPO)
075.9
±38.7
—
REEF-w/o-Imit
No imitation prior (PPO from scratch)
0−−8.2
±04.6
—
Table 1: Ablation performance of REEF variants on the 3-DOF flapper benchmark.
Figure 2: Each component addresses a distinct constraint. (A) Training reward of the full REEF pipeline and five ablation variants on the 3-DOF flapper benchmark. Curves show smoothed episode reward versus training time; shaded regions denote ±1 SEM across seeds. Dashed and solid segments indicate pre-training and online learning phases, respectively. Ablation configurations and equivalent baseline algorithms are defined in Table 1 . (B) Kinematic trajectories and hydrodynamic force evolution across the three V-STAR stages. Top: representative flapping-cycle trajectories with instantaneous force vectors and foil orientations. Bottom: thrust ( Cx ) and lateral-force ( Cy ) coefficients along the same cycle, with stage transitions and cycle-averaged thrust Cx annotated. Abbreviations: SEM, standard error of the mean.
Figure 3: Learned policies access force regimes beyond parameterized search through vortex orchestration. (A) Thrust–lift Pareto envelope of the 3-DOF flapper. Each point denotes the cycle-averaged force coefficients (Cx,Cy) of one policy from brute-force grid search (BF), evolutionary multi-objective optimization (EMOA) [ 10 ] , or REEF. Insets show representative REEF policies along the Pareto front, including thrust-dominated REEF1, balanced REEF2, and lift-dominated REEF3, together with the strongest EMOA representative, EMOA4. (B) Simultaneous PIV vorticity fields and force traces for REEF1, EMOA4, and REEF3 over representative cycles. Vorticity snapshots show wing motion and leading-edge vortex (LEV) evolution, while Cx(t) and Cy(t) traces mark the corresponding force peaks. Compared with EMOA4, REEF policies coordinate LEV formation, growth, and force projection with wing orientation, producing expanded thrust–lift performance.
Figure 4: REEF generalizes to drag-based and momentum-jet propulsion through morphology-specific vortex strategies. (A) Thrust–lift Pareto envelope of the 2-DOF webbed foot. Points denote cycle-averaged force coefficients (Cx,Cz) from BF, EMOA, and REEF; insets show representative trajectories. (B) Thrust–lift Pareto envelope of the 1-DOF caudal fin, with representative REEF and EMOA trajectories. (C) Force traces and PIV vorticity fields for webbed-foot paddling. Snapshots compare REEF at t=0.11,0.16,0.20 s with BF at t=0.29,0.34,0.38 s, showing that REEF retains an attached vortex while reorienting the paddle into a thrust-favorable posture. (D) Wake structure and momentum-jet profile for the caudal fin. REEF forms a focused downstream jet, quantified by the time-averaged streamwise velocity increment Δu along monitoring lines at c and 1.25c downstream of the trailing edge. Colorbars show spanwise vorticity; blue and red denote opposite signs.
Figure 5: Force-centric policies transfer zero-shot to locomotion across morphologies. (A) Quadrupedal flapper robot driven by SHOAL-trained REEF policies or the strongest parameterized representative (PARA) from the BF/EMOA pool. Image sequences show body displacement over time, with instantaneous streamwise force Fx below; bars report forward displacement of three REEF policies and corresponding PARA representatives as mean ±1 s.d. across three trials. (B) Quadrupedal webbed-foot robot under the same protocol. (C) Caudal-fin module tested in a circulating flow channel on near-frictionless air bearings; quadrupedal robots translate along streamwise guide rails with vertical spring compliance. Mechanical and gait-synchronization details are provided in Supplementary Note 5.
FSI lab, Westlake University, Hangzhou, 310030, China. · Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China. · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China +3
Department of Mechanical Engineering, University of Washington, Seattle, WA 98195, United States · AI Institute in Dynamic Systems, University of Washington, Seattle, WA 98195, United States · Chair of Fluid Mechanics and Institute of Aerodynamics, RWTH Aachen University, Aachen, Germany +9