Organizations: Department of Mechanical Engineering, College of Design and Engineering, and Advanced Robotics Centre, National University of Singapore, Singapore. · Department of Electronic and Information Engineering, School of Engineering, Westlake University, Hangzhou, China. · Department of Electrical and Computer Engineering, National University of Singapore, Singapore.
Does effective swimming with multiple underactuated soft arms require explicit coordination between arms? We hypothesize that an eight-arm swimmer can control its swimming speed and direction by independently learning each arm's actuation amplitude under a shared rhythmic drive, without prescribed inter-arm phase coupling. To test this hypothesis, we compare three reinforcement learning-based control strategies using proximal policy optimization (PPO): (1) learning arm-specific actuation amplitudes without inter-arm phase coupling, (2) learning actuation amplitudes with prescribed inter-arm phase coupling, and (3) directly learning actuator commands through end-to-end PPO without an explicit rhythmic structure. Across 200 episodes with randomized three-dimensional velocity commands, the controller without phase coupling satisfies the tracking criterion in 169 episodes, compared with 117 for the phase-coupled controller and 28 for end-to-end PPO. Straight-line swimming and commanded direction-change trials further demonstrate more accurate velocity tracking without phase coupling. These findings suggest that effective eight-arm swimming does not require prescribed inter-arm phase coupling, but can instead be achieved through learned arm-specific amplitude modulation within a rhythmic control structure.
Figures & tables
Fig. 1: Non-coordinated PPO for an underactuated eight-arm underwater robot. (a) Idealized morphology, multibody realization, and a bent-arm schematic. The orange motor symbol marks the actuated base hinge; ten rigid links and nine passive hinges approximate arm compliance. (b) PPO learning and closed-loop control. Observations from MuJoCo drive the policy, while rewards guide updates to the policy and value networks. The policy modulates a shared Hopf oscillator and the actuator mapping. (c) Nine-dimensional policy modulation consists of one common frequency f and eight arm-specific amplitude gains A1,…,A8 . The frequency drives a single oscillator, whose common waveform is distributed to all arm channels. Each gain scales its channel before a fixed control midpoint and clipping map it to a bounded actuator input ui . The hydrodynamics inset in (b) is a conceptual illustration generated using OpenAI’s image-generation tool.
Parameter
Value
Central plate radius / thickness
30 / 12 mm
Arm count / angular spacing
8 / 45∘
Arm base-to-tip reach
Approximately 160 mm
Segment count / segment length
10 per arm / 15 mm
Proximal segment width / thickness
20 / 10 mm
Distal segment width / thickness
10 / 5 mm
TABLE I: Geometry and joint parameters of the simulated underactuated underwater robot
Fig. 2: Velocity-tracking performance over 200 randomly sampled commands from the training target-velocity domain. (a) Distribution of the error between the commanded velocity and the measured mean velocity over the final 2 s of each trial. Colored dotted lines mark controller-specific medians; the black dashed line indicates the success threshold of 0.012m/s . (b) Tracking success rate grouped by commanded speed ∥v∗∥2 . Overall success rates are shown in the legend.
Fig. 3: Responses to a constant world-frame velocity command v∗=[0,0,0.075]Tm/s over a 15-s trial. (a) Signed velocity component projected onto the commanded direction; the dashed red line denotes 0.075m/s . (b) Body tilt φ , defined as the angle between the body and world +z axes. This angle measures tilt relative to the world vertical and does not capture rotation about that axis.
Fig. 4: Straight-line swimming comparison for (a) Non-coordinated PPO, (b) Coordinated PPO, and (c) Direct PPO. Robot configurations are overlaid at 2.5-s intervals from t=0 to 15 s. The red line is the reference trajectory obtained by integrating the constant velocity command, and the green line is the measured trajectory of the central platform origin.
Fig. 5: Commanded 90 ∘ velocity-direction transition in the world y – z plane. The command points along +z for the first 5 s, rotates toward +y over the next 1 s, and then remains along +y . (a)–(c) show Non-coordinated PPO, Coordinated PPO, and Direct PPO. Red and green curves denote the reference path and central-platform trajectory, respectively; robot snapshots illustrate changes in configuration. The command does not prescribe a 90 ∘ body-yaw rotation.
Fig. 6: Trajectory components during the commanded velocity-direction transition: (a) world-frame z position and (b) world-frame y position. Red dashed curves indicate the integrated reference velocity command, and shading identifies the 5–6-s command-transition interval.
Fig. 7: Three-dimensional swimming demonstration with Non-coordinated PPO with red and green arrows denote the commanded and measured world-frame velocities, respectively. Prospective interface in which circular joystick motion changes the lateral command direction while another input sets the upward component. The gamepad inset is a conceptual illustration generated using OpenAI’s image-generation tool.
We propose a new method for synthesizing physically-based swimming motions. Physically-based character animation aims to generate physically valid, controllable, and natural-looking motions which can respond to unexpected disturbances, where one dictating factor of difficulty is the complexity of the task, especially the level of sophistication of the required interactions with the environment. Existing research has succeeded in various tasks in static and dynamic environments. We push the difficulty further to swimming, which requires full-body coordination and continuous interactions with fluids, a new level of complexity when it comes to interacting with the environment. This complexity imposes challenges in learning control under volatile environmental forces, generalizing control to different environments and swimming styles, lack of data references, and prohibitively slow physical simulation which is inevitable during control learning. To this end, we propose SWIM, a new imitation method for swimming motions, which can learn from a single swimming motion and generalize to unseen environments, body conditions, and swimming styles. Extensive evaluation and comparison demonstrate that SWIM is data-efficient, stable, robust, and generalizable, outperforming alternative methods across multiple classes of tasks and metrics.
Octopus crawling motivates soft robots that exploit redundancy, yet discovering and organizing diverse coordination modes for adaptation remains challenging. To address this, we introduce a Diffusion-based Uncertainty-aware Optimization (DUO) algorithm that learns demonstration-free crawling controllers for a simulated, muscle-actuated CyberOctopus. This work represents the first application of diffusion-based control to soft multi-arm robots in contact-rich simulations. By embedding a variety of locomotion behaviors within a shared control distribution, this approach enables the simulated octopus to navigate dynamic physical constraints, demonstrating that learned coordination diversity inherently facilitates robust adaptation. The main contributions include: (i) a symmetry-structured policy representation that folds radially equivalent controllers into a canonical directional sector, (ii) an online black-box optimization strategy, the DUO algorithm, that discovers and retains diverse coordination modes, and (iii) a control editing technique that adapts existing controllers to novel actuator constraints without retraining. These results show how learned coordination diversity makes motor abundance a practical resource for adaptation in soft multi-arm robots.
Seung Hyun Kim, Heng-Sheng Chang, Kimia Kazemi +2
Mechanical Science and Engineering, University of Illinois Urbana-Champaign · Coordinated Science Laboratory, University of Illinois Urbana-Champaign · Carl R. Woese Institute for Genomic Biology, University of Illinois Urbana-Champaign +1
Soft robots are valuable robophysical platforms for studying body-caudal undulatory locomotion, but their compliant bodies are difficult to control precisely under changing hydrodynamic loading. Conventional proportional-integral-derivative (PID) feedback stabilizes periodic undulation in static water, but can accumulate flow-dependent tracking delay and increasing inter-trial variability when environmental flow becomes non-trivial. Here, we evaluate whether augmenting PID control with a Linear Repetitive Learning Estimation Scheme (PID-LRLES) recovers tracking accuracy and repeatability under dynamic flow. The LRLES generalizes classical integral action from constant to periodic, non-constant references, while using a stable transfer-function realization whose poles have negative real parts to avoid the long-term instability issues of classical repetitive control. Closed-loop experiments were carried out in a recirculating flow tank at five bulk flow speeds spanning 0 to 32.6 cm s^-1, using an embedded soft capacitive bending sensor at a 1 kHz control-loop rate. With controller gains tuned once in static water and then held fixed across all conditions, PID-LRLES tracked the periodic bending-envelope reference more closely than the PID baseline and significantly reduced the inter-trial spread of the per-trial RMSE (paired Wilcoxon signed-rank test, p = 1.8 x 10^-4, n = 25). Embedded soft proprioception and cycle-to-cycle learning act as complementary contributors to robustness: the sensor exposes the periodic hydrodynamic bias in body deformation, while the learning term absorbs it over recent oscillation cycles. By reducing flow-dependent control-induced variability, the approach provides an enabling layer for future robophysical studies seeking to isolate the effects of morphology, sensing, and environmental flow on aquatic locomotion.
Fabian Schwab, Federico Allione, Bingcheng Wang +5
1Engineering Sciences Department, Swiss Federal Laboratories for Materials Science and Technology, Switzerland · Institute for NeuroInformatics, University of Zurich, Switzerland · Department of Electronic Engineering, University of Rome Tor Vergata, Italy