Accurate Open-Loop Control of a Soft Continuum Robot Using Visually Learned Latent Dynamics
Authors: Henrik Krauss, Johann Licher, Naoya Takeishi, Annika Raatz, Takehisa Yairi
Organizations: Department of Advanced Interdisciplinary Studies, The University of Tokyo, Tokyo, 153-8904, Japan. · Institute of Assembly Technology and Robotics, Leibniz University Hannover, 30823 Germany. · Research Center for Advanced Science and Technology, The University of Tokyo, Tokyo, 153-8904, Japan.
This work addresses open-loop control of a soft continuum robot (SCR) from video-learned latent dynamics. Visual Oscillator Networks (VONs) from previous work are used, which provide mechanically interpretable 2D oscillator latents through an attention broadcast decoder (ABCD). Open-loop, single-shooting optimal control is performed in latent space to track image-specified waypoints without camera feedback. An interactive SCR live simulator enables design of static, dynamic, and extrapolated targets and maps them to model-specific latent waypoints. On a two-segment pneumatic SCR, Koopman, MLP, and oscillator dynamics, each with and without ABCD, are evaluated on setpoint and dynamic trajectories. ABCD-based models consistently reduce image-space tracking error. The VON and ABCD-based Koopman models attain the lowest mean squared errors (MSEs). Ablation and simulation stress tests confirm that training and architecture choices support open-loop performance, including static holding, stable extrapolated equilibria, and relaxation to rest. To the best of our knowledge, this is the first demonstration of reliable long-horizon open-loop control of a physical soft robot using interpretable latent dynamics models learned solely from video.
Figures & tables
Fig. 1 : Overview of this study’s approach: an SCR live simulator is employed to generate target states for open-loop optimal control of the physical SCR. The feasibility of this method is validated across diverse trajectories.
Name
n
T
K
wQ
wQ˙
wQk
wQ˙k
wQf
wQ˙f
Setp. Normal
9
100
2
1
0.002
0
0
0
0
Setp. Extrap.
6
100
2
1
0.002
0
0
0
0
Dyn. Normal
5
200
8
1
0
0
0
0
0.002
Dyn. Fast
3
150
8
1
0
0
0
0
0.002
Dyn. Upswing
3
150
3
0
0
1
0.002
0
0
Dyn. Long
5
1000
22
1
0
0
0
0
0.002
TABLE I : Trajectory types and control parameters. n = number of trajectories tested.
Fig. 2 : (a) Model performances in open-loop control of real hardware over various trajectory types (image MSEs). (b) Mean absolute error (MAE) of input pressure prediction from open-loop setpoint control in simulation, compared to setpoints in the dataset, over multi-step prediction errors (50 samples each), including ablations.
Fig. 3 : (a) Hardware examples for trajectories evaluated for open-loop optimal control. The real trajectories shown are based on the VON model. All target trajectories are accurately achieved, except the upswing, where an additional plot highlights a deviation between set and measured pressure inputs. (b) Predicted and real latent trajectories and input pressures for the fast dynamic trajectory and upswing trajectory (c), depicted on the left.
Fig. 4 : Simulation stress tests. (a) Static holding. (b) Ramp-up to extrapolated pressures, with VON excitation vs. stiffness forces. (c) In-dataset range harmonic excitation then release. Top shows image MSEs and bottom final decoded observations.
Centre for Systems and Control Indian Institute of Technology Bombay Mumbai, India · School of Mechanical and Aerospace Engineering Nanyang Technological University Singapore