cs.RONov 9, 2026

Accelerated Reinforcement Learning for Soft Robotics: Hours Instead of Years

Authors: Roshan S. Kaundinya, Hehui Zheng, Robert K. Katzschmann, George Haller

Organizations: Institute for Mechanical Systems, ETH Zürich, Leonhardstrasse 21, 8092, Zürich, Switzerland. · Soft Robotics Lab, ETH Zürich, Tannenstrasse 3, 8092, Zürich, Switzerland.

Abstract

Reinforcement learning (RL) requires large amounts of interaction data or a reliable simulator, both hard to obtain for nonlinear soft robotic hardware. Recent work shows that a time-scale separation inherent to soft robot dynamics justifies their reduction to low-dimensional invariant manifolds, known as adiabatic spectral submanifolds (aSSMs). We demonstrate that an aSSM-reduced model, a simple polynomial ordinary differential equation learned purely from pre-collected data, can serve as an RL training environment. On a pneumatically actuated soft arm, we train a trajectory-tracking policy in 3 hours on a six-dimensional aSSM-reduced model and deploy it without fine-tuning, whereas reproducing this policy training extrapolates to 3 years of hardware interaction. Our policy tracks dynamic targets with 40% lower error than otherwise identical policies trained on a 4.8-million-parameter recurrent neural network (RNN) or a 1.7-million-parameter transformer, which take 13 and 100 hours to identify, against 5 minutes for our mathematically justified 6,924-parameter model.

Figures & tables

Explore similar work

CardsList
  1. Rapidly Learning Soft Robot Control via Implicit Time-Stepping

    Nov 10, 2025Andrew Choi, Dezhong Tong, Xiaonan HuangRobot Policy LearningRobotics Simulation

  2. Orientation Control of Soft Robots via Adiabatic Spectral Submanifolds

    Sep 13, 2026Aron Karakai, Roshan S. Kaundinya, Mike Yan Michelis +2Robotic ControlSoft Robotics

  3. Data-Driven Precision Control of Soft Helicoid Robots

    Nov 9, 2026Roshan S. Kaundinya, Mark Leone, Roberto Sabella +5Data-Driven ControlSoft Robotics