RAFL: Generalizable Sim-to-Real of Soft Robots with Residual Acceleration Field Learning
Authors: Dong Heon Cho, Boyuan Chen
Organizations: Department of Computer Science, Duke University, 308 Research Dr., Durham, 27705, NC, USA · Department of Mechanical Engineering and Material Science, 121 Hudson Hall, Durham, 27708, NC, USA · Department of Electrical and Computer Engineering, 100 Science Dr., Durham, 27708, NC, USA
Differentiable simulators enable gradient-based optimization of soft robots over material parameters, control, and morphology, but accurately modeling real systems remains challenging due to the sim-to-real gap. This issue becomes more pronounced when geometry is itself a design variable. System identification reduces discrepancies by fitting global material parameters to data; however, when constitutive models are misspecified or observations are sparse, identified parameters often absorb geometry-dependent effects rather than reflect intrinsic material behavior. More expressive constitutive models can improve accuracy but substantially increase computational cost, limiting practicality. We propose a residual acceleration field learning (RAFL) framework that augments a base simulator with a transferable, element-level corrective dynamics field. Operating on shared local features, the model is agnostic to global mesh topology and discretization. Trained end-to-end through a differentiable simulator using sparse marker observations, the learned residual generalizes across shapes. In both sim-to-sim and sim-to-real experiments, our method achieves consistent zero-shot improvements on unseen morphologies, while system identification frequently exhibits negative transfer. The framework also supports continual refinement, enabling simulation accuracy to accumulate during morphology optimization.
Figures & tables
Fig. 1: RAFL overview. Our generalizable residual acceleration field framework is trained end-to-end to reduce error on recorded sparse marker trajectories, and is agnostic to mesh topology, allowing zero-shot transfer to unseen morphologies.
Fig. 2: RAFL pipeline overview. The network predicts element-level residual accelerations from local deformation/rate features, which are mass-scaled and accumulated via FEM quadrature to form residual forces. Trained end-to-end from sparse marker trajectories, the model transfers zero-shot across morphologies and discretizations.
Feature
Dimension
Description
\textscdiag(Σe)−1
3
Principal Stretches centered by 1
\textscflattened(FeTFe−I)
9
Right Cauchy Green Tensor centered by identity matrix
det(Fe)−1
1
Determinant of Fe centered by 1
\textscdiag(ReTDeRe)
3
Strain Rates along principal axes (co-rotated to rest frame)
Fig. 3: Example generalization trajectories for non-scaled geometries. SysID suffered from negative transfer, where the tuned simulation (i.e. blue curve) showed worse performance compared to the original simulation (i.e. red curve). Our framework showed consistent improvement in all generalization cases, and showed more qualitatively accurate oscillatory behavior within the trajectories.
Fig. 4: Sim-to-real trajectory visualization across beam geometries. The leftmost frame in each row shows the initial state from the recorded real-world video, with surface markers highlighted by a red bounding box. Subsequent panels show simulated frames over the first 0.4 seconds (sampled every 5 frames). Rows compare SysID (both) and our RAFL model. Blue spheres indicate measured marker positions from the real system, while red spheres indicate simulated marker positions.
Fig. 5: Expanded cross-shape generalization results. SysID frequently failed to transfer between morphologies, leading to negative transfer. In contrast, RAFL successfully demonstrated zero-shot accuracy improvement between meshes in most cases. RAFL also achieved better accuracy than SysID in most settings.
Fig. 6: Mean residual force norm per vertex. Residual forces collected for SRL are highly dependent on vertex position compared to those learned implicitly through RAFL.
Fig. A1: Mean marker errors of the generalization trajectories for non-scaled geometries. SysID suffers from negative transfer with greater mean marker error than the untuned simulation in the shorter and thicker beams.
Fig. A2: Final frames of representative generalization trajectories for non-scaled shorter and thicker geometries. Black arrows indicate the equilibrium configuration produced by the untuned simulator. Red arrows highlight negative transfer in SysID, where the equilibrium configuration deviates further from the real trajectory than the untuned simulation. Green arrows indicate successful zero-shot transfer with RAFL.