cs.ROOct 5, 2026

Physics Residual Dynamics and Reduced Order Whole-Body Planning for Obstacle Aware Human Robot Cloth CoTransportation

Authors: Moein Forouhar, Kosar Behnia, Anirvan Dutta, Hamid Sadeghian, Ville Kyrki, Sami Haddadin, Gokhan Alcan, Eckehard Steinbach

Organizations: Technical University of Munich (TUM), Germany · Tampere University, Finland · Aalto University, Finland · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Human--robot co-transportation of deformable objects requires predicting object deformation during motion, since obstacle clearance depends on both the grasp points and the unactuated interior. We present a hierarchical planning framework that combines a learned cloth model with a reduced-order whole-body model of a dual-arm mobile manipulator. A physics-residual conditional recurrent variational autoencoder (p-cRVAE) predicts the full cloth configuration from grasp-point observations by learning a residual correction to a computationally efficient linearized physics model, limiting error accumulation over 40-step planning horizon. The predicted cloth dynamics are embedded in a model predictive path integral (MPPI) planner using a reduced-order representation of a dual-arm mobile manipulator that preserves the non-holonomic base constraint and arm workspace limits. An MPC layer subsequently refines the sampled motion into smooth, executable references for whole-body control. The reduced-order formulation achieves tracking performance comparable to the full 17-DoF model while reducing computation time by approximately 80%. Across four co-transportation scenarios and two carrying speeds, the proposed framework maintains cloth-obstacle clearance where a corner-following baseline results in collisions, while whole-body refinement reduces final cloth deformation from 0.93,m to 0.28,m.

Figures & tables

Fig. 1 : Setup for human–robot co-transportation of a rectangular cloth in the presence of obstacles. The planner jointly uses a high-fidelity cloth model (p-cRVAE) with a reduced-order robot model (ROM) to generate feasible hand and base trajectories while following the user and avoiding cloth–obstacle collisions.
Fig. 2 : Structure of the proposed p-cRVAE. The coarse physics Φc\Phi^{c} advances the cloth state under the commanded grasp motion, and the residual decoder DθD_{\theta} predicts a per-particle correction conditioned on the belief state hk\mathbf{h}_{k} and the latent variable zk\mathbf{z}_{k} .
Fig. 3 : The performance of the MPC refinement layer is evaluated under the full model vs. The reduced-order model as prediction models for the planner. The full model considers the complete kinematic chains of both arms, while the reduced-order model approximates each arm with a simple link that captures only the corresponding end-effector position and not the arms configurations. The reference trajectory gk:k+Hr\mathbf{g}^{r}_{k:k+H} generated by the MPPI planner contains jerky motion. Both planners track the reference trajectory and provide a smoother reference to the whole-body controller.
Fig. 4 : Comparison of the planner performance between full and the reduced-order types. Both versions effectively track the reference gkr\mathbf{g}_{k}^{r} while smoothing its jerky motion. However, the optimizer using the full model has a 5x higher computational cost than the one using the reduced model.
Fig. 5 : Training data and open-loop prediction accuracy. ( 5(a) ) Grasp-point coordination patterns used to generate the dataset, shown as the settled and final sheet for a representative episode of each pattern. ( 5(b) ) Per-node prediction error over the rollout, with and without the coarse backbone (mean ±\pm one standard deviation over held-out episodes), and spatial error maps over the mesh on a shared scale. Teal markers denote the measured points.
Fig. 6 : Comparison of the MuJoCo simulations and corresponding real-robot experiments for the four human–robot co-transportation scenarios. In real experiment, the green shadow represents the current position of the cloth and red shadow shows the goal position.
Scene , vhv_{h}∑canc\sum c_{\mathrm{anc}} [ 10610^{6} ] ↓\downarrowdev T [cm] ↓\downarrow∑cact\sum c_{\mathrm{act}} [ 10310^{3} ] ↓\downarrowclr [cm] ↑\uparrow
 B-IB-IIProp.B-IB-IIProp.B-IB-IIProp.B-IB-IIProp.
S1, 0.0750.10 ±\pm 0.021.65 ±\pm 0.030.13 ±\pm 0.0132.5 ±\pm 1.450.0 ±\pm 0.228.7 ±\pm 4.22.00 ±\pm 0.052.07 ±\pm 0.142.03 ±\pm 0.02–––
S1, 0.1250.14 ±\pm 0.011.70 ±\pm 0.010.16 ±\pm 0.0033.1 ±\pm 0.549.8 ±\pm 0.332.2 ±\pm 0.32.06 ±\pm 0.052.03 ±\pm 0.042.02 ±\pm 0.03–––
S2, 0.0750.27 ±\pm 0.011.93 ±\pm 0.020.24 ±\pm 0.0129.5 ±\pm 2.067.6 ±\pm 1.929.5 ±\pm 2.32.06 ±\pm 0.022.50 ±\pm 0.122.11 ±\pm 0.17–––
S2, 0.1250.53 ±\pm 0.022.03 ±\pm 0.030.29 ±\pm 0.0126.1 ±\pm 1.866.7 ±\pm 0.329.3 ±\pm 0.82.03 ±\pm 0.042.39 ±\pm 0.112.11 ±\pm 0.05–––
S3, 0.0750.39 ±\pm 0.011.23 ±\pm 0.010.25 ±\pm 0.0122.3 ±\pm 0.344.6 ±\pm 0.530.1 ±\pm 1.22.24 ±\pm 0.032.45 ±\pm 0.092.26 ±\pm 0.07–––
TABLE I : End-to-end results on the four evaluation scenes at two carry speeds. Mean ±\pm std over three repeats of a fixed 300-frame run. ∑canchor\sum c_{\mathrm{anchor}} and ∑cact\sum c_{act} are integrated over the run. dev T is the midpoint-anchored mean node deviation of the full 225-node sheet at the final frame. Scenes 1–3 carry no obstacle, so clearance is undefined there. B-I: corner-following only, no shape prediction. B-II: no whole-body NMPC.
Ablation∑canc\sum c_{\mathrm{anc}} [ 10610^{6} ] ↓\downarrowdev T [cm] ↓\downarrow∑cact\sum c_{\mathrm{act}} [ 10310^{3} ] ↓\downarrowclr [cm] ↑\uparrowsolve [ms] ↓\downarrow
H=10H{=}100.51 ±\pm 0.0029.4 ±\pm 0.91.11 ±\pm 0.03−- 3.7 ±\pm 0.431.9 ±\pm 0.2
H=40H{=}40 (ours)0.53 ±\pm 0.0128.3 ±\pm 1.92.18 ±\pm 0.041.3 ±\pm 0.396.4 ±\pm 0.3
H=60H{=}600.65 ±\pm 0.0425.3 ±\pm 4.42.66 ±\pm 0.091.6 ±\pm 0.4139.3 ±\pm 0.6
wobs ⁣× ⁣10w_{\mathrm{obs}}\!\times\!100.66 ±\pm 0.0225.0 ±\pm 2.32.15 ±\pm 0.029.7 ±\pm 0.296.6 ±\pm 1.3
wshape ⁣× ⁣10w_{\mathrm{shape}}\!\times\!105.92 ±\pm 0.3022.6 ±\pm 3.02.13 ±\pm 0.06−- 7.3 ±\pm 0.697.1 ±\pm 0.3
TABLE II : Ablations on scene4 yobs40 at 0.125 m/s, the cell with the tightest time budget. Mean ±\pm std over three repeats. H=40H{=}40 is the proposed horizon and is read from the same runs as Prop. in Table I .

Explore similar work

CardsList

  1. Deformable Object Manipulation under Partial Observability via Real-Time Full-Shape Estimation

    Sep 9, 2026Kosar Behnia, Ville Kyrki, Gokhan AlcanDeformable Linear ObjectsStatistical Shape Model

    Faculty of Engineering and Natural Sciences, Tampere Univ., Finland. · Electrical Engineering and Automation Department, Aalto Univ., Finland.

  2. Learning from Humans for Proactive Assistance in Human-Robot Collaborative Transport

    Sep 21, 2026Elvin Yang, Christoforos MavrogiannisHuman-Robot CollaborationWhole-Body Control

    Department of Robotics, University of Michigan, Ann Arbor, United States

  3. Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement

    Jun 23, 2026Xin Liu, Yulin Li, Ziming Li +7Motion-Aware Clothing Dynamics

    National University of Singapore · Shanghai Jiao Tong University