Interaction-Aware Whole-Body Control for Compliant Object Transport
Authors: Hao Zhang, Yves Tseng, Ding Zhao, H. Eric Tseng
Organizations: Department of Electrical Engineering, the University of Texas at Arlington, 76010 Arlington, USA · Department of Mechanical Engineering, Carnegie Mellon University, 15213 Pittsburgh, USA
Cooperative object transport in unstructured environments remains challenging for assistive humanoids because strong, time-varying interaction forces can make tracking-centric whole-body control unreliable, especially in close-contact support tasks. This paper proposes a bio-inspired, interaction-oriented whole-body control (IO-WBC) that functions as an artificial cerebellum - an adaptive motor agent that translates upstream (skill-level) commands into stable, physically consistent whole-body behavior under contact. This work structurally separates upper-body interaction execution from lower-body support control, enabling the robot to maintain balance while shaping force exchange in a tightly coupled robot-object system. A trajectory-optimized reference generator (RG) provides a kinematic prior, while a reinforcement learning (RL) policy governs body responses under heavy-load interactions and disturbances. The policy is trained in simulation with randomized payload mass/inertia and external perturbations, and deployed via asymmetric teacher-student distillation so that the student relies only on proprioceptive histories at runtime. Extensive experiments demonstrate that IO-WBC maintains stable whole-body behavior and physical interaction even when precise velocity tracking becomes infeasible, enabling compliant object transport across a wide range of scenarios.
Figures & tables
Fig. 1 : The proposed IO-WBC architecture that bridges skill-level HRC commands with low-level execution through a kinematic-prior-based RG, enabling stable coordination between upper-body interaction and lower-body support under strong robot–object coupling.
Fig. 2 : The comprehensive learning and execution pipeline of IO-WBC. The RG is trained via supervised learning to provide kinematic priors. The interaction-oriented WBC policy is developed through teacher-student distillation, where a privileged teacher guides a student policy to decode interaction dynamics from proprioceptive history.
Fig. 3 : Comprehensive performance comparison across algorithmic variants. The first two panels report the norm errors in the lifting and pushing scenarios, while the two rightmost panels compare the load capacity and success rates.
Parameter
Value / Range
RG layers (hidden)
[128, 64]
IO-WBC policy layers (hidden)
[512, 256, 128]
Proprioceptive history ( Ht )
25 steps
Control frequency
50 Hz
Discount factor ( γ )
0.99
GAE factor ( λ )
0.95
TABLE I : Detailed hyperparameters for RL training and curriculum specifications.
Fig. 4 : Visualization of the learning hierarchy in simulation. The framework encompasses single-agent skill learning for the IO-WBC layer and MARL for high-level HRC coordination.
Fig. 5 : Real-world deployment of IO-WBC on the HRC tasks between human and Unitree G1, including path-constrained collaborative pushing and lifting, as well as collaborative super-heavy object lifting. Specifically, the red dashed regions highlight the adaptive vertical synchronization and postural resilience during the transport of a 7 kg box and an 18 kg tire.
Fig. 6 : Tracking errors vs. lifting mass. IO-WBC maintains near-constant performance, while baselines exhibit exponential error growth in Eα,Eh,Ev, and Eψ .
Mass
Algorithm
Eα ( ↓ )
Eh ( ↓ )
Ev ( ↓ )
Eψ ( ↓ )
0 kg
IO-WBC
0.125
0.015
0.046
0.134
WBC
0.126
0.015
0.047
0.135
w/o RG
0.188
0.051
0.048
0.403
w/o Distill.
0.184
0.023
0.177
0.118
4 kg
IO-WBC
0.132
0.016
0.048
0.139
WBC
0.166
0.034
0.098
0.158
TABLE II : Detailed comparison of HRC command tracking errors under cooperative lifting (0 kg–8 kg).
Mass
Algorithm
Eα ( ↓ )
Eh ( ↓ )
Ev ( ↓ )
Eψ ( ↓ )
0 kg
IO-WBC
0.125
0.015
0.046
0.134
WBC
0.126
0.015
0.047
0.135
w/o RG
0.286
0.049
0.048
0.413
w/o Distill.
0.184
0.023
0.177
0.118
30 kg
IO-WBC
0.131
0.016
0.225
0.143
WBC
0.211
0.055
0.356
0.246
TABLE III : Detailed comparison of HRC command tracking errors under cooperative pushing (0 kg–60 kg).
Fig. 7 : Tracking errors vs. pushing mass. Despite rising velocity errors ( Ev ), IO-WBC preserves high-fidelity postural control ( Eα,Eh ) up to 60 kg.
Whole-body compliant control is essential for deploying heavy humanoids under high payload in human-centric environments. Most prior force-aware learning-based pipelines focus on end-effector resistance, per-link upper-body springs, or end-effector stiffness modulation, leaving arbitrary-site perturbations on heavy platforms with lower-body engagement largely unaddressed. We close this gap with CompliantWBC comprising: (1) A base policy trained with RL to maximize compliance-fidelity reward, guided by a multi-site whole-body impedance reference controller, extending classical Cartesian impedance to any controlled link; (2) A bounded residual policy that edits the per-link impedance equilibrium over a frozen base, correcting the coarse but structured wrench estimate supplied by a force encoder co-trained behind a gradient barrier; (3) A Phong-weighted force-origin sampler with an axis-decoupled pelvis anchor induces lower-body-inclusive compliance curriculum training via two interpretable parameters. We evaluate CompliantWBC in simulation against both compliant and stiff baselines, achieving best compliant fidelity of 2.58cm deviation from analytical solutions, and demonstrate it on a real heavy humanoid across static/dynamic force reaction, board wiping, squat under payload, and cooperative payload transport. Project website: https://dotandung.github.io/compliantwbc/
Tan-Dzung Do, Cuc T. Trinh, Tuan Dat Phuong +4
VinRobotics · National University of Singapore · VinUniversity +1
General-purpose motion trackers enable humanoid robots to follow diverse whole-body motions while maintaining balance, but are trained only on flat ground, failing to exploit bipedal mobility over complex terrain. Cross-terrain controllers, meanwhile, are task-specific or accept only low-dimensional locomotion commands. We introduce InterTrack, the first behavior world model (BWM) for robust whole-body tracking with environment interaction. Its Transformer jointly predicts the next action, state, and behavior distribution, learning environment-conditioned dynamics. To scale interaction training data, an automatic annotation pipeline reconstructs 3D support geometry from retargeted motions. At deployment, the policy handles commands implausible in the current environment in a "best-effort" manner. Quantitatively, InterTrack achieves an 81.3% success rate on terrain interaction (4.3 times the best evaluated baseline) and a 99.3% fall-recovery rate, while also improving free-space tracking and outperforming three leading tracking baselines across all of these regimes. To our knowledge, we provide the first demonstration of real-time cross-terrain whole-body teleoperation on a humanoid robot, alongside object interaction, stable responses to missing supports, and robust recovery from falls.
Ziyang Cheng, Tianshu Tang, Jinxin Lan +19
Tsinghua University · GigaAI · University of Shanghai for Science and Technology +3
Achieving versatile and natural whole-body humanoid interaction control remains challenging due to the high cost of whole-body teleoperation data. We present ZeroWBC, a teleoperation-free framework that learns humanoid whole-body interaction from human egocentric videos paired with synchronized whole-body motion and text annotations. ZeroWBC adopts a generation-then-tracking formulation to tackle the static scene whole-body interaction control problem. Given an initial egocentric image and a language instruction, a fine-tuned Vision-Language Model generates future human whole-body motion tokens, which are decoded into continuous motions and retargeted to the humanoid. The resulting reference motions, together with root and key body-part trajectories, are then executed by a general interactive motion tracking policy. To improve interaction performance, we introduce an interaction-oriented tracking reward that prioritizes global root and key body-part trajectory alignment while preserving natural whole-body motion. Experiments on the Unitree G1 humanoid robot show that ZeroWBC enables diverse scene-aware behaviors without robot teleoperation demonstrations. These results suggest a scalable paradigm for learning natural humanoid whole-body interaction from human egocentric data.
Haoran Yang, Jiacheng Bao, Yucheng Xin +5
University of Science and Technology of China · Shanghai AI Laboratory · Tsinghua University +2