How to reuse a pretrained locomotion policy for precise loco-manipulation is still an open problem. We propose an online method to learn the closed-loop dynamics under a frozen locomotion policy. The learned model predicts floating-base motion and improves loco-manipulation accuracy. However, unobserved latent conditions of the policy introduce heterogeneous responses, meaning that the same observed state will lead to different motions. Direct learning without accounting for this heterogeneity introduces bias and fluctuations into dynamics estimates, degrading floating-base prediction and end-effector tracking. To address this problem, a contaminated Gaussian model is proposed to approximate the effects of latent conditions. The model uses two Gaussian components to approximate predictable and unpredictable responses under the same observed state. Then, the contaminated Gaussian model is combined with a Gaussian process to construct an online learner using random Fourier features (RFFs). The learner estimates each response's probability of belonging to the predictable component and uses it to weight online updates, capturing predictable responses while reducing bias and fluctuations caused by heterogeneous observations. With consistent and unbiased dynamics estimates, model predictive control (MPC) predicts floating-base motion accurately and enables precise loco-manipulation. A finite-sample error bound is established to ensure uniform convergence in online learning. Simulation and hardware experiments demonstrate consistent end-effector tracking across different manipulator configurations and tasks, with errors of 0.01 m.
Figures & tables
Fig. 1: Motivation for online learning heterogeneous closed-loop dynamics to achieve precise loco-manipulation with a frozen locomotion policy. Unobserved latent conditions zt will produce different actions at and closed-loop responses for the same observed state st , challenging the unique-response assumption of homogeneous modeling. Our method accounts for this heterogeneity during online dynamics learning, enabling precise loco-manipulation without retraining the locomotion policy.
Fig. 2: Hardware demonstrations of the proposed method, which learns locomotion dynamics online to enable reuse of the frozen locomotion policy and generalization across diverse loco-manipulation tasks without retraining: (a) object grasping, (b) door opening, (c) disturbance recovery, (d) end-effector trajectory tracking, and (e) manipulation on the move.
Fig. 3: Control structure of the proposed method. The control inputs are the locomotion command cℓ,k (green) and the manipulator joint-velocity command ua,k (red). Locomotion commands are specified within the frozen policy’s command space and mapped to leg-joint actions aℓ,k (blue), while manipulator commands are applied directly. The coordinated base and manipulator motion regulates the end-effector pose χe,k .
Fig. 4: Heterogeneous locomotion dynamics illustrated by residuals of the base lateral velocity vy and roll rate ωx under lateral forces Fy=0 , 30 , and 60 N. Blue circles, orange squares, and vermilion triangles denote samples under these force levels, respectively. Matching cross markers indicate the corresponding group means. (a) A homogeneous fit to pooled residuals places the mean between latent-conditioned responses and combines within-condition variability with between-condition differences. The black star and shaded ellipse denote the pooled mean and covariance region, respectively, while colored dashed arrows connect the pooled mean to the group means. (b) Heterogeneous residual distributions exhibit distinct means and within-condition covariances under different latent conditions. The solid blue, dashed orange, and dotted vermilion ellipses and their shaded interiors represent the corresponding within-condition covariance regions, while the gray dashed ellipse shows the pooled covariance for comparison. (c) The proposed contaminated Gaussian approximation uses two components sharing the reference mean of the Fy=0 N condition, with reference covariance Σ0 and inflated covariance κΣ0 to account for additional dispersion. The solid purple and dashed teal ellipses denote Σ0 and κΣ0 , respectively.
Fig. 5: Overall architecture of the proposed method. (a) The contaminated Gaussian residual model approximates heterogeneous responses using a shared reference mean and two covariance scales, Σ0 and κΣ0 . The blue and red distributions represent reference and nonreference responses, respectively. (b) The online learner uses an RFF-based Gaussian process to estimate the reference residual mean and quantify epistemic uncertainty. Posterior learnability probabilities account for this uncertainty when weighting model updates, limiting the influence of nonreference responses. The blue curve and shaded band illustrate the learned mean and epistemic uncertainty, respectively. (c) Residual-augmented MPC rolls out the learned residual mean, epistemic covariance, and regime probability over the prediction horizon. The predicted epistemic uncertainty corrects the rollout by attenuating residual compensation, limiting MPC’s reliance on unreliable residual predictions in insufficiently learned regions. The red and blue trajectories illustrate the initial and corrected rollouts, respectively, with ellipses indicating epistemic uncertainty. (d) The optimized whole-body command combines locomotion commands mapped to leg-joint actions by the frozen policy and manipulator joint-velocity commands applied directly to the robot.
Fig. 6: Simulation results of end-effector tracking of a helical trajectory. The black dashed curve denotes the desired trajectory, while the gray, blue, and red curves correspond to Nominal MPC , SSI-MPC , and the proposed method Ours , respectively. (a) Successive robot configurations during helical trajectory tracking in simulation. (b) Cartesian end-effector trajectories. (c) Y – Z projection of one complete revolution. The proposed method Ours more closely follows the desired trajectory than Nominal MPC and SSI-MPC .
Fig. 8: Simulation results of multi-directional external disturbances. The gray, blue, and red curves correspond to Nominal MPC , SSI-MPC , and the proposed method Ours , respectively. Yellow shaded regions indicate the force intervals F1 – F3 . (a) Multi-directional external disturbances in simulation, with 100 N forces sequentially applied along the three Cartesian axes. Red arrows indicate force directions. (b) End-effector position tracking error. (c) Residual prediction error norm. (d) Posterior probability ρk+ defined in ( 25 ). The proposed method Ours identifies nonreference responses through the posterior probability, achieving smaller tracking and residual prediction errors after disturbance removal than either Nominal MPC or SSI-MPC .
Fig. 9: Simulation results of end-effector trajectory tracking with the Kinova Gen2 [ 38 ] manipulator at different mounting positions. Front, middle, and rear correspond to longitudinal offsets of +0.15 , 0 , and −0.15 m from the quadruped’s geometric center, respectively. (a) End-effector position tracking error. (b) Residual prediction error. Gray, blue, and red bars represent Nominal MPC , SSI-MPC , and Ours , respectively. Bar heights and error bars indicate the mean and standard deviation across repeated experiments. Ours consistently achieves lower tracking and residual prediction errors than Nominal MPC and SSI-MPC across all mounting positions.
Fig. 11: Hardware experimental results for helical trajectory tracking. (a) Snapshots of the trajectory tracking. The dashed curve denotes the desired end-effector trajectory. (b) End-effector position tracking error. (c) One-step residual prediction error norm. (d) Multi-step residual prediction skill, where the solid curve denotes the mean and the shaded region indicates the pointwise 95% confidence interval. Time in (b) and (c) is normalized by the task duration. The proposed method reduces the end-effector tracking and residual prediction errors through online learning, retaining the high prediction skill throughout the 1.0 s MPC horizon.
Fig. 14: Hardware experimental results for force recovery. (a) Responses to three force perturbations along the Cartesian axes while a fixed end-effector position is prescribed. (b) End-effector position tracking error (top) and one-step residual prediction error norm (bottom). (c) Posterior unlearnability probability ρk+ , which rises during each perturbation and returns close to zero afterward. The proposed method identifies nonreference responses through the posterior probability, supporting compliant recovery while restoring low tracking errors after disturbance removal.
Fig. 15: Hardware experimental results for door opening. (a) Snapshots of handle approach, door opening, and traversal. (b) End-effector position tracking error (top) and one-step residual prediction error norm (bottom). (c) Multi-step residual prediction skill, where the solid curve denotes the mean and the shaded region indicates the pointwise 95% confidence interval. Time in (b) is normalized by the task duration. The proposed method adapts online to sustained contact dynamics, reducing tracking and residual prediction errors as the door interaction is learned.
Continuous legged manipulation requires accurate end-effector tracking while the base keeps walking. Combining reinforcement learning (RL) with model predictive control (MPC) suits this task: the learned policy provides robust locomotion, while MPC coordinates the base and arm to compensate for tracking errors. However, MPC can compensate only for base motion that it can predict, and a learned policy's command response varies with gait phase, contact, and payload. We present ReCo, a framework that couples response-consistent locomotion with policy-aware MPC for legged manipulation. Response shaping trains the policy to respond to commands consistently and repeatably across randomized dynamics. An identified closed-loop response model then lets MPC jointly plan locomotion commands and arm motion. On the simulation benchmark, ReCo reduces position and orientation root-mean-square error (RMSE) by 28.7% and 27.4% relative to the best baseline for each metric. Real-world experiments demonstrate onboard continuous legged manipulation with coordinated base and arm motion.
Kuankuan Sima, Yichao Gao, Chenxi Gu +2
Department of Electrical and Computer Engineering, National University of Singapore, Singapore 119077, Singapore · Department of Mechanical Engineering, National University of Singapore, Singapore 119077, Singapore
Force-controlled loco-manipulation requires a whole-body policy to coordinate locomotion and arm motion while regulating end-effector interaction forces. This is challenging under floating-base dynamics and changing support contacts, particularly when end-effector force/torque sensing is unavailable for control. This paper presents a force-aware reinforcement learning approach with hybrid sensorless force estimation for wheeled-legged loco-manipulation. The proposed method provides a structured estimate of the end-effector force as an explicit policy observation, enabling force-guided contact behavior without using an end-effector force/torque sensor for control. The force estimate is obtained by combining generalized momentum observation, contact-constrained wrench projection, and temporal residual learning: the model-based components extract the physically structured part of the whole-body disturbance, while the residual network compensates the remaining motion-dependent bias. The estimated force is integrated into a mode-conditioned whole-body policy with an axis-wise force/position selector, allowing free-space motion, pure force regulation, and hybrid force/position control within one controller. Simulation results demonstrate improved sensorless force estimation and force-control performance. Hardware experiments further validate the proposed controller through quantitative valve-rotation and hybrid wiping evaluations, together with force-guided door opening and zero-force human-guided motion on a real wheeled-legged platform.
Xuanqi Zeng, Jiaming Wang, Tianlin Zhang +5
Department of Mechanical and Automation Engineering at the Chinese University of Hong Kong, Shatin, Hong Kong
Complex whole-body loco-manipulation requires executing diverse maneuvers, such as crouching, grasping, and locomotion, within a single long-horizon task. While performing simple pick-and-place behaviors is well-established, scaling to complex, long-horizon tasks remains difficult. Currently, this challenge is addressed by relying on human motion data and correspondingly humanoid platforms, which limits platform independence and introduces substantial data processing overhead. Instead, we extend proximal policy optimization (PPO) to train a single policy for multi-phase tasks across diverse robotic hardware, leveraging prior task structure to shape behavior. Our method adapts PPO using a phase-conditioned critic, phase-gated reward functions, and a multi-phase reset curriculum to guide exploration. To support varied hardware, we employ a decoupled hierarchical control architecture that separates lower-level balancing dynamics from high-level phase execution. We demonstrate multi-stage loco-manipulation skills on hardware on both the Unitree G1 and a Boston Dynamics Spot equipped with a manipulator arm, and validate our design choices through ablations. Our approach enables the acquisition of complex loco-manipulation behaviors purely through environment interaction while directly incorporating high-level task structure into policy learning.