Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers

Snehal Jauhri; Vignesh Prasad; Georgia Chalvatzaki

cs.ROApr 14, 2026

Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers

Authors: Snehal Jauhri, Vignesh Prasad, Georgia Chalvatzaki

Organizations: PEARL Lab, Computer Science Department, Technische Universität Darmstadt, Germany · Hessian.AI, Darmstadt, Germany · Robotics Institute, Germany

Abstract

Mobile Manipulation (MoMa) of articulated objects, such as opening doors, drawers, and cupboards, demands simultaneous, whole-body coordination between a robot's base and arms. Classical whole-body controllers (WBCs) can solve such problems via hierarchical optimization, but require extensive hand-tuned optimization and remain brittle. Learning-based methods, on the other hand, show strong generalization capabilities but typically rely on expensive whole-body teleoperation data or heavy reward engineering. We observe that even a sub-optimal WBC is a powerful structural prior: it can be used to collect data in a constrained, task-relevant region of the state-action space, and its behavior can still be improved upon using offline reinforcement learning. Building on this, we propose WHOLE-MoMa, a two-stage pipeline that first generates diverse demonstrations by randomizing a lightweight WBC, and then applies offline RL to identify and stitch together improved behaviors via a reward signal. To support the expressive action-chunked diffusion policies needed for complex coordination tasks, we extend offline implicit Q-learning with Q-chunking for chunk-level critic evaluation and advantage-weighted policy extraction. On three tasks of increasing difficulty using a TIAGo++ mobile manipulator in simulation, WHOLE-MoMa significantly outperforms WBC, behavior cloning, and several offline RL baselines. Policies transfer directly to the real robot without finetuning, achieving 80% success in bimanual drawer manipulation and 68% in simultaneous cupboard opening and object placement, all without any teleoperated or real-world training data.

Figures & tables

Fig. 2 : A whole-body controller (WBC) provides a strong but sub-optimal motion-generation prior: it solves a multi-objective optimization problem for the various embodiments in question, $\min_{x}\;w_{ee}\|e_{ee}\|^{2}+w_{base}\|e_{base}\|^{2}+w_{posture}\|e_{posture}\|^{2}$ , under constraints such as self-collision and joint limits, but optimizing the current objective does not guarantee a good future interaction state. Here, $w$ and $e$ denote task weights and tracking errors, respectively.

Fig. 3 : WHOLE-MoMa pipeline. Parameter-randomized WBC rollouts produce whole-body demonstrations scored with a reward $r_{t}$ combining task success, articulation joint angle change $\Delta q_{\text{art}}$ , and end-effector or base distance reduction ( $\Delta d_{\text{ee}}$ or $\Delta d_{\text{base}}$ , depending on the task). Transitions are grouped into horizon- $H$ chunks with discounted return $R_{\text{chunk}}(t)$ . An implicit Q-chunked transformer $Q_{\theta}(s_{t},\mathbf{a}_{t:t+H-1})$ is learned via chunked TD error and used to compute the advantage $A(s_{t},\mathbf{a}_{t:t+H-1})$ . A Diffusion Policy $\pi_{\phi}$ is then trained with the chunk-level AWR objective $\mathcal{L}_{\text{AWR}}^{\text{chunk}}$ , weighting the diffusion BC loss $\mathcal{L}_{\text{diff}}$ by the advantage. The resulting policy is deployed directly on the real robot using marker-based or 6D pose-tracking state estimation.

Task	Reward $(r)$ / Return $(R)$	Weight ( $w$ )
Door	$r_{\mathrm{success}}$	$1.0$
	$R_{\mathrm{art}}=\sum_{t}\Delta q_{\mathrm{art},t}$	$1.0$
	$R_{\mathrm{ee}}=\sum_{t}-\Delta d_{\mathrm{ee},t}$	$1.0$
	$R_{\mathrm{base}}=\sum_{t}-\Delta d_{\mathrm{base},t}$	$1.0$
Drawer /
Cupboard	$r_{\mathrm{success}}$	$1.0$

TABLE II: Reward

(r)

/ Return

(R)

terms per task.

\Delta q_{\mathrm{art}}

denotes articulation progress, and

\Delta d_{\mathrm{ee}}

and

\Delta d_{\mathrm{base}}

denote end-effector and base distance reduction.

Panel of Figure 4 — Fig. 4 : Simulated and real whole-body mobile manipulation environments. Top two rows: simulated tasks at increasing complexity: level 1 Door (push open and pass through), level 2 Drawer (close one and open another bimanually), level 3 Cupboard (open door and place object inside). Bottom row: real-world tasks RealDrawerOpenOneCloseAnother and RealCupboardOpenAndPlace on a Tiago++ holonomic mobile manipulator.

Panel of Figure 5 — Fig. 5 : RealDrawerOpenOneCloseAnother task qualitative comparison in the real world. Top left: approach phase. Top right: the WBC gets stuck in a local optimum and fails to fully open/close the corresponding drawers. Bottom left: the behavior-cloned diffusion policy is too imprecise and breaks the handle. Bottom right: WHOLE-MoMa successfully performs simultaneous drawer closing and opening. Videos of the executions are available on the project website: sites.google.com/view/whole-moma .

Panel of Figure 6 — Fig. 6 : Pose-tracking state estimation on the RealCupboardOpenAndPlace task. Left: successful articulation with accurate pose tracking. Right: failure case where insufficient pose-tracking precision causes the policy to stall or oscillate during articulation.

Explore similar work

Jun 29, 2026

WARP: Whole-Body Retargeting for Learning from Offline Human Demonstrations

Direct transfer from human demonstration to learnable robot action is a crucial step towards scalable whole-body mobile manipulation. While human data scales better than mobile teleoperation, it requires overcoming significant embodiment gaps. Existing retargeting methods yield imprecise or inconsistent solutions, causing action multi-modality that prevents supervised policies from reliably converging. We present Whole-body-Aware Retargeting from human Pose (WARP), an offline pipeline that explicitly models embodiment differences to extract precise, unique whole-body actions. WARP leverages a closed-form Shoulder-Elbow-Wrist (SEW) geometric solver for exact end-effector tracking while preserving whole-body structural intent. Paired with lazy mobile-base control, it extracts accurate, consistent robot trajectories. Evaluations show WARP provides highly reliable data for open-loop real-world replay. To our knowledge, WARP is the first framework to achieve zero-shot whole-body mobile manipulation directly from offline human demonstrations, eliminating the need for human-in-the-loop teleoperation action data. More details on https://warp-retarget.github.io/

Zhenyang Chen, Chuizheng Kong, Chuye Zhang +4

Georgia Institute of Technology, Atlanta, Georgia 30332

Mobile Manipulation Cross-Embodiment Robot Learning Whole-Body Robotic Manipulation Motion Retargeting Robotic Manipulation Robot Imitation Learning

Ask PDF

Apr 14, 2026

Scalable Trajectory Generation for Whole-Body Mobile Manipulation

Robots deployed in unstructured environments must coordinate whole-body motion -- simultaneously moving a mobile base and arm -- to interact with the physical world. This coupled mobility and dexterity yields a state space that grows combinatorially with scene and object diversity, demanding datasets far larger than those sufficient for fixed-base manipulation. Yet existing acquisition methods, including teleoperation and planning, are either labor-intensive or computationally prohibitive at scale. The core bottleneck is the lack of a scalable pipeline for generating large-scale, physically valid, coordinated trajectory data across diverse embodiments and environments. Here we introduce AutoMoMa, a GPU-accelerated framework that unifies AKR modeling, which consolidates base, arm, and object kinematics into a single chain, with parallelized trajectory optimization. AutoMoMa achieves 5,000 episodes per GPU-hour (over

80\times

faster than CPU-based baselines), producing a dataset of over 500k physically valid trajectories spanning 330 scenes, diverse articulated objects, and multiple robot embodiments. Prior datasets were forced to compromise on scale, diversity, or kinematic fidelity; AutoMoMa addresses all three simultaneously. Training downstream IL policies further reveals that even a single articulated-object task requires tens of thousands of demonstrations for SOTA methods to reach

\approx 80\%

success, confirming that data scarcity -- not algorithmic limitations -- has been the binding constraint. AutoMoMa thus bridges high-performance planning and reliable IL-based control, providing the infrastructure previously missing for coordinated mobile manipulation research. By making large-scale, kinematically valid training data practical, AutoMoMa showcases generalizable whole-body robot policies capable of operating in the diverse, unstructured settings of the real world.

Yida Niu, Xinhai Chang, Xin Liu +2

Robot Motion Generation Mobile Manipulation Whole-Body Robotic Manipulation Synthetic Data Generation for Robotics Trajectory Optimization

Ask PDF

Oct 13, 2026

I3L: Iterative Interactive Imitation Learning for Whole-Body Mobile Manipulation

Policies trained offline from teleoperated demonstrations can degrade outside their training distribution. This gap is particularly acute in long-horizon whole-body mobile manipulation, where long horizons amplify compounding errors and coordinated arm, torso, and base motion makes exhaustive demonstration coverage impractical. We introduce I3L, an integrated hardware and learning system that combines whole-body correction with iterative policy refinement to address this last-mile reliability gap. Its low-cost, open-source JoyLo+ interface uses policy-tracking leader arms and touch-triggered takeover to enable coordinated corrections across the body parts, while iterative deployment and retraining allow operators to target failures of the updated policy. We evaluate I3L across three robot embodiments and eight simulated and real-world tasks, including four long-horizon whole-body mobile manipulation tasks. Controlled simulation experiments show that four-round HG-DAgger achieves the highest final success on both tasks. Under this protocol, I3L improves autonomous success over the base policies by an average of 68.3 percentage points and achieves at least 90% success on every task, outperforming demonstration-only training by 35.6 percentage points on average with the same number of training trajectories. Averaged across tasks, takeover duration per episode decreases by 72% over four rounds. Together, these results show that I3L helps close the last-mile gap between offline training and reliable deployment and provides practical insights into collecting and using corrections for long-horizon whole-body mobile manipulation. Videos, visualizations, and appendix are at https://i-3l.github.io/.

Wensi Ai, Christopher Luey, Karthik Dharmarajan +7

Stanford University · Lambda Inc.

Robot Imitation Learning Interactive Imitation Learning Mobile Manipulation Long-Horizon Robotic Manipulation Whole-Body Robotic Manipulation

Ask PDF