Scalable Trajectory Generation for Whole-Body Mobile Manipulation

Yida Niu; Xinhai Chang; Xin Liu; Ziyuan Jiao; Yixin Zhu

cs.ROApr 14, 2026

Scalable Trajectory Generation for Whole-Body Mobile Manipulation

Authors: Yida Niu, Xinhai Chang, Xin Liu, Ziyuan Jiao, Yixin Zhu

Abstract

Robots deployed in unstructured environments must coordinate whole-body motion -- simultaneously moving a mobile base and arm -- to interact with the physical world. This coupled mobility and dexterity yields a state space that grows combinatorially with scene and object diversity, demanding datasets far larger than those sufficient for fixed-base manipulation. Yet existing acquisition methods, including teleoperation and planning, are either labor-intensive or computationally prohibitive at scale. The core bottleneck is the lack of a scalable pipeline for generating large-scale, physically valid, coordinated trajectory data across diverse embodiments and environments. Here we introduce AutoMoMa, a GPU-accelerated framework that unifies AKR modeling, which consolidates base, arm, and object kinematics into a single chain, with parallelized trajectory optimization. AutoMoMa achieves 5,000 episodes per GPU-hour (over $80\times$ faster than CPU-based baselines), producing a dataset of over 500k physically valid trajectories spanning 330 scenes, diverse articulated objects, and multiple robot embodiments. Prior datasets were forced to compromise on scale, diversity, or kinematic fidelity; AutoMoMa addresses all three simultaneously. Training downstream IL policies further reveals that even a single articulated-object task requires tens of thousands of demonstrations for SOTA methods to reach $\approx 80\%$ success, confirming that data scarcity -- not algorithmic limitations -- has been the binding constraint. AutoMoMa thus bridges high-performance planning and reliable IL-based control, providing the infrastructure previously missing for coordinated mobile manipulation research. By making large-scale, kinematically valid training data practical, AutoMoMa showcases generalizable whole-body robot policies capable of operating in the diverse, unstructured settings of the real world.

Figures & tables

Dataset	Robot	# Episodes	Coord.	# Scenes	Action	Method
RT-1 Robot Action [ 4 ]	Google Robot	73,499	Yes	10	End-effector pose	VR teleoperation
NYU VINN [ 33 ]	Hello Stretch	435	Yes	3	End-effector pose	Kinesthetic teaching
BC-Z [ 18 ]	Google Robot	39,350	Yes	2–3	End-effector pose	VR teleoperation
ETH Agent Affordances [ 35 ]	Franka	120	No	50	End-effector pose	Scripted policy
QUT Dexterous Manip. [ 5 ]	Franka	200	No	1	End-effector pose	VR teleoperation
CMU Stretch [ 1 , 29 ]	Hello Stretch	135	No	10	End-effector pose	Scripted Policy

Table 1: Comparison of AutoMoMa with existing mobile manipulation datasets. Existing datasets are constrained by their acquisition methods: teleoperation yields high-fidelity but small-scale data, while scripted policies lack whole-body coordination. AutoMoMa overcomes these limitations through gpu -accelerated automated planning, simultaneously achieving large scale, broad diversity, and high-fidelity joint-space trajectories—a combination no prior dataset provides. “ Coord. ”: presence of whole-body base-arm coordination.

Figure 2: An example of the akr construction. The akr unifies independent kinematic trees into a single serial chain, enabling joint whole-body optimization of the base, arm, and object. The mobile base’s planar motion is modeled via a virtual base (blue), while a virtual joint (black) couples the manipulator (orange) to the target object (green). For articulated objects, the kinematic tree is inverted to reconfigure the kinematic root to the grasp point, forming a continuous chain (highlighted in yellow) rooted at the virtual world frame.

Figure 3: The AutoMoMa data generation pipeline. Starting from a task specification triplet $(\mathcal{S},\mathcal{O},\mathcal{R})$ (left), AutoMoMa proceeds through four stages: (i) Task Specification defines the environmental, robotic, and object context; (ii) Problem Instantiation transforms raw scene assets into planning-ready primitives via esdf construction and akr assembly with spherical collision approximations; (iii) Trajectory Generation solves for optimal akr states under task-specific constraints to produce physically valid whole-body motions; and (iv) Rendering in NVIDIA Isaac Sim produces synchronized rgbd sequences and point clouds. The resulting trajectories span diverse scenes, objects, and robot embodiments (right).

Panel of Figure 4 — Figure 4: Trajectory generation performance across six representative household scenes. (a) Test scenes with increasing spatial confinement. (b) Generation throughput (valid trajectories per second) decreases as scene clutter increases collision-checking overhead. (c) Average translational effort of the mobile base per trajectory (error bars: standard deviation). (d) Average rotational effort of the manipulator, reflecting compensatory whole-body motion in constrained environments.

Figure 5: Distribution of trajectory base positions. Blue and orange spheres denote start and goal base placements, respectively, illustrating the broad spatial coverage achieved by the ik clustering strategy.

Panel of Figure 6 — Figure 6: Data scaling experiments. (a) In a single scene, the mobile base policy requires substantially more data than the fixed-base counterpart, with a persistent seen/unseen gap indicating manifold memorization. (b) Increasing scene diversity from 1 to 30 steadily improves generalization to unseen environments. (c) With 30 scenes, higher per-scene trajectory density further refines execution precision, enabling consistent generalization across seen and unseen scenes.

Figure 7: Architectural generalization of AutoMoMa . When evaluated across the same 30-scene setup as DP3 [ 49 ] , both DP [ 6 ] and ACT [ 13 ] exhibit consistent performance gains with increasing trajectory density, demonstrating AutoMoMa ’s compatibility with diverse whole-body il architectures.

Figure 8: Per-object success rates at 100k trajectories. Success rates of the DP3 policy evaluated on five representative SAPIEN [ 47 ] objects. The bar plot compares performance under unseen (orange) and seen (blue) environments.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 9 — Figure A1: Downstream scene split. 30 training and 10 test scenes are sampled from the 300 procedurally generated scenes for policy training and evaluation.

Panel of Figure 10 — Figure A2: Two scene sources in AutoMoMa . (a) AI2-THOR scenes with articulated objects substituted from SAPIEN. (b) Procedurally generated layouts from Infinigen.

Hyperparameter	Value
Environment and Collision
Object offset base (quaternion)	[0, 0, 0, 1, 0, 0, 0]
Expanded dimension	0.3
Disable collision	False
Collision type	Voxel / Mesh
ik mesh pitch	0.05

Appendix

Table A1: Hyperparameters for AutoMoMa data generation.

Hyperparameter	Value
Observation and Action
Number of points in point cloud	4096
Point feature dimension	3
Proprioceptive input dim	11
History length (observation steps)	3
Prediction horizon	8

Appendix

Table A2: Hyperparameters for DP3 [ 49 ] .

Hyperparameter	Value
Observation and Action
Number of camera views	3
Proprioceptive input dim	11
History length (observation steps)	3
Prediction horizon	8
Number of action steps	6

Appendix

Table A3: Hyperparameters for DP [ 6 ] .

Hyperparameter	Value
Observation and Action
Number of camera views	3
Action dim	11
Action chunk size	6
Temporal aggregation	False
Mobile base mode	relative

Appendix

Table A4: Hyperparameters for ACT [ 13 ] .

Figure A3: Trajectory failure caused by collision. Sphere-based geometry approximations occasionally fail to capture the original shape precisely, resulting in unintended collisions during planning.

Figure A4: Trajectory failure caused by fixed-base constraint violation. The planned trajectory erroneously involves movements inconsistent with the object’s fixed-base constraint.

Figure A5: Representative successful trajectory. The planned trajectory correctly respects both collision and motion constraints, producing a physically plausible whole-body motion.

Panel of Figure 18 — Figure A6: Representative successful inference trajectories across diverse scenes. Trained on large-scale AutoMoMa data, DP3 [ 49 ] produces feasible whole-body trajectories across varied spatial layouts and articulated-object configurations.

Figure A7: Representative inference failure. Small inconsistencies in base or arm pose predictions compound over time, eventually pushing the robot into an infeasible configuration.

Panel of Figure 20 — Figure A8: Real-world validation on a UR5-Ridgeback platform. Planned trajectories for drawer opening and cabinet door opening are executed smoothly without collision or constraint violation.

Figure A9: Scaling analysis at 100k trajectories. Training DP3 on a 100k-trajectory dataset spanning multiple articulated objects and scenes improves cross-object capability and generalization to unseen environments, compared with smaller-scale training subsets.

Panel of Figure 22 — Figure A10: Ablation on start-state diversity. Top: Increasing unique start states (50–1,000) monotonically improves success on the 1,000-trajectory benchmark. Bottom: For fixed dataset sizes (6,400 trajectories), more diverse start states improve performance; the full 12,800-trajectory dataset achieves the highest success rate.

Panel of Figure 23 — Figure A11: Inference examples for the pick task using a policy trained on 1,000 AutoMoMa trajectories.

Panel of Figure 24 — Figure A12: Fixed-grasp vs . grasp-switching. Grasp switching enables larger object opening angles by avoiding link collisions.

Explore similar work

Apr 14, 2026

Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllers

Mobile Manipulation (MoMa) of articulated objects, such as opening doors, drawers, and cupboards, demands simultaneous, whole-body coordination between a robot's base and arms. Classical whole-body controllers (WBCs) can solve such problems via hierarchical optimization, but require extensive hand-tuned optimization and remain brittle. Learning-based methods, on the other hand, show strong generalization capabilities but typically rely on expensive whole-body teleoperation data or heavy reward engineering. We observe that even a sub-optimal WBC is a powerful structural prior: it can be used to collect data in a constrained, task-relevant region of the state-action space, and its behavior can still be improved upon using offline reinforcement learning. Building on this, we propose WHOLE-MoMa, a two-stage pipeline that first generates diverse demonstrations by randomizing a lightweight WBC, and then applies offline RL to identify and stitch together improved behaviors via a reward signal. To support the expressive action-chunked diffusion policies needed for complex coordination tasks, we extend offline implicit Q-learning with Q-chunking for chunk-level critic evaluation and advantage-weighted policy extraction. On three tasks of increasing difficulty using a TIAGo++ mobile manipulator in simulation, WHOLE-MoMa significantly outperforms WBC, behavior cloning, and several offline RL baselines. Policies transfer directly to the real robot without finetuning, achieving 80% success in bimanual drawer manipulation and 68% in simultaneous cupboard opening and object placement, all without any teleoperated or real-world training data.

Snehal Jauhri, Vignesh Prasad, Georgia Chalvatzaki

PEARL Lab, Computer Science Department, Technische Universität Darmstadt, Germany · Hessian.AI, Darmstadt, Germany · Robotics Institute, Germany

Mobile Manipulation Whole-Body Robotic Manipulation Offline RL Bimanual Robotic Manipulation

Ask PDF

Jun 29, 2026

WARP: Whole-Body Retargeting for Learning from Offline Human Demonstrations

Direct transfer from human demonstration to learnable robot action is a crucial step towards scalable whole-body mobile manipulation. While human data scales better than mobile teleoperation, it requires overcoming significant embodiment gaps. Existing retargeting methods yield imprecise or inconsistent solutions, causing action multi-modality that prevents supervised policies from reliably converging. We present Whole-body-Aware Retargeting from human Pose (WARP), an offline pipeline that explicitly models embodiment differences to extract precise, unique whole-body actions. WARP leverages a closed-form Shoulder-Elbow-Wrist (SEW) geometric solver for exact end-effector tracking while preserving whole-body structural intent. Paired with lazy mobile-base control, it extracts accurate, consistent robot trajectories. Evaluations show WARP provides highly reliable data for open-loop real-world replay. To our knowledge, WARP is the first framework to achieve zero-shot whole-body mobile manipulation directly from offline human demonstrations, eliminating the need for human-in-the-loop teleoperation action data. More details on https://warp-retarget.github.io/

Zhenyang Chen, Chuizheng Kong, Chuye Zhang +4

Georgia Institute of Technology, Atlanta, Georgia 30332

Mobile Manipulation Cross-Embodiment Robot Learning Whole-Body Robotic Manipulation Motion Retargeting Robotic Manipulation Robot Imitation Learning

Ask PDF

Aug 2, 2026

DreamTrajectory: Trajectory-Guided Action Generation with World Model Alignment for Mobile Manipulation

Mobile manipulation requires a robot to coordinate base and arm motion under continuously changing viewpoints and contact conditions, within an action space far larger than that of fixed-base manipulation. Existing Vision-Language-Action (VLA) policies are limited in two respects. (i)They map observations directly to whole-body action chunks, searching this large action space without an explicit task-space motion plan, which makes coordinated base--arm prediction imprecise. (ii)They execute the predicted chunk open-loop, without checking whether the actions can realize the motion the policy intended, so control errors and unmodeled contacts accumulate into a gap between planned and realized motion. We present DreamTrajectory, a trajectory-guided framework for language-conditioned mobile manipulation that introduces one component for each limitation. Addressing(i), DreamTrajectory jointly predicts an intention-level end-effector trajectory and a whole-body action chunk in a single action expert, so that the trajectory explicitly guides base--arm action generation instead of remaining implicit. Addressing(ii), a lightweight trajectory world model predicts the trajectory that a candidate action chunk would induce, and a test-time search--predict--score procedure selects the candidate best aligned with the planned trajectory. On MS-HAB, trajectory guidance raises average success from 32.3% to 47.5% and test-time refinement further to 54.8%, with the largest gains on contact-rich articulated-object tasks. On three real-world mobile manipulation tasks, the corresponding average success rates are 63.3%, 81.7%, and 90.0%.

Zheng Yang, Wenjie Zhang, Xiangyu Chen +7

Hong Kong University of Science and Technology (Guangzhou) · Ola Dimensions

Trajectory Generation World Models for Robotics Mobile Manipulation Vision-Language-Action Models

Ask PDF