Cross-Embodiment Motion Planning by Coupling Body-Agnostic World Model with Differentiable Ego-Dynamics
Organizations: School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore · Collaborative Initiative, Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore
Abstract
Motion planning in clutter is embodiment-specific: the feasible free space and reachable commands depend on each robot's geometry and dynamics. Existing cross-embodiment policies output normalized commands that disregard these differences, or compensate through per-robot training and distillation. World models can in principle pool cross-embodiment data, yet command-conditioned models bind their latent dynamics to a single action space, while pose-conditioned models omit the mapping from commands to body-specific motion. On flat ground, however, embodiment affects the planner's observations only through the camera ego-motion. This letter proposes Hourglass, an hourglass-shaped architecture that factorizes cross-embodiment motion planning at this interface: a body-agnostic world model and each robot's differentiable ego-dynamics, which map commands to that ego-motion, are coupled only through it. The world model is trained on data pooled across embodiments and camera mounts; each robot's ego-dynamics is a reduced-order template with identified physical parameters. A generalist parameter-conditioned policy is learned once at this junction: each imagined step rolls out the template and then the world model, and imagination training through each robot's template makes the policy body-aware. On seven simulated robots of five locomotion types, the generalist policy outperforms imitation-learning and diffusion-policy baselines on the same world model by 3.8--10.6 percentage points in success rate and matches per-robot specialists without dedicated training, while approaching its parameter-conditioned MPPI teacher within 1.4--2.8 points without an online map. After three minutes of teleoperated driving for identification, the synthetic-data-trained model transfers zero-shot to a real skid-steer robot unseen in training, navigating unseen indoor scenes.
Figures & tables
| Robot | Type | / | Cam. | ||||
| (m/s) | (rad/s) | (s) | (m) | (m) | (m) | ||
| Training | |||||||
| Carter | wheeled | 1.50 | 2.00 | 0.01 / 0.01 | 0.10 | 0.04 | |
| Spot | quadrupedal | 1.50 | 2.00 | 0.24 / 0.09 | 0.10 | 0.33 | |
| H1 | humanoid | 1.00 | 1.00 | 0.35 / 0.07 | 0.15 | 0.19 | |
| G1 | humanoid | 1.00 | 1.00 | 0.48 / 0.05 | 0.10 | 0.27 | |
| B2: random boxes, feasible episodes | B1: fixed layout, all episodes | |||||||||||||||||||||
| SR (%) | Coll. | SPL | Time | SR (%) | Coll. | SPL | Time | |||||||||||||||
| Method | Carter | Spot | H1 | G1 | M20 | Go2-W | TRON1 | Mean | (%) | (%) | (s) | Carter | Spot | H1 | G1 | M20 | Go2-W | TRON1 | Mean | (%) | (%) | (s) |
| BC- [ 7 ] | 65.7 | 68.2 | 77.4 | 71.8 | 70.6 | 77.3 | 67.8 | 71.2 ∗ | 9.2 | 63.8 | 14.6 | 82.7 | 83.6 | 84.5 | 83.4 | 85.8 | 83.4 | 76.4 | 82.8 ∗ | 3.1 | 75.7 | 13.4 |
| DP- [ 9 ] | 75.4 | 76.4 | 80.1 | 76.1 | 71.2 | 78.0 | 70.8 | 75.5 ∗ | 7.9 | 67.8 | 14.1 | 86.7 | 87.2 | 86.4 | 85.6 | 82.8 | 85.3 | 76.7 | 84.4 ∗ | 4.0 | 76.9 | 13.4 |
| DWA- [ 24 ] | 44.5 | 62.4 | 78.2 | 58.4 | 40.5 | 70.4 | 24.0 | 54.1 ∗ | 2.9 | 49.1 | 14.4 | 69.4 | 79.8 | 85.2 | 73.9 | 69.4 | 85.5 | 51.9 | 73.6 ∗ | 0.9 | 66.9 | 13.2 |
| MPPI- [ 24 ] | 95.6 | 86.6 | 89.7 | 90.6 | 77.8 | 92.4 | 49.6 | 83.2 ∗ | 3.2 | 70.1 | 13.0 | 98.8 | 95.0 | 89.7 | 94.5 | 92.8 | 95.8 | 70.5 | 91.0 ∗ | 1.6 | 78.0 | 12.4 |
| B2 | B1 | |||||
|---|---|---|---|---|---|---|
| Variant | SR | Coll. | SPL | SR | Coll. | SPL |
| Hourglass | 81.8 | 8.9 | 73.0 | 88.2 | 2.7 | 81.5 |
| w/o (normalized commands) | 72.6 ∗ | 11.9 | 61.7 | 82.9 ∗ | 3.9 | 72.8 |
| w/o egocentric spatial memory | 78.3 ∗ | 9.0 | 70.1 | 86.5 ∗ | 2.9 | 79.3 |
| untrained memory clearance prediction head | 78.6 ∗ | 8.9 | 70.9 | 86.8 ∗ | 2.9 | 80.4 |
| 2 s instead of 5 s horizon | 79.9 ∗ | 8.5 | 72.6 | 87.5 | 2.9 | 81.2 |
Motion Planning
Motion planning finds safe, feasible paths and trajectories for robots and vehicles amid geometric, dynamic, and task constraints.
See all 440 papers