Organizations: School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore · Collaborative Initiative, Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore
Motion planning in clutter is embodiment-specific: the feasible free space and reachable commands depend on each robot's geometry and dynamics. Existing cross-embodiment policies output normalized commands that disregard these differences, or compensate through per-robot training and distillation. World models can in principle pool cross-embodiment data, yet command-conditioned models bind their latent dynamics to a single action space, while pose-conditioned models omit the mapping from commands to body-specific motion. On flat ground, however, embodiment affects the planner's observations only through the camera ego-motion. This letter proposes Hourglass, an hourglass-shaped architecture that factorizes cross-embodiment motion planning at this interface: a body-agnostic world model and each robot's differentiable ego-dynamics, which map commands to that ego-motion, are coupled only through it. The world model is trained on data pooled across embodiments and camera mounts; each robot's ego-dynamics is a reduced-order template with identified physical parameters. A generalist parameter-conditioned policy is learned once at this junction: each imagined step rolls out the template and then the world model, and imagination training through each robot's template makes the policy body-aware. On seven simulated robots of five locomotion types, the generalist policy outperforms imitation-learning and diffusion-policy baselines on the same world model by 3.8--10.6 percentage points in success rate and matches per-robot specialists without dedicated training, while approaching its parameter-conditioned MPPI teacher within 1.4--2.8 points without an online map. After three minutes of teleoperated driving for identification, the synthetic-data-trained model transfers zero-shot to a real skid-steer robot unseen in training, navigating unseen indoor scenes.
Figures & tables
Fig. 1: Hourglass: a body-agnostic world model and the differentiable ego-dynamics with physical parameters θ , which map the commands u to the camera ego-motion Δξ , are joined only through Δξ ; a cross-embodiment generalist policy π is learned at this junction.
Fig. 2: Overview of Hourglass. Left: the body-agnostic world model pψ , and the differentiable ego-dynamics fθ with physical parameters θ . Middle: in coupled latent imagination, the two modules interact through the ego-motion Δξk , and a single θ -conditioned policy is trained by the analytic gradient of J=∑ℓk . Right: deployment with the identified θ .
Robot
Type
vmax
ωmax
τv / τω
L×W
Cam.
εθ
(m/s)
(rad/s)
(s)
(m)
(m)
(m)
Training
Carter
wheeled
1.50
2.00
0.01 / 0.01
0.77×0.62
0.10
0.04
Spot
quadrupedal
1.50
2.00
0.24 / 0.09
0.87×0.44
0.10
0.33
H1
humanoid
1.00
1.00
0.35 / 0.07
0.44×0.56
0.15
0.19
G1
humanoid
1.00
1.00
0.48 / 0.05
0.48×0.40
0.10
0.27
TABLE I: Robots, Template Parameters and Method Parameters
Fig. 3: Benchmarks and robots: (a) fixed layout B1 and (b) random-box layout B2 of the mixed indoor scene; (c) training and zero-shot-test robots.
B2: random boxes, feasible episodes
B1: fixed layout, all episodes
SR (%)
Coll.
SPL
Time
SR (%)
Coll.
SPL
Time
Method
Carter
Spot
H1
G1
M20
Go2-W
TRON1
Mean
(%)
(%)
(s)
Carter
Spot
H1
G1
M20
Go2-W
TRON1
Mean
(%)
(%)
(s)
BC- θ [ 7 ]
65.7
68.2
77.4
71.8
70.6
77.3
67.8
71.2 ∗
9.2
63.8
14.6
82.7
83.6
84.5
83.4
85.8
83.4
76.4
82.8 ∗
3.1
75.7
13.4
DP- θ [ 9 ]
75.4
76.4
80.1
76.1
71.2
78.0
70.8
75.5 ∗
7.9
67.8
14.1
86.7
87.2
86.4
85.6
82.8
85.3
76.7
84.4 ∗
4.0
76.9
13.4
DWA- θ [ 24 ]
44.5
62.4
78.2
58.4
40.5
70.4
24.0
54.1 ∗
2.9
49.1
14.4
69.4
79.8
85.2
73.9
69.4
85.5
51.9
73.6 ∗
0.9
66.9
13.2
MPPI- θ [ 24 ]
95.6
86.6
89.7
90.6
77.8
92.4
49.6
83.2 ∗
3.2
70.1
13.0
98.8
95.0
89.7
94.5
92.8
95.8
70.5
91.0 ∗
1.6
78.0
12.4
TABLE II: Motion Planning Performance Across Seven Embodiments
Fig. 4: Example trajectories through five consecutive goals. Each goal-to-goal segment has a 25 s time limit.
Fig. 5: Validation of the factorization: (a) generalist vs. specialists; (b) world models without spatial memory differing only in conditioning: error increase when a robot is held out (top) and imagined-depth error (bottom).
B2
B1
Variant
SR
Coll.
SPL
SR
Coll.
SPL
Hourglass
81.8
8.9
73.0
88.2
2.7
81.5
w/o θ (normalized commands)
72.6 ∗
11.9
61.7
82.9 ∗
3.9
72.8
w/o egocentric spatial memory
78.3 ∗
9.0
70.1
86.5 ∗
2.9
79.3
untrained memory clearance prediction head
78.6 ∗
8.9
70.9
86.8 ∗
2.9
80.4
2 s instead of 5 s horizon
79.9 ∗
8.5
72.6
87.5
2.9
81.2
TABLE III: Ablation of the Method Components
Fig. 6: Zero-shot transfer to unseen robots and model mismatch: (a) zero-shot outcomes; (b) SR difference to MPPI- θ and Privileged MPPI on the seven training robots, sorted by template error εθ .
Fig. 7: Real robot: (a) Scout Mini, a skid-steer robot; (b–d) indoor test scenes; (e–g) dynamics identification from teleoperated driving.
Fig. 8: Zero-shot deployment on the real robot.
Fig. 9: Offline replay of the run in Fig. 8 (a): real depth and world-model imagination roll-outs following the camera ego-motion or by counterfactual templates.
Collecting contact-rich robot experiences at scale remains a major bottleneck for generalizable manipulation. Beyond data quantity, robot learning also requires diverse experiences across embodiments, viewpoints, and scenes. Human egocentric videos provide abundant physical interactions, but each video captures only a narrow slice of experience under a single body, camera trajectory, and environment. We propose AnyWorld, a cross-embodiment world modeling framework that expands a single human interaction into diverse robot-native rollouts without paired human-robot demonstrations. Our model factorizes an interaction into action, camera, and embodiment: action controls capture the motion structure, camera controls specify viewpoint evolution, and the target embodiment context defines the acting body and its interaction geometry. This formulation enables independent recomposition of embodiment, viewpoint, and scene factors, allowing a single model to generate many robot-domain experiences while preserving the underlying dynamics and object interactions. We train the model with large-scale human interaction pretraining followed by mixed-embodiment fine-tuning. Experiments show that our model supports controllable recomposition across embodiments, viewpoints, and scenes, and we further demonstrate that the generated data can improve manipulation performance on the RoboCasa GR1 tabletop benchmark and a real IRON humanoid robot. Beyond aggregate gains, we test whether unpaired human experience can be recomposed into robot-native video-action pairs that target a policy gap. Controlled IRON interventions correct a spurious completion prior and establish language-grounded spatial target selection; an action-only counterfactual intervention fails to learn the latter reliably, showing that both action calibration and visual recomposition are necessary.
Cheng Chen, Jerry Bai, Jiacheng Wei +11
Nanyang Technological University · XPENG Robotics · Zhejiang University +2
Action-conditioned world models are promising learned simulators for robotic manipulation, yet evaluating them exclusively on training robots fails to reveal whether they capture physical dynamics or merely memorize visual patterns. To answer whether a model can faithfully render a robot it has never seen, we introduce XEWorld, a controlled cross-embodiment testbed for world models that isolates embodiments by evaluating held-out robots within physically identical scenes. Our systematic analysis uncovers a shared architectural bottleneck: current models act primarily as 2D visual pattern matchers whose generalization is governed by visual similarity rather than physical kinematic similarity. Driven by this limitation, they struggle to translate abstract numeric joint actions into coherent visual trajectories, and fail to predict dynamic visual changes from static initial observations. Consequently, successfully rendering an unseen embodiment zero-shot strictly requires heavily grounded cues, specifically pixel-space actions and explicit spatial-temporal alignment. Even when bypassing this zero-shot barrier via few-shot adaptation, the forced appearance recovery triggers catastrophic forgetting of seen embodiments. Together, these failures expose a critical inability to apply learned physical dynamics to novel visual appearances, highlighting that achieving true cross-embodiment generalization requires architectural innovations that decouple visual appearance from underlying physical dynamics.
Yixiang Chen, Jiabing Yang, Yuan Xu +10
New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Shandong University +1
Scalable robot imitation learning relies on large-scale heterogeneous data from diverse robots or body-free data, making Cartesian end-effector actions a key interface for embodiment-agnostic policy learning. However, end-effector-only abstraction leaves Cartesian policies unaware of the deployed robot body, making them brittle under robot-specific constraints such as whole-body collision avoidance. To overcome this limitation, we present EmbodiSteer, a training-free framework that steers embodiment-agnostic visuomotor policies toward zero-shot, embodiment-aware deployment. EmbodiSteer keeps policy learning in Cartesian space while efficiently lifting inference-time diffusion sampling into the target robot's joint space via forward kinematics and Jacobian-based updates. With whole-body collision-aware guidance over joint trajectories after each denoising step, the arm can be steered away from collisions while preserving learned end-effector behavior. Across 9 simulated robots, EmbodiSteer reduces collision rate by 46.1 percentage points and increases task success by 28.5 percentage points over Cartesian-only execution; on two physical robots in highly constrained scenarios, the respective improvements are 90.0 and 36.7 percentage points. Our project page is at https://frankwang67.github.io/EmbodiSteer-Page.
Shihefeng Wang, Kangchen Lv, Mingrui Yu +1
Department of Automation, Tsinghua University · Beijing Key Laboratory of Embodied Intelligence Systems · Institute for Embodied Intelligence and Robotics, Tsinghua University