Visual world models predict the consequences of actions from images, but planning with them is typically done by optimizers such as CEM and MPPI. However, these optimizers stall on long-horizon tasks where progress requires motion the objective does not reward, such as moving around a block before pushing it. This paper revisits kinodynamic sampling-based motion planning for world models, using the learned model as the propagator of a tree search in latent space. Our key insight is that the sampling strategies devised for high-dimensional search in configuration space apply directly to the latent space of a world model, where they rapidly expand through intermediate states that myopic optimizers never sample. We present two exploration strategies for the high-dimensional latent space, nearest-neighbor-guided RRT and projection-based KPIECE, with GPU-native batched expansion for efficient device utilization. We evaluate with an off-the-shelf DINO-WM on four simulated domains and a real-world block-pushing task. Our approaches match or exceed the success rate of CEM and MPPI at up to 13.8x lower planning time in simulation, and improve the success rate by 20% on the real robot.
Figures & tables
Fig. 1 : Planning paradigms and receding-horizon execution in latent space. (a) Shooting methods (CEM) iteratively collapse trajectories toward a mean, risking entrapment in local minima around non-convex obstacles. (b) Tree search (RRT) finds a valid path through the latent space, committing only its first nexec actions. (c) System execution introduces minor drift from the planned path, requiring re-grounding and replanning from the new observation. (d) KPIECE projects states onto a grid and prioritizes under-explored frontier cells (shaded) for subsequent expansions.
Fig. 2 : The four simulation environments . The start image of a representative problem from each environment is overlaid with a green goal position.
Fig. 3 : Qualitative comparison of search exploration on point_maze . The initial plan of the same problem (start in blue, goal outlined in green). The depicted trees are thinned out to be ∼ 5x sparser. MCTS, RRT, and KPIECE reach the goal in the first plan, while CEM and MPPI execute their full budget.
point_maze ( H=10 )
wall ( H=5 )
Planner
Succ. \penalty↑
Time s \penalty↓
Cost \penalty↓
Nodes \penalty↓
Succ. \penalty↑
Time s \penalty↓
Cost \penalty↓
Nodes \penalty↓
RRT (ours)
49/50
0.61 (1.55)
5.0
2 307
39/50
0.26 (0.77)
3.0
770
KPIECE (ours)
48/50
0.53 (1.24)
5.0
1 412
42/50
0.20 (0.41)
3.0
769
RRT + polish (ours)
50/50
2.88 (4.41)
5.5
2 819
43/50
0.56 (0.80)
3.0
769
KPIECE + polish (ours)
49/50
2.36 (3.84)
4.0
1 283
45/50
0.42 (1.14)
2.0
513
MCTS [ 18 ]
48/50
9.95 (10.72)
8.0
1 276
43/50
1.18 (1.02)
5.0
384
TABLE I : Main comparison. Evaluated over N=50 problems per domain across identical world-model checkpoints and MPC settings. Time shows median wall-clock time with mean in parentheses. Cost indicates median solution cost. Nodes denotes median world-model single-step evaluations for a whole problem, summed over its replans (inserted tree nodes for tree methods; samples×horizon×optimization steps taken for CEM and MPPI); it is therefore a multiple of the per-replan budgets in Table II whenever the median solved problem needs more than one replan. Time , Cost and Nodes are all computed exclusively over solved trials. Every arm evaluates the world model in fp16 ( Table III ), so the wall-clock column compares planners rather than numerics.
CEM (Samples × Iters)
Env
Stat
30×5
30×10
100×5
300×10
300×30
Budget
750
1 500
2 500
15 000
45 000
point_maze
Succ.
24/50
19/50
32/50
48/50
50/50 ∗
Med s
1.35
2.98
0.98
4.53
16.07
wall
Succ.
2/50
11/50
41/50
44/50 ∗
44/50
Med s
0.89
1.71
0.40
1.61
7.37
TABLE II : Ablation: CEM budget. Performance across five settings of sample size × optimization steps. Budget indicates world-model state evaluations per replan.
Precision
Rel. Error
CEM succ.
CEM med. s
fp32
—
45/50
45.58
tf32
1.67×10−3
44/50
33.85
fp16
2.54×10−3
46/50
13.33
TABLE III : Ablation: model quantization. Evaluating the world model on pusht ( N=50 ) across different numerical representations. Relative err. refers to the error compared to the fp32 model.
Figure 7
Fig. 5 : Ablation: case study. Visualization of the first search tree in P27 for CEM, RRT, and KPIECE, shown in the end-effector’s (EE) XY position (top right corner) and the block’s XY position (main plot). Planned fixed-horizon trajectory for EE and prediction of block motions are plotted as lines, while the lighter color indicates a deeper node. The star indicates the goal position.
Tier
Planner
Succ.
Repl.
Plan s
Err. mm
1
RRT + polish (ours)
10/10
2.0
10.2
5.1
KPIECE + polish (ours)
10/10
1.0
5.8
3.3
CEM
10/10
1.0
9.4
6.0
MPPI
10/10
1.0
6.0
4.6
2
RRT + polish (ours)
10/10
2.0
10.2
12.2
KPIECE + polish (ours)
10/10
4.0
24.9
13.1
TABLE IV : Real-robot results. YAM arm block pushing ( 30 total problems across 3 tiers). Succ. indicates trials reaching <25 mm final block distance. Repl. is the median MPC replans, Plan s is the median total planning time per problem, and Err. is the mean final block error on successful trials.