World Action Models (WAMs) improve planning by incorporating future world evolution into action generation, yet existing methods allocate a fixed imagination budget to every scene. We propose RISE (Refining Imagination through SElective Rollout), a system-level adaptive imagination framework that makes sequential Roll/Stop decisions according to the expected planning benefit of continued rollout. At each step, a Latent Evaluator estimates the risk revealed by the current prefix and how much planning could improve if imagination continues, while a Rollout Gate weighs this expected benefit against additional computation cost. Since factual driving logs expose only one realized future, we further construct CounterDrive, a counterfactual dataset with diverse outcomes and risk levels, to enrich future dynamics and provide localized risk supervision. Each retained sample undergoes expert verification and annotation of trajectory validity, incident onset, and causal category, providing a reusable resource for safety-critical world-modeling research. Experiments on NAVSIM and nuScenes show that RISE achieves the best overall planning performance while reducing unnecessary rollout, with additional transfer results supporting its plug-in generality across WAM architectures.
Figures & tables
Figure 1: Comparison of various imagination strategies in WAMs. (a) Imagine and Plan jointly uses observed and predicted latents for planning, (b) Imagine then Plan performs future rollout before action generation, and (c) No Imagination plans directly from the observation. (d) RISE introduces a lightweight Scheduler that adaptively routes computation to further prediction or final planning.
Figure 2: Overview of RISE. The WAM learns future prediction and variable-prefix planning. The Latent Evaluator predicts a Risk Profile Rh and a Future Planning Gain Profile Bh , supervised by counterfactual risks and all-horizon planning outcomes. The Rollout Gate balances the predicted gain against computation cost, routing xh>0 to further rollout and xh≤0 to planning.
Method
L2 (m) ↓
Collision Rate ↓
1s
2s
3s
Avg.
1s
2s
3s
Avg.
BEV-Planner
0.30
0.52
0.83
0.55
0.10
0.37
1.30
0.59
LAW
0.26
0.57
1.01
0.61
0.14
0.21
0.54
0.30
World4Drive
0.23
0.47
0.81
0.50
0.02
0.12
0.33
0.16
WorldRFT
0.21
0.44
0.76
0.47
0.10
0.11
0.23
0.15
DAWN
0.17
0.31
0.52
0.33
0.00
0.10
0.23
0.11
Table 1: Performance on nuScenes.
Method
NC ↑
DAC ↑
EP ↑
C ↑
TTC ↑
PDMS ↑
DrivingGPT
98.9
90.7
79.7
95.6
94.9
82.4
LAW
97.4
93.3
78.8
100
91.9
83.8
World4Drive
97.4
94.3
79.9
100
92.8
85.1
Epona
97.9
95.1
80.4
99.9
93.8
86.2
DriveVLA-W0
98.4
95.3
80.9
100
95.2
87.2
PWM
98.6
95.9
81.8
100
95.4
88.1
Table 2: Performance on NAVSIMv1.
Method
NC ↑
DAC ↑
DDC ↑
TL ↑
EP ↑
TTC ↑
LK ↑
HC ↑
EC ↑
EPDMS ↑
DAWN [ 16 ]
97.3
92.0
99.1
99.7
87.4
96.6
96
98.3
85.5
83.2
DreamerAD [ 30 ]
98.0
97.2
99.5
99.8
87.8
97.4
97.5
98.3
72.4
85.1
DriveLaW [ 26 ]
98.7
96.9
99.6
99.8
87.5
98.3
97.6
98.4
77.4
88.6
EponaV2 [ 27 ]
98.5
97.4
99.5
99.9
87.9
98.1
97.7
98.2
77.4
88.9
Latent-WAM [ 22 ]
98.1
97.3
99.6
99.8
87.7
97.3
97.6
98.1
87.3
89.3
DriveFuture [ 6 ]
98.8
99.1
99.6
99.9
86.6
98.4
96.4
98.3
74.8
89.9
Table 3: Performance on NAVSIMv2.
Scheduler
CounterDrive
EPDMS ↑
PDMS ↑
88.9
89.7
✓
89.8
90.5
✓
90.4
91.2
✓
✓
90.8
91.5
Table 4: Ablation of the Scheduler and CounterDrive on NAVSIM. Both components improve performance.
Method
# Avg. Rollout
Latency (ms)
EPDMS ↑
Random Stop
2.03
264.075
89.5
Latent Margin
2.98
308.532
89.7
Scheduler
2.40
287.429
90.8
Table 5: Ablations on different adaptive rollout strategies. Random Stop stops at a random horizon from 0 to 4, while Latent Margin stops when consecutive latents converge.
Figure 3: Performance under different rollout depths.
Figure 4: Representative scenes grouped by their preferred rollout depths. Scenes with denser traffic and more involved interactions generally benefit from additional rollout.
Figure 5: Qualitative trajectory comparison on NAVSIM at depths h=4 to h=0 . The rows show front-camera views and corresponding bird’s-eye-view scenes. Human, Drive-JEPA, and RISE trajectories are shown in green, yellow, and red, respectively. RISE remains aligned with human trajectories and feasible lane geometry across scene-dependent rollout depths.
CounterDrive
AUC ↑
Acc ↑
@1s
@2s
@3s
@4s
0.49
0.52
0.51
0.50
0.51
✓
0.96
0.95
0.96
0.93
0.96
Table 6: Effect of CounterDrive on safety-critical evaluation. AUC@ h measures hazard–safe risk ranking at horizon h . Acc measures accident recognition.
Method
NC ↑
DAC ↑
EP ↑
C ↑
TTC ↑
PDMS ↑
DAWN
98.7
95.9
84.3
100
96.0
89.1
+ Scheduler
99.9
96.8
87.0
100
98.3
90.3
Table 7: Transfer of the RISE Scheduler to DAWN. Adding the Scheduler consistently improves planning performance without modifying the underlying WAIM.
Figure 6: Trajectory predictions at different fixed rollout depths. Predictions remain nearly identical in simple scenes but vary noticeably in turning and intersection cases.
Latent Evaluator
Gate
EPDMS ↑
PDMS ↑
89.8
90.5
✓
90.6
91.2
✓
90.3
90.9
✓
✓
90.8
91.5
Table 8: Ablation of the Latent Evaluator and Rollout Gate on NAVSIM with all other components fixed.
Strategy
Training Distribution
EPDMS ↑
Uniform
[0.20,0.20,0.20,0.20,0.20]
89.8
Extremes
[0.50,0,0,0,0.50]
87.4
Short-heavy
[0.225,0.225,0.225,0.225,0.10]
89.6
No-full
[0.25,0.25,0.25,0.25,0]
89.2
Table 9: Ablation of prefix-depth sampling for variable-prefix Planner training on NAVSIM. The distributions correspond to h=0,…,4 .
Steps m
PDMS ↑
EPDMS ↑
0
90.5
89.8
1
90.7
90.1
2
90.9
90.2
4
90.9
90.2
8
90.9
90.2
Table 10: Effect of the number of risk-refinement steps on NAVSIM.
λ
EPDMS ↑
0
90.0
0.001
90.7
0.005
90.8
0.01
88.6
0.05
88.4
Table 11: Effect of the computation-cost weight λ on NAVSIM planning performance.
Figure 7: Sample 1 of CounterDrive.
Figure 8: Sample 2 of CounterDrive.
Component
Prompt content and constraints
A. Near-field key-frame description
VLM input
One front-facing dashcam key frame.
Role
You are a near-field object-description assistant for dashcam images. Given a front-facing dashcam image, describe only objects immediately surrounding the camera in exactly one sentence. Do not analyze accidents and do not describe the ego vehicle.
Spatial range
Only consider objects within approximately 30 m in front of the camera and 20 m on either side of the road.
Include
Include vehicles immediately ahead using their vehicle type only, such as “double-decker bus” or “SUV.” Also include nearby roadside objects, such as trees, guardrails, building facades, streetlights, or traffic signs. Mention the nearby road surface only when necessary.
Exclude
Do not mention city names, street names, distant landmarks, distant buildings, weather, illumination, time of day, detailed traffic conditions, or inferred geographical locations.
Table 12: VLM instructions for constructing the variable components of the Wan prompt.
Component
Content
Wan input
The source key frame and the concatenated text prompt described below.
Fixed system prompt
“Dashcam perspective with fixed camera height and pitch. The camera height must remain unchanged, and no abrupt camera cuts may occur. The video must remain in a first-person driving view, and the ego vehicle must never appear in the image.”
Key-frame description
A one-sentence near-field scene description generated by the VLM using Part A of Table 12 .
Accident description
A one-sentence accident description generated by the VLM using Part B of Table 12 . The sentence specifies the accident location and involved object.
Concatenation rule
<FIXED CAMERA PROMPT> + <KEY-FRAME DESCRIPTION> + <ACCIDENT DESCRIPTION>
Complete example
“Dashcam perspective with fixed camera height and pitch. The camera height must remain unchanged, and no abrupt camera cuts may occur. The video must remain in a first-person driving view, and the ego vehicle must never appear in the image. An SUV is immediately ahead, and a sedan is traveling in the adjacent left lane. The ego vehicle directly rear-ends the SUV ahead.”
Table 13: Construction of the prompt supplied to Wan for counterfactual video generation.
World Action Models (WAMs) jointly predict future visual observations and actions, but typically execute a fixed number of actions before replanning, regardless of whether the imagined future remains consistent with reality. We formulate adaptive WAM execution as future--reality verification and propose Future Forward Dynamics Causal Attention (FFDC), a lightweight verifier that reasons over predicted actions, predicted visual dynamics, real observations, and instruction-conditioned semantics. FFDC allows continued execution when a plan remains reliable and triggers replanning when its validity deteriorates. We combine verification with Mixture-of-Horizon Training to improve long-horizon trajectory coverage. On RoboTwin, FFDC-WAM achieves the highest average success rate among the evaluated methods in both clean and randomized settings. Under randomization, it improves success by 2.54 percentage points over Base-Motus while reducing WAM planning calls by 69.10% and completion time by 33.88%. Notably, across six real-world tasks on a Unitree G1D robot, FFDC-WAM improves average success by 9.17% over Base-Motus-LC16 while reducing calls by 74.06% and completion time by 51.31%.
Rui Wang, Yue Zhang, Canyang Chen +3
Southern University of Science and Technology, Shenzhen, China · The University of Hong Kong, Hong Kong, China
A plausible scene evolution depends on the maneuver being considered, while a good maneuver depends on how the scene may evolve. Existing World Action Models (WAMs) largely miss this reciprocity, treating world prediction and action generation as either isolated parallel branches or rigid predict-then-plan pipelines. We formalize this perspective as World-Action Interactive Models (WAIMs), and instantiate it in autonomous driving with DAWN (Denoising Actions and World iNteractive model), a simple yet strong latent generative baseline. DAWN operates in a compact semantic latent space and couples a World Predictor with a World-Conditioned Action Denoiser: the predicted world hypothesis conditions action denoising, while the denoised action hypothesis is fed back to update the world prediction, so that both are recursively refined during inference. Rather than eliminating test-time world evolution altogether or rolling out the full future in pixel space, DAWN performs a short explicit latent rollout that is sufficient to support long-horizon trajectory generation in complex interactive scenes. Experiments show that DAWN achieves strong planning performance and favorable safety-related results across multiple autonomous driving benchmarks. More broadly, our results suggest that interactive world-action generation is a principled path toward truly actionable world models.
Hongbo Lu, Liang Yao, Chenghao He +6
1COWARobot Co. Ltd · 2Shanghai Jiao Tong University · 3Hohai University
Post-training VLA policies typically rely on supervised fine-tuning with costly expert demonstrations or reinforcement learning with expensive and potentially unstable real-world exploration. World models offer a promising alternative by evaluating candidate behaviors through imagined futures, yet effective post-training requires more than accurate prediction: imagination must be scheduled where it is useful, bounded within reliable horizons, and translated into trustworthy policy supervision. In robotic manipulation, the value of imagination varies substantially across execution stages, while extended rollouts can accumulate prediction errors and introduce unreliable learning signals. We introduce WISE (World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models), a unified framework that coordinates when and how world-model imagination is used during policy refinement. WISE selectively invokes imagination at interaction-relevant states, performs bounded multi-view rollouts, evaluates candidate futures using progress and completion signals, and uses their relative outcomes to refine actions generated from real interaction contexts. Extensive experiments with both π0 and π0.5 demonstrate consistent improvements across diverse manipulation tasks while reducing GPU computation time by approximately 80% compared with full imagination. Real-world evaluations further show substantial gains in robustness and generalization under diverse real-world distribution shifts.
Chenhao Zhang, Hanyu Zhao, Hang Cheng +2
1Tsinghua University · 2Beijing Academy of Artificial Intelligence (BAAI)