RISE: Adaptive Imagination for World Action Models

Hongbo Lu; Liang Yao; Chenghao He; Hao Han; Fan Liu; Wenlong Liao; Tao He; Pai Peng

cs.CVAug 20, 2026

RISE: Adaptive Imagination for World Action Models

Authors: Hongbo Lu, Liang Yao, Chenghao He, Hao Han, Fan Liu, Wenlong Liao, Tao He, Pai Peng

Organizations: COWARobot Co. Ltd · Shanghai Jiao Tong University · Hohai University

Abstract

World Action Models (WAMs) improve planning by incorporating future world evolution into action generation, yet existing methods allocate a fixed imagination budget to every scene. We propose RISE (Refining Imagination through SElective Rollout), a system-level adaptive imagination framework that makes sequential Roll/Stop decisions according to the expected planning benefit of continued rollout. At each step, a Latent Evaluator estimates the risk revealed by the current prefix and how much planning could improve if imagination continues, while a Rollout Gate weighs this expected benefit against additional computation cost. Since factual driving logs expose only one realized future, we further construct CounterDrive, a counterfactual dataset with diverse outcomes and risk levels, to enrich future dynamics and provide localized risk supervision. Each retained sample undergoes expert verification and annotation of trajectory validity, incident onset, and causal category, providing a reusable resource for safety-critical world-modeling research. Experiments on NAVSIM and nuScenes show that RISE achieves the best overall planning performance while reducing unnecessary rollout, with additional transfer results supporting its plug-in generality across WAM architectures.

Figures & tables

Figure 1: Comparison of various imagination strategies in WAMs. (a) Imagine and Plan jointly uses observed and predicted latents for planning, (b) Imagine then Plan performs future rollout before action generation, and (c) No Imagination plans directly from the observation. (d) RISE introduces a lightweight Scheduler that adaptively routes computation to further prediction or final planning.

Figure 2: Overview of RISE. The WAM learns future prediction and variable-prefix planning. The Latent Evaluator predicts a Risk Profile

R_{h}

and a Future Planning Gain Profile

B_{h}

, supervised by counterfactual risks and all-horizon planning outcomes. The Rollout Gate balances the predicted gain against computation cost, routing

x_{h}>0

to further rollout and

x_{h}\leq 0

to planning.

Method	L2 (m) $\downarrow$				Collision Rate $\downarrow$
Method	1s	2s	3s	Avg.	1s	2s	3s	Avg.
BEV-Planner	0.30	0.52	0.83	0.55	0.10	0.37	1.30	0.59
LAW	0.26	0.57	1.01	0.61	0.14	0.21	0.54	0.30
World4Drive	0.23	0.47	0.81	0.50	0.02	0.12	0.33	0.16
WorldRFT	0.21	0.44	0.76	0.47	0.10	0.11	0.23	0.15
DAWN	0.17	0.31	0.52	0.33	0.00	0.10	0.23	0.11

Table 1: Performance on nuScenes.

Method	NC $\uparrow$	DAC $\uparrow$	EP $\uparrow$	C $\uparrow$	TTC $\uparrow$	PDMS $\uparrow$
DrivingGPT	98.9	90.7	79.7	95.6	94.9	82.4
LAW	97.4	93.3	78.8	100	91.9	83.8
World4Drive	97.4	94.3	79.9	100	92.8	85.1
Epona	97.9	95.1	80.4	99.9	93.8	86.2
DriveVLA-W0	98.4	95.3	80.9	100	95.2	87.2
PWM	98.6	95.9	81.8	100	95.4	88.1

Table 2: Performance on NAVSIMv1.

Method	NC $\uparrow$	DAC $\uparrow$	DDC $\uparrow$	TL $\uparrow$	EP $\uparrow$	TTC $\uparrow$	LK $\uparrow$	HC $\uparrow$	EC $\uparrow$	EPDMS $\uparrow$
DAWN [ 16 ]	97.3	92.0	99.1	99.7	87.4	96.6	96	98.3	85.5	83.2
DreamerAD [ 30 ]	98.0	97.2	99.5	99.8	87.8	97.4	97.5	98.3	72.4	85.1
DriveLaW [ 26 ]	98.7	96.9	99.6	99.8	87.5	98.3	97.6	98.4	77.4	88.6
EponaV2 [ 27 ]	98.5	97.4	99.5	99.9	87.9	98.1	97.7	98.2	77.4	88.9
Latent-WAM [ 22 ]	98.1	97.3	99.6	99.8	87.7	97.3	97.6	98.1	87.3	89.3
DriveFuture [ 6 ]	98.8	99.1	99.6	99.9	86.6	98.4	96.4	98.3	74.8	89.9

Table 3: Performance on NAVSIMv2.

Scheduler	CounterDrive	EPDMS $\uparrow$	PDMS $\uparrow$
		88.9	89.7
	✓	89.8	90.5
✓		90.4	91.2
✓	✓	90.8	91.5

Table 4: Ablation of the Scheduler and CounterDrive on NAVSIM. Both components improve performance.

Method	# Avg. Rollout	Latency (ms)	EPDMS $\uparrow$
Random Stop	2.03	264.075	89.5
Latent Margin	2.98	308.532	89.7
Scheduler	2.40	287.429	90.8

Table 5: Ablations on different adaptive rollout strategies. Random Stop stops at a random horizon from 0 to 4, while Latent Margin stops when consecutive latents converge.

Figure 3: Performance under different rollout depths.

Figure 4: Representative scenes grouped by their preferred rollout depths. Scenes with denser traffic and more involved interactions generally benefit from additional rollout.

Figure 5: Qualitative trajectory comparison on NAVSIM at depths $h=4$ to $h=0$ . The rows show front-camera views and corresponding bird’s-eye-view scenes. Human, Drive-JEPA, and RISE trajectories are shown in green, yellow, and red, respectively. RISE remains aligned with human trajectories and feasible lane geometry across scene-dependent rollout depths.

CounterDrive	AUC $\uparrow$				Acc $\uparrow$
CounterDrive	@1s	@2s	@3s	@4s	Acc $\uparrow$
	0.49	0.52	0.51	0.50	0.51
✓	0.96	0.95	0.96	0.93	0.96

Table 6: Effect of CounterDrive on safety-critical evaluation. AUC@

h

measures hazard–safe risk ranking at horizon

h

. Acc measures accident recognition.

Method	NC $\uparrow$	DAC $\uparrow$	EP $\uparrow$	C $\uparrow$	TTC $\uparrow$	PDMS $\uparrow$
DAWN	98.7	95.9	84.3	100	96.0	89.1
+ Scheduler	99.9	96.8	87.0	100	98.3	90.3

Table 7: Transfer of the RISE Scheduler to DAWN. Adding the Scheduler consistently improves planning performance without modifying the underlying WAIM.

Figure 6: Trajectory predictions at different fixed rollout depths. Predictions remain nearly identical in simple scenes but vary noticeably in turning and intersection cases.

Latent Evaluator	Gate	EPDMS $\uparrow$	PDMS $\uparrow$
		89.8	90.5
	✓	90.6	91.2
✓		90.3	90.9
✓	✓	90.8	91.5

Table 8: Ablation of the Latent Evaluator and Rollout Gate on NAVSIM with all other components fixed.

Strategy	Training Distribution	EPDMS $\uparrow$
Uniform	$[0.20,0.20,0.20,0.20,0.20]$	89.8
Extremes	$[0.50,0,0,0,0.50]$	87.4
Short-heavy	$[0.225,0.225,0.225,0.225,0.10]$	89.6
No-full	$[0.25,0.25,0.25,0.25,0]$	89.2

Table 9: Ablation of prefix-depth sampling for variable-prefix Planner training on NAVSIM. The distributions correspond to

h=0,\ldots,4

Steps $m$	PDMS $\uparrow$	EPDMS $\uparrow$
0	90.5	89.8
1	90.7	90.1
2	90.9	90.2
4	90.9	90.2
8	90.9	90.2

Table 10: Effect of the number of risk-refinement steps on NAVSIM.

$\lambda$	EPDMS $\uparrow$
0	90.0
0.001	90.7
0.005	90.8
0.01	88.6
0.05	88.4

Table 11: Effect of the computation-cost weight

\lambda

on NAVSIM planning performance.

Component	Prompt content and constraints
A. Near-field key-frame description
VLM input	One front-facing dashcam key frame.
Role	You are a near-field object-description assistant for dashcam images. Given a front-facing dashcam image, describe only objects immediately surrounding the camera in exactly one sentence. Do not analyze accidents and do not describe the ego vehicle.
Spatial range	Only consider objects within approximately 30 m in front of the camera and 20 m on either side of the road.
Include	Include vehicles immediately ahead using their vehicle type only, such as “double-decker bus” or “SUV.” Also include nearby roadside objects, such as trees, guardrails, building facades, streetlights, or traffic signs. Mention the nearby road surface only when necessary.
Exclude	Do not mention city names, street names, distant landmarks, distant buildings, weather, illumination, time of day, detailed traffic conditions, or inferred geographical locations.

Table 12: VLM instructions for constructing the variable components of the Wan prompt.

Component	Content
Wan input	The source key frame and the concatenated text prompt described below.
Fixed system prompt	“Dashcam perspective with fixed camera height and pitch. The camera height must remain unchanged, and no abrupt camera cuts may occur. The video must remain in a first-person driving view, and the ego vehicle must never appear in the image.”
Key-frame description	A one-sentence near-field scene description generated by the VLM using Part A of Table 12 .
Accident description	A one-sentence accident description generated by the VLM using Part B of Table 12 . The sentence specifies the accident location and involved object.
Concatenation rule	<FIXED CAMERA PROMPT> + <KEY-FRAME DESCRIPTION> + <ACCIDENT DESCRIPTION>
Complete example	“Dashcam perspective with fixed camera height and pitch. The camera height must remain unchanged, and no abrupt camera cuts may occur. The video must remain in a first-person driving view, and the ego vehicle must never appear in the image. An SUV is immediately ahead, and a sedan is traveling in the adjacent left lane. The ego vehicle directly rear-ends the SUV ahead.”

Table 13: Construction of the prompt supplied to Wan for counterfactual video generation.

World Model-Based Planning

World model-based planning uses models of how environments change to compare imagined actions and choose goal-directed behavior.

See all 261 papers

When to Trust Imagination: Adaptive Action Execution for World Action Models
May 7, 2026Rui Wang, Yue Zhang, Canyang Chen +3Long-Horizon Robotic Manipulation World Models for Robotics
Southern University of Science and Technology, Shenzhen, China · The University of Hong Kong, Hong Kong, China
The DAWN of World-Action Interactive Models
May 12, 2026Hongbo Lu, Liang Yao, Chenghao He +6Diffusion-Based Planning World Model-Based Planning
1COWARobot Co. Ltd · 2Shanghai Jiao Tong University · 3Hohai University
WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models
Sep 3, 2026Chenhao Zhang, Hanyu Zhao, Hang Cheng +2Robustness of VLA Models World Models for Robotics
1Tsinghua University · 2Beijing Academy of Artificial Intelligence (BAAI)

May 7, 2026J/K move · Enter open · S save

When to Trust Imagination: Adaptive Action Execution for World Action Models

Rui Wang, Yue Zhang, Canyang Chen, Jiehong Lin+2

Southern University of Science and Technology, Shenzhen, China · The University of Hong Kong, Hong Kong, China

Long-Horizon Robotic Manipulation World Models for Robotics Action-Conditioned World Models Robot Task Planning Model-Based Planning World Action Models

Open paper AskPDF arXiv

World Action Models (WAMs) jointly predict future visual observations and actions, but typically execute a fixed number of actions before replanning, regardless of whether the imagined future remains consistent with reality. We formulate adaptive WAM execution as future--reality verification and propose Future Forward Dynamics Causal Attention (FFDC), a lightweight verifier that reasons over predicted actions, predicted visual dynamics, real observations, and instruction-conditioned semantics. FFDC allows continued execution when a plan remains reliable and triggers replanning when its validity deteriorates. We combine verification with Mixture-of-Horizon Training to improve long-horizon trajectory coverage. On RoboTwin, FFDC-WAM achieves the highest average success rate among the evaluated methods in both clean and randomized settings. Under randomization, it improves success by 2.54 percentage points over Base-Motus while reducing WAM planning calls by 69.10% and completion time by 33.88%. Notably, across six real-world tasks on a Unitree G1D robot, FFDC-WAM improves average success by 9.17% over Base-Motus-LC16 while reducing calls by 74.06% and completion time by 51.31%.

Abstract

World Model-Based Planning

Explore similar work

When to Trust Imagination: Adaptive Action Execution for World Action Models

The DAWN of World-Action Interactive Models

WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models