Cascade world-action models (WAMs) use predicted visual futures to condition robot actions, but inherit costly iterative inference from diffusion-pretrained experts. We ask what a single-step predicted future must preserve to support accurate single-step action prediction. In the evaluated WAM, resampling video noise has little effect on downstream actions, yet naively truncating both streams to one step substantially reduces task success. Video-space supervision only partially closes this gap, and a local sensitivity analysis shows that vision prediction errors have highly unequal consequences for action accuracy. These findings motivate DirectWAM, which converts pretrained video and action experts into single-step predictors without stochastic sampling through action-sufficient distillation, while retaining the explicit future-to-action cascade. After learning a one-step action model, we train the future predictor by decoding actions from its own predicted futures and backpropagating demonstration action error through the expert, with an optional future-latent reconstruction loss. The resulting objective evaluates the student-generated future through its downstream control consequences rather than relying on representation matching alone. DirectWAM achieves 97.1% average success across four LIBERO suites and 85.4% on RoboTwin 2.0, outperforming Flash-WAM at the same one-evaluation-per-stream budget. This advantage has also extended to three real-world tasks that we designed. Compared with the full iterative teacher, it reduces inference latency by 28.0×.
Figures & tables
Figure 1: Overview of DirectWAM. (a) Video randomness produces similar downstream actions. (b) Naive step truncation and visual supervision induce a noticeable performance gap. (c) DirectWAM is a deterministic one-step cascade WAM trained through action-sufficient distillation.
Benchmark
Contexts × samples
ρ(DV,DA)
Mean DA/DV
V → A var ratio
RoboTwin
237×32
0.69
0.025
6.6%→0.030%
LIBERO
64×16
0.80
0.138
1.6%→0.25%
Table 1: Three metrics measuring the effect of video variation on downstream actions. Same-context video-sampling variation is substantially attenuated by the action model.
Figure 2: Closed-loop effects of video-side stochasticity. (a) RoboTwin success rates of the teacher, fixed video noise, and averaging four future predictions. (b) Eight rollouts per scene under different video-noise seeds. Successful trajectories remain closely aligned.
Figure 3: Inference-step reduction and action-sensitive error recovery. (a) Success rates (%) across video/action step budgets. Performance degrades as the number of steps decreases. (b) The action-sensitive component holds only 0.63 – 0.85% of the discrepancy energy yet recovers 19.8 – 34.0% of the offline action-error improvement.
Method
Steps (Video / Action)
RoboTwin 2.0
LIBERO
Clean
Rand.
Average.
Long
Goal
Object
Spatial
Average.
Lingbot Baseline
Full / 50
91.2
90.2
90.7
98.2
93.0
97.2
98.0
96.6
Naive Step Reduction
2 / 1
76.2
71.5
73.9
97.0
90.8
96.8
96.6
95.3
1 / 2
67.8
68.0
67.9
90.8
82.8
94.0
94.2
90.5
1 / 1
57.3
52.4
54.9
88.6
77.8
87.8
91.2
86.4
Flash-WAM
1 / 1
82.1
74.2
78.2
95.8 †
96.0 †
92.6 †
96.0 †
95.1 †
Table 2: Main simulation results for cascade WAMs. Success rates (%) on RoboTwin 2.0 and LIBERO; higher is better. The full iterative LingBot teacher is italicized. Steps are video/action steps; Full denotes 20 video steps for LIBERO and 25 for RoboTwin, following the respective LingBot-VA evaluation configurations. RoboTwin results use 50 episodes per task across all 50 tasks. Bold denotes the better result between learned single-step methods (Flash-WAM and DirectWAM). The † denotes LIBERO results quoted from Flash-WAM because no corresponding checkpoint was released; these entries are therefore not treated as a strictly matched reproduction.
Figure 4: Efficiency and qualitative comparison on RoboTwin. (a) Success–latency trade-off; labels show speedups over full LingBot-VA (5042 ms). (b) Six examples of predicted futures from the full teacher, the one-step teacher, Flash-WAM, and DirectWAM.
Method
Grasp Banana
Pouring Water
Place Teapot
LingBot-VA
95 (19/20)
45 (9/20)
60 (12/20)
DirectWAM
95 (19/20)
55 (11/20)
60 (12/20)
Flash-WAM
90 (18/20)
10 (2/20)
40 (8/20)
Table 3: Real-world task success rates. Each method is evaluated over 20 trials per task. Results are reported as success rates (%) with successful/total trials. Bold indicates the best result per task.
Figure 5: Real-world task trajectories. Six keyframes per task are ordered from left to right. Each keyframe pairs the external view (left) with the wrist view (right).
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Contexts × samples
ρ(DV,DA)
Mean DA/DV
RV
RA
RMS attenuation
RoboTwin
237×32
0.69
0.025
6.6%
0.030%
20×
LIBERO
64×16
0.80
0.138
1.6%
0.25%
3×
Appendix
Table 4: Offline propagation of video-sampling variation. All actions within a benchmark are evaluated with the same frozen single-step model. RV and RA are fractions of total variance. The final column reports the separately recorded median RMS attenuation.
Model
Standard
Fixed 123
Fixed 456
Fixed 789
Fixed-noise mean
Avg-4
Iterative teacher
91.6
92.4
91.6
89.6
91.2
92.0
Single-step model
87.0
87.4
87.6
89.4
88.1
87.0
Appendix
Table 5: Video-sampling interventions on RoboTwin Clean. Success rates are percentages over 500 episodes per entry (50 tasks, 10 episodes per task). Fixed-noise mean is the arithmetic mean of the three seed-specific success rates.
Sample set
PCA-8
PCA-16
Same-context futures
0/237 (0.0%)
1/237 (0.4%)
Matched Gaussian controls
0.16%
0.0%
Two-context mixtures
97.4%
99.5%
Appendix
Table 6: Multimodality diagnostic and controls. Real-data entries give detected contexts over the 237-context audited set. Control entries report detection rates.
Figure 6: Stage-2 training dynamics. The action expert is frozen during the plotted phase. RoboTwin (left) uses λv=0.01 and shows both action loss and the weighted future-anchor loss. LIBERO-Long (right) uses λv=0 and shows only action loss. The vertical scale is logarithmic.
Prediction
GT-action error ↓
Recovery ↑
Vision-supervised zV
0.257
0%
Full replacement zA
0.216
100%
Selective replacement zsens
0.224
80%
Equal-norm random controls
–
1–2%
Appendix
Table 7: Selective replacement of learned prediction differences. Results use 64 RoboTwin contexts and the same frozen action model. The selected component accounts for 1.9% of the prediction-difference energy and recovers approximately 80% of the pooled offline action-error improvement. Dashes indicate that only the recovery summary is reported for the random controls.
Future representation
Transferred energy
Success rate (%) ↑
Vision-supervised zV
0%
78.2
Selective replacement zV+P(zA−zV)
1.4%
87.3
DirectWAM zA
100%
87.0
Appendix
Table 8: Closed-loop selective replacement of learned prediction differences. We keep the same downstream action model fixed and vary only the future representation supplied to it. The selective correction transfers only the locally action-sensitive component of the difference between the vision-supervised and DirectWAM futures.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
FID ↓
KID ×103↓
Teacher, 25 steps
30.37
0.938
0.051
2.8
0.56
Teacher, 8 steps
30.16
0.936
0.054
3.5
0.93
Teacher, 4 steps
29.64
0.932
0.060
5.0
1.78
Teacher, 2 steps
28.57
0.920
0.075
8.1
3.74
Teacher, 1 step
25.80
0.885
0.123
17.4
10.24
Flash-WAM
26.62
0.928
0.077
10.5
4.11
Appendix
Table 9: Visual prediction quality. Teacher variants use guidance scale 5 with the indicated number of video denoising steps. Flash-WAM and DirectWAM use one step. Bold indicates the best value among the listed methods.
World Action Models (WAMs) improve robot policy learning by jointly modeling future visual dynamics and actions. However, their scalability and generalization remain constrained by their reliance on costly expert demonstrations. We challenge this by asking whether future supervision for WAMs must originate from target-task expert trajectories. In this paper, we propose Vid2WAM, an offline distillation framework that transfers visual diffusion priors from a large video foundation model into a compact WAM student. Given an observation and language instruction, Vid2WAM distills supervision through two complementary channels: task-conditioned future rollouts directly supervise the student's future prediction branch, while an inverse dynamics model recovers embodiment-specific pseudo-actions for action learning. To robustly integrate synthetic and real supervision, we introduce source-aware residual action adaptation that learns source-specific corrections around a shared action backbone and mitigates interference from noisy pseudo-actions. During inference, both the video teacher and inverse dynamics model are discarded, leaving only the WAM student for efficient deployment. Simulation and real-world experiments demonstrate that Vid2WAM improves novel-task generalization and data efficiency under limited expert demonstrations while preserving low-latency inference.
Chenhao Qiu, Ruixiang Wang, Runyi Zhao +7
Fudan University · The Chinese University of Hong Kong, Shenzhen · Shanghai Innovation Institute
World Action Models (WAMs) connect visual prediction with robot control, but supplying predictive context often requires expensive future-video generation. Direct policies avoid this cost but lack an explicit interface for accessing future-indexed predictive information. We introduce ForeWAM, a World Action Model that separates forecasting from rendering to expose and shape latent predictive context for efficient control. Its core mechanism, Future-KV, performs a single Video DiT prefill over the current visual latent and noise-initialized future slots, then reuses the resulting key-value states throughout action denoising. To make this context relevant to control, we introduce dynamics registers supervised by latent actions from a frozen teacher during training, encouraging representations of interaction-induced transitions. This reusable context supports a lightweight, single-layer action decoder. We evaluate ForeWAM on LIBERO, LIBERO-Plus, RoboCasa, and real-world manipulation tasks. Without additional policy-level embodied pretraining, ForeWAM improves RoboCasa success by 9.7 percentage points over Fast-WAM at the same budget of 50 demonstrations per task, reaching 59.2%. With a single-layer decoder, it achieves 77.6% success on LIBERO-Plus and reduces policy-query latency to 88.7 ms on an NVIDIA A800, delivering a 6.27-fold speedup over Fast-WAM. These results show that latent predictive computation provides useful foresight for robust, efficient control without explicit future-video generation.
Jiakai Huang, Zhongbo Wu, Siyu Xu +5
Shanghai Jiao Tong University · ACE Robotics · Nanyang Technological University
World-Action Models (WAMs) have emerged as a promising paradigm for embodied control by coupling future visual prediction with action generation. However, most existing WAMs rely on photorealistic future prediction, which incurs high inference latency and makes real-time robot deployment difficult. This motivates a more efficient WAM design that preserves the control benefits of future visual prediction while reducing its inference cost. We introduce Efficient-WAM, a World-Action Model that reduces the cost of future imagination while preserving its control benefit. Efficient-WAM improves inference efficiency via a compact video expert transferred from WAN-2.2-5B, token-sparse video latents, and asymmetric video-action denoising that allocates fewer sampling steps to video than to actions. Instead of optimizing the future branch for visual fidelity, Efficient-WAM treats future video prediction as a compact guidance signal for action generation. Comprehensive experiments on RoboTwin 2.0 and real-world manipulation tasks show that Efficient-WAM maintains strong action performance despite visibly coarse future predictions. While maintaining competitive control capabilities, our 1B-parameter model can reduce per-chunk latency to around 100 ms during physical deployment, achieving a 30x speedup over existing WAMs.
Jiajun Li, Tiecheng Guo, Yifan Ye +9
1The University of Hong Kong · 2Peking University · 3Muka Robotics +2