Organizations: National University of Singapore · RoboScience · The Hong Kong University of Science and Technology (Guangzhou) · Zhejiang University · Harbin Institute of Technology (Shenzhen) · Tsinghua University · Nanyang Technological University · Xi’an Jiaotong-Liverpool University
World-Action Models improve robot control by predicting future visual dynamics, but iterative video denoising incurs substantial inference overhead. Recent approaches reduce this cost through partial future denoising, yet typically rely on fixed refinement budgets. Our empirical observations reveal that preferred refinement depths vary across tasks and states, while different future regions exhibit varying relevance to action generation. Motivated by these findings, we propose Elastic-WAM, which adaptively allocates world refinement computation across temporal denoising time and spatial future space. A unified spatiotemporal refinement demand field governs when to stop world refinement, how far to advance each denoising step, and which regions to refresh. We parameterize this field with a shared temporal basis and spatial amplitudes, and learn it through a three-stage action-grounded supervised procedure. At inference, the learned field guides recurrent sparse world refinement to improve task success while reducing inference overhead. Evaluations across diverse simulation benchmarks and real-world long-horizon manipulation demonstrate state-of-the-art control performance with reduced inference latency.
Figures & tables
Figure 2: Pipeline of Elastic-WAM. Top: The inference pipeline starts with dense joint refinement, followed by adaptive sparse world refinement with state-dependent denoising progression and spatial token selection. Once the world is committed, action refinement continues using the cached world representations. Bottom: The unified spatiotemporal refinement field consists of (a) a temporal branch that predicts a shared Beta distribution over denoising progression, (b) a spatial branch that estimates group-wise refinement amplitudes, and (c) a unified field that determines WHEN to commit the world, HOW FAR to advance denoising, and WHERE to refresh future.
Table 2Table 3
Figure 3: Qualitative comparison between Joint-WAM (top) and Elastic-WAM (bottom). (a) Task outcomes. (b) Action-to-video attention maps. (c) Spatial distribution of future video token update counts. Elastic-WAM achieves successful control through selective refinement while preserving the full attention context.
Figure 4: Real-world evaluation settings. We evaluate under three conditions: Nominal, Lighting OOD, and Object OOD. Lighting OOD introduces strong illumination changes, while Object OOD replaces the white cup with a silver-colored cup of a slightly different shape.
Nominal
Lighting OOD
Object OOD
Method
SR
PS
SR
PS
SR
PS
Fast-WAM [ 40 ]
0.0
12.5
0.0
5.0
0.0
10.0
Joint-WAM [ 47 ]
30.0
37.5
30.0
35.0
30.0
35.0
Elastic-WAM (Ours)
50.0
57.5
50.0
52.5
30.0
47.5
Table 7: Real-world long-horizon manipulation results. SR denotes full-task success rate (%), and PS denotes average task progress (%). The best results are highlighted in bold .
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Precision-sensitive manipulation on RoboCasa-GR1. Qualitative comparisons between Fast-WAM (top) and Elastic-WAM (bottom) on (a) Bottle into Cabinet and (b) Can into Drawer. Keyframes progress from left to right, highlighting the final placement stage. Fast-WAM fails to complete the placement, whereas Elastic-WAM successfully finishes both tasks.
Figure 6: Qualitative visualization of real-world long-horizon manipulation. Keyframes illustrate a successful four-stage tea-preparation rollout using Elastic-WAM. Stage 1: Grasping, lifting, aligning, and placing the white cup on the uncovered teapot. Stage 2: Grasping and manipulating the covered teapot to pour tea into the white cup. Stage 3: Retrieving and returning the white cup to its original position. Stage 4: Pouring tea into the black cup and returning the uncovered teapot. The head-camera keyframes and corresponding wrist-camera views illustrate the progression of robot actions throughout the task.
World-Action Models (WAMs) have emerged as a promising paradigm for embodied control by coupling future visual prediction with action generation. However, most existing WAMs rely on photorealistic future prediction, which incurs high inference latency and makes real-time robot deployment difficult. This motivates a more efficient WAM design that preserves the control benefits of future visual prediction while reducing its inference cost. We introduce Efficient-WAM, a World-Action Model that reduces the cost of future imagination while preserving its control benefit. Efficient-WAM improves inference efficiency via a compact video expert transferred from WAN-2.2-5B, token-sparse video latents, and asymmetric video-action denoising that allocates fewer sampling steps to video than to actions. Instead of optimizing the future branch for visual fidelity, Efficient-WAM treats future video prediction as a compact guidance signal for action generation. Comprehensive experiments on RoboTwin 2.0 and real-world manipulation tasks show that Efficient-WAM maintains strong action performance despite visibly coarse future predictions. While maintaining competitive control capabilities, our 1B-parameter model can reduce per-chunk latency to around 100 ms during physical deployment, achieving a 30x speedup over existing WAMs.
Jiajun Li, Tiecheng Guo, Yifan Ye +9
1The University of Hong Kong · 2Peking University · 3Muka Robotics +2
World Action Models (WAMs) jointly generate robot actions and predict future world states, transferring priors from video pretraining to robot control. However, future visual prediction is computationally expensive, so existing WAMs often rely on long action chunks to amortize inference cost across control steps, at the cost of closed-loop responsiveness. We present \method, a dual-system WAM that preserves broader-horizon world-action generation while enabling high-frequency closed-loop action updates by decoupling global planning and local refinement. \systwo periodically performs high-noise bidirectional denoising over a broader world-action chunk to establish a global plan, while wrist-only \sysone extracts a temporally aligned short window from the intermediate denoising state and completes low-noise refinement using the latest wrist observations, which provide action-aligned cues about local geometry, motion, and contact during interaction. The two systems operate asynchronously along a shared denoising trajectory: each global plan is reused across multiple local updates, while \sysone repeatedly incorporates fresh interaction feedback. Across zero-shot manipulation tasks on Franka and Galbot, \method improves success over the strongest evaluated baseline by 4.5 percentage points on average, while achieving a 16.6× critical-path speedup. Further studies show that role-matched egocentric and UMI data improve success by 14 percentage points, and that the decoupled design naturally supports edge--cloud deployment with substantially lower communication overhead than the baseline.
Yixin Zheng, Jiangran Lyu, Yuntian Deng +6
Institute of Automation, Chinese Academy of Sciences · Galbot · Peking University +2
World Action Models (WAMs) couple action generation with future visual prediction for robotic manipulation. However, completing the joint video-action denoising process at each replanning cycle incurs substantial latency, delaying action updates and limiting closed-loop responsiveness. We present Rolling-WAM, a formulation that distributes joint denoising across successive replanning cycles. Our method maintains a sliding window of video-action chunks at staggered noise levels. At each step, a rolling noise schedule fully denoises the imminent action chunk for execution, while partially refining farther-future chunks. As the window advances with new camera observations, the retained future chunks continue their denoising process. This distributes the computational cost over time while carrying an evolving visual-action context across chunk boundaries. Evaluations on LIBERO, RoboTwin, and a real-world Unitree G1 humanoid show that Rolling-WAM achieves competitive manipulation performance. By removing the need to denoise the entire prediction horizon from scratch, it delivers a 4.5x steady-state replanning speedup over standard joint WAMs.
Yinghua Zhou, Junjie Ye, Yiqi Zhao +8
University of Southern California · Brown University · Fudan University +1