Organizations: National University of Singapore · RoboScience · The Hong Kong University of Science and Technology (Guangzhou) · Zhejiang University · Harbin Institute of Technology (Shenzhen) · Tsinghua University · Nanyang Technological University · Xi’an Jiaotong-Liverpool University
World-Action Models improve robot control by predicting future visual dynamics, but iterative video denoising incurs substantial inference overhead. Recent approaches reduce this cost through partial future denoising, yet typically rely on fixed refinement budgets. Our empirical observations reveal that preferred refinement depths vary across tasks and states, while different future regions exhibit varying relevance to action generation. Motivated by these findings, we propose Elastic-WAM, which adaptively allocates world refinement computation across temporal denoising time and spatial future space. A unified spatiotemporal refinement demand field governs when to stop world refinement, how far to advance each denoising step, and which regions to refresh. We parameterize this field with a shared temporal basis and spatial amplitudes, and learn it through a three-stage action-grounded supervised procedure. At inference, the learned field guides recurrent sparse world refinement to improve task success while reducing inference overhead. Evaluations across diverse simulation benchmarks and real-world long-horizon manipulation demonstrate state-of-the-art control performance with reduced inference latency.
Figures & tables
Figure 2: Pipeline of Elastic-WAM. Top: The inference pipeline starts with dense joint refinement, followed by adaptive sparse world refinement with state-dependent denoising progression and spatial token selection. Once the world is committed, action refinement continues using the cached world representations. Bottom: The unified spatiotemporal refinement field consists of (a) a temporal branch that predicts a shared Beta distribution over denoising progression, (b) a spatial branch that estimates group-wise refinement amplitudes, and (c) a unified field that determines WHEN to commit the world, HOW FAR to advance denoising, and WHERE to refresh future.
Table 2Table 3
Figure 3: Qualitative comparison between Joint-WAM (top) and Elastic-WAM (bottom). (a) Task outcomes. (b) Action-to-video attention maps. (c) Spatial distribution of future video token update counts. Elastic-WAM achieves successful control through selective refinement while preserving the full attention context.
Figure 4: Real-world evaluation settings. We evaluate under three conditions: Nominal, Lighting OOD, and Object OOD. Lighting OOD introduces strong illumination changes, while Object OOD replaces the white cup with a silver-colored cup of a slightly different shape.
Nominal
Lighting OOD
Object OOD
Method
SR
PS
SR
PS
SR
PS
Fast-WAM [ 40 ]
0.0
12.5
0.0
5.0
0.0
10.0
Joint-WAM [ 47 ]
30.0
37.5
30.0
35.0
30.0
35.0
Elastic-WAM (Ours)
50.0
57.5
50.0
52.5
30.0
47.5
Table 7: Real-world long-horizon manipulation results. SR denotes full-task success rate (%), and PS denotes average task progress (%). The best results are highlighted in bold .
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Precision-sensitive manipulation on RoboCasa-GR1. Qualitative comparisons between Fast-WAM (top) and Elastic-WAM (bottom) on (a) Bottle into Cabinet and (b) Can into Drawer. Keyframes progress from left to right, highlighting the final placement stage. Fast-WAM fails to complete the placement, whereas Elastic-WAM successfully finishes both tasks.
Figure 6: Qualitative visualization of real-world long-horizon manipulation. Keyframes illustrate a successful four-stage tea-preparation rollout using Elastic-WAM. Stage 1: Grasping, lifting, aligning, and placing the white cup on the uncovered teapot. Stage 2: Grasping and manipulating the covered teapot to pour tea into the white cup. Stage 3: Retrieving and returning the white cup to its original position. Stage 4: Pouring tea into the black cup and returning the uncovered teapot. The head-camera keyframes and corresponding wrist-camera views illustrate the progression of robot actions throughout the task.