Organizations: University of Chinese Academy of Sciences · Beijing Zhongguancun Academy · Institute of Automation, Chinese Academy of Sciences · University of Science and Technology of China · Nankai University · Harbin Institute of Technology · Eastern Institute of Technology, Ningbo
World Action Models (WAMs), often built upon pretrained video-generation world models, have emerged as a promising framework for general-purpose and open-ended robot control. However, their real-world deployment often suffers from discontinuous and jerky execution. Existing methods typically address this issue by either improving inference efficiency or enhancing action continuity across prediction chunks, without jointly addressing both challenges. To bridge this gap, we propose StreamingWAM, a streaming-diffusion framework for WAM-based robot control. StreamingWAM combines a noise ladder for chunk-wise action prediction, a prefix rollout module for cross-chunk continuity, and dense video-history memory for temporally consistent conditioning. Experiments on LIBERO, LIBERO-Plus, RoboTwin 2.0, and real-world robotic tasks demonstrate that StreamingWAM improves action continuity across static and dynamic settings, achieving competitive performance while reducing steady-state replanning latency to 72.9 ms in simulation and 57.3 ms on the real robot. The method also produces smoother action transitions, while the streaming staircase reduces sensitivity to the execution horizon K.
Figures & tables
Figure 1: Overview of our streaming world-action model. (a) Standard policies regenerate a complete action chunk at every replan. (b) StreamingWAM rolls the action buffer forward with one denoising step per replan. (c) StreamingWAM performs strongly across simulation and real-world benchmarks.
Figure 2: Overview of StreamingWAM. (a) Joint video–action architecture with a clean action prefix and staircase denoising. (b) Standard multi-step inference versus single-step streaming inference.
Perturbation type
Method
EP
LIBERO
Cam.
Robot
Lang.
Light
Bkgnd.
Noise
Layout
Overall
With embodied pre-training
π 0.5
✓
96.9
75.4
77.5
85.6
96.9
94.6
89.7
85.7
85.7
GE-ACT
✓
96.5
60.7
77.0
77.4
95.8
86.0
90.9
80.2
80.3
VLA-JEPA
✓
97.2
63.3
67.1
85.4
95.6
93.6
66.3
85.1
79.5
X-VLA
✓
98.1
23.4
89.7
75.7
88.2
96.0
62.7
71.8
71.4
Table 1: Success rates (%) on LIBERO and LIBERO-Plus . The final column reports success pooled over all LIBERO-Plus evaluation instances. FastWAM-Joint follows its official evaluation configuration; detailed inference settings are provided in Table 6 . EP: embodied pre-training. Bold : best per column; underline : second best.
Figure 4: Inference efficiency and action continuity in simulation and on the real robot. Panel (c) is a system-level summary: for Fast-WAM and FastWAM-Joint, the latency coordinate is the standard 10-step latency, while overall success uses the task-specific real-robot deployment in which dynamic tasks use one denoising step.
Figure 5: Real-robot tasks. (a)–(c) Static; (d)–(f) dynamic. Each panel shows one successful rollout.
Figure 6: Real-robot results. Each of the five manipulation tasks (left) reports the number of successful trials out of 50; whack-a-mole (right) reports total valid lit-button activations across 50 × 30 s sessions.
Variant
K=1
2
4
8
16
Spread
Base ( Fast-WAM-Joint )
37.01
50.77
61.03
72.76
69.21
35.75
+ Temporal context (P)
39.14
59.36
65.83
73.71
72.49
34.57
+ Staircase (S)
57.13
60.01
61.39
65.60
53.65
11.95
+ P + S (full model)
75.45
77.22
79.36
78.73
72.86
6.50
w/ single-width K=4
71.52
74.08
75.31
73.74
70.39
4.92
w/ sparse history
56.35
67.80
69.86
68.45
60.43
13.51
Table 3: Ablation studies on the full 10,030-episode LIBERO-Plus evaluation (success rate, %). The top block compares the temporal-context package (P) and streaming staircase (S); the bottom block varies one design choice from the full model. Spread is max − min across the reported K values.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Layers
Hidden dim.
FFN dim.
Heads × dim.
Act./prop.
Cameras
Resolution ( H×W )
LIBERO / LIBERO-Plus
30/30
3,072/1,024
14,336/4,096
24×128
7/8
2, horiz.
224×448
RoboTwin 2.0
30/30
3,072/1,024
14,336/4,096
24×128
14/14
3, top + 2 bottom
384×320
Real robot
30/30
3,072/1,024
14,336/4,096
24×128
8/8
3, vertical
384×224
Appendix
Table 4: Core model configuration by benchmark. Video/Action values are shown as V/A.
Benchmark
Method
Shared Joint stage
Second stage
Total steps
Global batch
LIBERO / LIBERO-Plus
Fast-WAM-Joint
34,720
+ 40,000 Joint
74,720
80→64
StreamingWAM
34,720
+ 40,000 streaming
74,720
80→64
RoboTwin 2.0
Fast-WAM-Joint
135,776
+ 135,776 Joint
271,552
64→64
StreamingWAM
135,776
+ 135,776 streaming
271,552
64→64
Real robot
Fast-WAM-Joint
20,000
+ 30,000 Joint
50,000
128→128
StreamingWAM
20,000
+ 30,000 streaming
50,000
128→128
Appendix
Table 5: Training schedule by benchmark.
LIBERO / LIBERO-Plus
RoboTwin 2.0
Real robot
Method
Steps
H
K
Steps
H
K
Steps
H
K
π0.5 Async
10
10
5
10
50
50
10
16
4
π0.5 RTC
10
10
5
10
50
50
10
16
4
Fast-WAM
10
32
10
10
32
24
10/1
32
4
Faster-WAM
10
32
4
10
32
4
10
32
4
Fast-WAM-Joint Async
10
32
10
10
32
24
10/1
32
4
Appendix
Table 6: Inference hyperparameters used in our evaluations. H is the predicted future-action horizon (the 16 clean prefix actions of StreamingWAM are excluded), and K is the number of actions executed per replan. 10→1 denotes a one-time 10-step warm start followed by one rolling SDP update per replan. For real-robot non-streaming rows, 10/1 means static/dynamic tasks.
Method
H
Steps
n
Latency (ms)
LIBERO-Plus (%)
Fast-WAM-Joint
32
10
30
381.6 ± 6.9
68.8
Fast-WAM
32
10
30
182.6 ± 3.2
51.5
Faster-WAM
32
10
30
139.4 ± 1.7
75.0
Ours (steady)
16
1
300
72.9 ± 0.3
79.4
Ours (initial warm start)
16
10
30
406.2 ± 0.8
—
Real-robot deployment (same RTX 5090)
Appendix
Table 7: Inference latency comparison. All measurements on the same RTX 5090 with CUDA synchronous timing. n : number of timed forward passes.
Method
SR (%)
Trans. jump ↓
Rot. jump ↓
Bnd./int. ratio ↓
Jerk ↓
Fast-WAM
60.2
0.1350
0.02903
1.831
5.775
Fast-WAM-Joint
68.8
0.1379
0.02681
1.840
5.710
Ours
79.0
0.0684
0.00879
1.105
2.341
Ours vs. Fast-WAM-Joint
− 50.4%
− 67.2%
—
− 59.0%
Appendix
Table 8: Action-continuity metrics on LIBERO-Plus (500-episode subset, 1-step inference, K=4 ). ↓ : lower is better.
Suite
BG
Robot
Cam
Lang
Noise
Obj
Light
All
object
Joint
64.1
57.8
49.2
97.7
68.0
77.4
97.3
72.2
Ours
83.9
82.2
83.6
97.7
92.2
83.9
98.0
88.6
spatial
Joint
74.0
66.3
55.6
93.3
72.1
90.4
99.0
78.5
Ours
73.6
83.7
64.4
97.4
87.5
96.4
97.9
86.1
libero-10
Joint
32.2
72.3
30.8
96.6
64.1
87.8
84.7
66.3
Ours
53.6
88.3
54.7
100.0
76.6
92.9
98.5
80.1
Appendix
Table 9: Per-suite per-perturbation success rate (%) on LIBERO-Plus (10,030 episodes). Ours uses K=4 ; Joint denotes the official-configuration Fast-WAM-Joint baseline from Table 1 . BG: background textures; Robot: robot initial states; Cam: camera viewpoints; Lang: language instructions; Noise: sensor noise; Obj: objects layout; Light: light conditions.
Figure 7: Component contributions on LIBERO-Plus at K=4 . The clean action prefix with dense visual history (P) and the streaming staircase (S) have complementary effects and combine to achieve the strongest result.
Warm-start steps
Successes
Total
Success (%)
0
7,443
10,030
74.2
1
7,934
10,030
79.1
4
7,919
10,030
79.0
8
7,927
10,030
79.0
10
7,960
10,030
79.4
Appendix
Table 11: Warm-start ablation on LIBERO-Plus at K=4 .
Figure 8: Real-robot model input: three camera views (exterior 1, exterior 2, wrist) vertically concatenated into a 384×224 frame ( H×W ).
Figure 9: Qualitative moving-target prediction before contact. Four future frames are replayed offline from recorded conveyor-task inputs and action chunks. FastWAM-Joint uses single-frame context; StreamingWAM uses five-frame dense history. In external camera 3, the predicted/recorded apple displacement is 1.38/6.93 pixels for FastWAM-Joint and 9.66/5.93 pixels for StreamingWAM; the cosine between predicted and recorded displacement directions is 0.497 and 0.995, respectively. The examples are stage-matched independent trials, so they support a qualitative anticipation interpretation but do not isolate history as the sole cause.
π0.5
FastWAM-Joint
Fast-WAM
StreamingWAM
Task
Async
RTC
Async
RTC
Async
Async
Static tasks
Block stacking
29
31
16
18
13
28
Open drawer
45
46
38
40
36
46
Weigh-and-place
23
25
18
19
15
42
Dynamic tasks
Appendix
Table 12: Real-robot per-task results. Number of successful trials for the five manipulation tasks (50 trials each); valid lit-button activations for whack-a-mole (50 × 30 s sessions).
Figure 10: Real-robot task execution under streaming single-step replanning. Each row shows seven keyframes from one successful episode.
World action models (WAMs) that use future visual prediction at inference time incur substantial generation costs. Asynchronous execution reduces waiting by overlapping inference with robot motion, but visual predictions used for subsequent action generation must anticipate the effects of actions already scheduled for execution during inference. We introduce Streaming-WAM, which couples action-conditioned world modeling with asynchronous robot control to account for committed actions in future visual prediction. At each streaming update, the model conditions future visual prediction on the latest observation and the committed actions, which form the fixed prefix of the next action chunk. The resulting action-conditioned visual features guide generation of the remaining actions within the same joint update, so the continuation is informed by the scene changes expected during execution of the fixed prefix. On LIBERO, Streaming-WAM achieves an average success rate of 98.35% and reduces mean episode time by a factor of 2.93 relative to Fast-WAM. On the real-world Stamp Paper task, mean episode time falls from 90 s with synchronous Joint-WAM to 38 s with Streaming-WAM. These results show that Streaming-WAM supports efficient asynchronous control while maintaining high task success rates.
World-Action Models (WAMs) improve robotic manipulation by conditioning action generation on predicted future observations, but future prediction adds further inference overhead to already expensive iterative action generation. Action chunking can amortize this cost over multiple actions, yet performance degrades over long execution horizons because later actions remain conditioned on stale observations. We introduce STAIRCASE POLICY, a streaming inference and training framework that turns a flow-matching VLA into a JEPA-style WAM and partitions a large action chunk into sub-chunks at staggered denoising stages. Near-term actions are executed as soon as they become available, while later actions continue to be refined. At each sub-chunk boundary, the future latent is re-predicted from the latest observation and used to update all unexecuted actions, enabling long-horizon execution without repeated full policy inference. The resulting future-prediction error can further serve as a signal for adaptive chunking. S-WAM achieves 97.7% on LIBERO and 87.9% on LIBERO-Plus, and improves performance across multiple policy backbones and real-robot tasks. It reaches 292.7 executed actions per second, 3.62× the throughput of conventional execution at comparable accuracy, while reducing time-to-first-action from 123.6 to 73.3 ms. With additional inference optimizations, throughput further increases to 642.9 actions per second.
Guoheng Sun, Chen Chen, Jin Wang +2
University of Maryland, College Park · Independent Researcher · Oxford Robotics Institute, University of Oxford
Cascade world-action models (WAMs) use predicted visual futures to condition robot actions, but inherit costly iterative inference from diffusion-pretrained experts. We ask what a single-step predicted future must preserve to support accurate single-step action prediction. In the evaluated WAM, resampling video noise has little effect on downstream actions, yet naively truncating both streams to one step substantially reduces task success. Video-space supervision only partially closes this gap, and a local sensitivity analysis shows that vision prediction errors have highly unequal consequences for action accuracy. These findings motivate DirectWAM, which converts pretrained video and action experts into single-step predictors without stochastic sampling through action-sufficient distillation, while retaining the explicit future-to-action cascade. After learning a one-step action model, we train the future predictor by decoding actions from its own predicted futures and backpropagating demonstration action error through the expert, with an optional future-latent reconstruction loss. The resulting objective evaluates the student-generated future through its downstream control consequences rather than relying on representation matching alone. DirectWAM achieves 97.1% average success across four LIBERO suites and 85.4% on RoboTwin 2.0, outperforming Flash-WAM at the same one-evaluation-per-stream budget. This advantage has also extended to three real-world tasks that we designed. Compared with the full iterative teacher, it reduces inference latency by 28.0×.