StreamingWAM: Streaming World Action Modeling via Chunk-Wise Staircase Flow Matching

Shaojie Ren; Jingwen Sun; Zezhi Liu; Hongjin Chen; Xin Jin; Wei Zou; Zhibo Chen

cs.ROOct 11, 2026

StreamingWAM: Streaming World Action Modeling via Chunk-Wise Staircase Flow Matching

Authors: Shaojie Ren, Jingwen Sun, Zezhi Liu, Hongjin Chen, Xin Jin, Wei Zou, Zhibo Chen

Organizations: University of Chinese Academy of Sciences · Beijing Zhongguancun Academy · Institute of Automation, Chinese Academy of Sciences · University of Science and Technology of China ·

Abstract

World Action Models (WAMs), often built upon pretrained video-generation world models, have emerged as a promising framework for general-purpose and open-ended robot control. However, their real-world deployment often suffers from discontinuous and jerky execution. Existing methods typically address this issue by either improving inference efficiency or enhancing action continuity across prediction chunks, without jointly addressing both challenges. To bridge this gap, we propose StreamingWAM, a streaming-diffusion framework for WAM-based robot control. StreamingWAM combines a noise ladder for chunk-wise action prediction, a prefix rollout module for cross-chunk continuity, and dense video-history memory for temporally consistent conditioning. Experiments on LIBERO, LIBERO-Plus, RoboTwin 2.0, and real-world robotic tasks demonstrate that StreamingWAM improves action continuity across static and dynamic settings, achieving competitive performance while reducing steady-state replanning latency to 72.9 ms in simulation and 57.3 ms on the real robot. The method also produces smoother action transitions, while the streaming staircase reduces sensitivity to the execution horizon K.

Figures & tables

Figure 1: Overview of our streaming world-action model. (a) Standard policies regenerate a complete action chunk at every replan. (b) StreamingWAM rolls the action buffer forward with one denoising step per replan. (c) StreamingWAM performs strongly across simulation and real-world benchmarks.

Figure 2: Overview of StreamingWAM. (a) Joint video–action architecture with a clean action prefix and staircase denoising. (b) Standard multi-step inference versus single-step streaming inference.

With embodied pre-training
			Perturbation type
Method	EP	LIBERO	Cam.	Robot	Lang.	Light	Bkgnd.	Noise	Layout	Overall
$\pi$ 0.5	✓	96.9	75.4	77.5	85.6	96.9	94.6	89.7	85.7	85.7
GE-ACT	✓	96.5	60.7	77.0	77.4	95.8	86.0	90.9	80.2	80.3
VLA-JEPA	✓	97.2	63.3	67.1	85.4	95.6	93.6	66.3	85.1	79.5
X-VLA	✓	98.1	23.4	89.7	75.7	88.2	96.0	62.7	71.8	71.4

Table 1: Success rates (%) on LIBERO and LIBERO-Plus . The final column reports success pooled over all LIBERO-Plus evaluation instances. FastWAM-Joint follows its official evaluation configuration; detailed inference settings are provided in Table 6 . EP: embodied pre-training. Bold : best per column; underline : second best.

With embodied pre-training
Method	EP	Clean	Random	Avg.
Spatial Forcing	✓	77.2	9.5	43.4
$\pi$ 0.5	✓	70.7	46.0	58.4
X-WAM	✓	70.0	25.8	47.9
X-VLA	✓	68.0	20.9	44.5
EventVLA	✓	65.6	15.7	40.7

Table 2: RoboTwin 2.0 success rate (%). EP: embodied pre-training; bold/underline: best/second best.

Panel of Figure 4 — Figure 4: Inference efficiency and action continuity in simulation and on the real robot. Panel (c) is a system-level summary: for Fast-WAM and FastWAM-Joint, the latency coordinate is the standard 10-step latency, while overall success uses the task-specific real-robot deployment in which dynamic tasks use one denoising step.

Panel of Figure 5 — Figure 5: Real-robot tasks. (a)–(c) Static; (d)–(f) dynamic. Each panel shows one successful rollout.

Figure 6: Real-robot results. Each of the five manipulation tasks (left) reports the number of successful trials out of 50; whack-a-mole (right) reports total valid lit-button activations across 50

\times

30 s sessions.

Variant	$K{=}1$	$2$	$4$	$8$	$16$	Spread
Base ( Fast-WAM-Joint )	37.01	50.77	61.03	72.76	69.21	35.75
+ Temporal context (P)	39.14	59.36	65.83	73.71	72.49	34.57
+ Staircase (S)	57.13	60.01	61.39	65.60	53.65	11.95
+ P + S (full model)	75.45	77.22	79.36	78.73	72.86	6.50
w/ single-width $K{=}4$	71.52	74.08	75.31	73.74	70.39	4.92
w/ sparse history	56.35	67.80	69.86	68.45	60.43	13.51

Table 3: Ablation studies on the full 10,030-episode LIBERO-Plus evaluation (success rate, %). The top block compares the temporal-context package (P) and streaming staircase (S); the bottom block varies one design choice from the full model. Spread is max

-

min across the reported

K

values.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Layers	Hidden dim.	FFN dim.	Heads $\times$ dim.	Act./prop.	Cameras	Resolution ( $H\!\times\!W$ )
LIBERO / LIBERO-Plus	30/30	3,072/1,024	14,336/4,096	$24\!\times\!128$	7/8	2, horiz.	$224\!\times\!448$
RoboTwin 2.0	30/30	3,072/1,024	14,336/4,096	$24\!\times\!128$	14/14	3, top + 2 bottom	$384\!\times\!320$
Real robot	30/30	3,072/1,024	14,336/4,096	$24\!\times\!128$	8/8	3, vertical	$384\!\times\!224$

Appendix

Table 4: Core model configuration by benchmark. Video/Action values are shown as V/A.

Benchmark	Method	Shared Joint stage	Second stage	Total steps	Global batch
LIBERO / LIBERO-Plus	Fast-WAM-Joint	34,720	$+$ 40,000 Joint	74,720	$80{\rightarrow}64$
	StreamingWAM	34,720	$+$ 40,000 streaming	74,720	$80{\rightarrow}64$
RoboTwin 2.0	Fast-WAM-Joint	135,776	$+$ 135,776 Joint	271,552	$64{\rightarrow}64$
	StreamingWAM	135,776	$+$ 135,776 streaming	271,552	$64{\rightarrow}64$
Real robot	Fast-WAM-Joint	20,000	$+$ 30,000 Joint	50,000	$128{\rightarrow}128$
	StreamingWAM	20,000	$+$ 30,000 streaming	50,000	$128{\rightarrow}128$

Appendix

Table 5: Training schedule by benchmark.

	LIBERO / LIBERO-Plus			RoboTwin 2.0			Real robot
Method	Steps	$H$	$K$	Steps	$H$	$K$	Steps	$H$	$K$
$\pi_{0.5}$ Async	10	10	5	10	50	50	10	16	4
$\pi_{0.5}$ RTC	10	10	5	10	50	50	10	16	4
Fast-WAM	10	32	10	10	32	24	$10/1$	32	4
Faster-WAM	10	32	4	10	32	4	10	32	4
Fast-WAM-Joint Async	10	32	10	10	32	24	$10/1$	32	4

Appendix

Table 6: Inference hyperparameters used in our evaluations.

H

is the predicted future-action horizon (the 16 clean prefix actions of StreamingWAM are excluded), and

K

is the number of actions executed per replan.

10\!\rightarrow\!1

denotes a one-time 10-step warm start followed by one rolling SDP update per replan. For real-robot non-streaming rows,

10/1

means static/dynamic tasks.

Method	$H$	Steps	$n$	Latency (ms)	LIBERO-Plus (%)
Fast-WAM-Joint	32	10	30	381.6 $\pm$ 6.9	68.8
Fast-WAM	32	10	30	182.6 $\pm$ 3.2	51.5
Faster-WAM	32	10	30	139.4 $\pm$ 1.7	75.0
Ours (steady)	16	1	300	72.9 $\pm$ 0.3	79.4
Ours (initial warm start)	16	10	30	406.2 $\pm$ 0.8	—
Real-robot deployment (same RTX 5090)

Appendix

Table 7: Inference latency comparison. All measurements on the same RTX 5090 with CUDA synchronous timing.

n

: number of timed forward passes.

Method	SR (%)	Trans. jump $\downarrow$	Rot. jump $\downarrow$	Bnd./int. ratio $\downarrow$	Jerk $\downarrow$
Fast-WAM	60.2	0.1350	0.02903	1.831	5.775
Fast-WAM-Joint	68.8	0.1379	0.02681	1.840	5.710
Ours	79.0	0.0684	0.00879	1.105	2.341
Ours vs. Fast-WAM-Joint		$-$ 50.4%	$-$ 67.2%	—	$-$ 59.0%

Appendix

Table 8: Action-continuity metrics on LIBERO-Plus (500-episode subset, 1-step inference,

K{=}4

\downarrow

: lower is better.

Suite		BG	Robot	Cam	Lang	Noise	Obj	Light	All
object	Joint	64.1	57.8	49.2	97.7	68.0	77.4	97.3	72.2
object	Ours	83.9	82.2	83.6	97.7	92.2	83.9	98.0	88.6
spatial	Joint	74.0	66.3	55.6	93.3	72.1	90.4	99.0	78.5
spatial	Ours	73.6	83.7	64.4	97.4	87.5	96.4	97.9	86.1
libero-10	Joint	32.2	72.3	30.8	96.6	64.1	87.8	84.7	66.3
libero-10	Ours	53.6	88.3	54.7	100.0	76.6	92.9	98.5	80.1

Appendix

Table 9: Per-suite per-perturbation success rate (%) on LIBERO-Plus (10,030 episodes). Ours uses

K{=}4

; Joint denotes the official-configuration Fast-WAM-Joint baseline from Table 1 . BG: background textures; Robot: robot initial states; Cam: camera viewpoints; Lang: language instructions; Noise: sensor noise; Obj: objects layout; Light: light conditions.

Figure 7: Component contributions on LIBERO-Plus at

K=4

. The clean action prefix with dense visual history (P) and the streaming staircase (S) have complementary effects and combine to achieve the strongest result.

Warm-start steps	Successes	Total	Success (%)
0	7,443	10,030	74.2
1	7,934	10,030	79.1
4	7,919	10,030	79.0
8	7,927	10,030	79.0
10	7,960	10,030	79.4

Appendix

Table 11: Warm-start ablation on LIBERO-Plus at

K{=}4

Figure 8: Real-robot model input: three camera views (exterior 1, exterior 2, wrist) vertically concatenated into a $384{\times}224$ frame ( $H{\times}W$ ).

Figure 9: Qualitative moving-target prediction before contact. Four future frames are replayed offline from recorded conveyor-task inputs and action chunks. FastWAM-Joint uses single-frame context; StreamingWAM uses five-frame dense history. In external camera 3, the predicted/recorded apple displacement is 1.38/6.93 pixels for FastWAM-Joint and 9.66/5.93 pixels for StreamingWAM; the cosine between predicted and recorded displacement directions is 0.497 and 0.995, respectively. The examples are stage-matched independent trials, so they support a qualitative anticipation interpretation but do not isolate history as the sole cause.

Static tasks
	$\pi_{0.5}$		FastWAM-Joint		Fast-WAM	StreamingWAM
Task	Async	RTC	Async	RTC	Async	Async
Block stacking	29	31	16	18	13	28
Open drawer	45	46	38	40	36	46
Weigh-and-place	23	25	18	19	15	42
Dynamic tasks

Appendix

Table 12: Real-robot per-task results. Number of successful trials for the five manipulation tasks (50 trials each); valid lit-button activations for whack-a-mole (50

\times

30 s sessions).

Panel of Figure 10 — Figure 10: Real-robot task execution under streaming single-step replanning. Each row shows seven keyframes from one successful episode.

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 584 papers

Streaming-WAM: Action-Conditioned World-Action Model for Asynchronous Robot Manipulation
Sep 24, 2026Xuyao Huang, Yixuan Wang, Zengyao Ye +4Robotic Control World Models for Robotics
SJTU · SDU · ECUST +1
Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunks
Sep 29, 2026Guoheng Sun, Chen Chen, Jin Wang +2Long-Horizon Robotic Manipulation Action-Conditioned World Models
University of Maryland, College Park · Independent Researcher · Oxford Robotics Institute, University of Oxford
De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control
Oct 20, 2026Shaoyi Zheng, Zhongtao Jiang, Yue Peng +4Efficient Inference for World Action Models World Models for VLA Models
ByteDance. · New York University (NYU).

Sep 24, 2026J/K move · Enter open · S save

Streaming-WAM: Action-Conditioned World-Action Model for Asynchronous Robot Manipulation

Xuyao Huang, Yixuan Wang, Zengyao Ye, Boyuan Zhao+3

SJTU · SDU · ECUST+1

Robotic Control World Models for Robotics World Model-Based Planning Action-Conditioned World Models Efficient Inference for World Action Models Robotic Manipulation

Open paper AskPDF arXiv

World action models (WAMs) that use future visual prediction at inference time incur substantial generation costs. Asynchronous execution reduces waiting by overlapping inference with robot motion, but visual predictions used for subsequent action generation must anticipate the effects of actions already scheduled for execution during inference. We introduce Streaming-WAM, which couples action-conditioned world modeling with asynchronous robot control to account for committed actions in future visual prediction. At each streaming update, the model conditions future visual prediction on the latest observation and the committed actions, which form the fixed prefix of the next action chunk. The resulting action-conditioned visual features guide generation of the remaining actions within the same joint update, so the continuation is informed by the scene changes expected during execution of the fixed prefix. On LIBERO, Streaming-WAM achieves an average success rate of 98.35% and reduces mean episode time by a factor of 2.93 relative to Fast-WAM. On the real-world Stamp Paper task, mean episode time falls from 90 s with synchronous Joint-WAM to 38 s with Streaming-WAM. These results show that Streaming-WAM supports efficient asynchronous control while maintaining high task success rates.

Abstract

Robotic Control

Explore similar work

Streaming-WAM: Action-Conditioned World-Action Model for Asynchronous Robot Manipulation

Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunks

De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control