Organizations: 1Sun Yat-sen University · 2Shenzhen Loop Area Institute · 3The Hong Kong University of Science and Technology (Guangzhou) · 4Xi’an Jiaotong University · 5Southern University of Science and Technology · 6Beijing Institute of Technology · 7Peng Cheng Laboratory · 8The Chinese University of Hong Kong Shenzhen · 9Ainoro · 10King Abdullah University of Science and Technology
World-Action Models (WAMs) have emerged as a promising paradigm for embodied generalist policies by jointly modeling future world evolution and robot actions across diverse manipulation tasks. However, representing diverse futures is only the first step: a capable WAM must also generate futures that match the requirements of the current task and execution progress. In long-horizon manipulation where similar observation-action segments occur across different tasks and stages, existing approaches may generate locally plausible yet misdirected rollouts to drift toward wrong task stage, i.e., a phenomenon we term Stage Confusion. To address this problem, we propose TaskWAM, a closed-loop, stage-aware framework that represents fine-grained task requirements with Mode Tokens and tracks execution progress with State Tokens, forming a structured task state to steer world--action generation toward intended future. We further introduce StageBench, a controlled multi-stage manipulation benchmark with parameterized stage variations and automated evaluation.Extensive experiments on both simulation benchmarks and real-world tasks demonstrate that TaskWAM effectively reduces confusion and achieves competitive performance with state-of-the-art methods. Moreover, TaskWAM provides strong plug-and-play transfer across heterogeneous WAM architectures and robot embodiments, including single-arm robots, dual-arm grippers, dexterous hands, and humanoid robots.
Figures & tables
Figure 1: From plausible futures to reliable stage progression. Top (left): Overlapping task trajectories can induce Stage Confusion: WAMs may follow plausible yet incorrect execution routes. Top (right): StageBench systematically evaluates this challenge through controlled overlapping multi-stage trajectories across eight task families and more than 2.2 million demonstration frames. Bottom: Visualization of TaskWAM performing reliable long-horizon manipulation across diverse robot embodiments and stage progressions. Visually similar stages are highlighted with matching color borders to indicate potential stage confusion.
Figure 2: Overall framework. TaskWAM augments a World-Action DiT through explicit task-state grounding and conditioning. (a) Mode Tokens retrieve mode representations from a learnable codebook and ground them with text context to encode fine-grained task requirements. (b) State Tokens aggregate action-observation interactions into a causal interaction history that tracks execution progress. (c) Task-state-conditioned steering conditions the world and action streams through attention and modulation, while residual flow guidance steers generation toward intended futures.
Figure 3: The visualization of task families in StageBench (see Appendix B.1 for more details).
Method
Kitchen Manipulation
Bimanual Manipulation
Avg. Suc.
Cooking & Cheese
Cooking & Juice
Cooking & Cleanup
Cooking & Breakfast
Size-Order Cubes
Color-Order Cups
Arm-Assigned Hammering
Repeated Bread Cutting
GR00T N1.7
43
33
9
28
60
28
55
43
37.4
π0.5
52
54
43
40
69
46
75
57
54.5
Motus
27
28
27
12
15
10
47
20
23.3
Cosmos Policy
35
53
35
55
18
7
45
13
32.6
w/ TaskWAM
57
61
45
63
29
22
56
31
45.5
Table 1: Performance comparison on diverse long-horizon manipulation tasks of StageBench. Our method outperforms all baselines in Success Rate ( Suc. , % ↑ ). Best results are in bold .
Task
Motus
π0.5
LingBot-VA
LaWAM
AHA-WAM
Fast-WAM
w/ TaskWAM
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean ( Δ )
Rand.( Δ )
Open Laptop
95
91
90
96
92
94
100
100
98
97
98
100
100
100
Place Bread Basket
91
94
77
64
97
95
92
85
89
94
91
93
98
97
Place Bread Skillet
86
83
85
66
95
90
90
83
91
87
90
93
94
98
Place Object Scale
88
85
86
80
96
95
95
88
96
87
90
97
96
99
⋯
⋯
Table 2: Performance comparison on RoboTwin 2.0 Benchmark (Clean vs. Randomized, 50 tasks). We show the representative tasks and full comparison results is in Appendix B.2 .
Figure 4: Stage-wise completion over long-horizon execution. TaskWAM maintains higher completion across all stage transitions.
Figure 5: Real-world robot platform. (a) Franka Research 3 with third-person and wrist-view RealSense D435 cameras; (b-c) dual-arm AgileX Piper (6-DoF) and Nero (7-DoF) equipped with two wrist cameras and one head-mounted camera, respectively; (c) Unitree G1 humanoid robot equipped with BrainCo Revo2 dexterous hands and operated via PICO VR for whole-body teleoperation.
Task
Setting
Cosmos Policy
FastWAM
TaskWAM
Δ
S1→S2→S3
S1→S2→S3
S1→S2→S3
Sequential Cup Nesting
Single-Arm Franka
45.0 → 40.0 → 30.0
55.0 → 45.0 → 35.0
70.0 → 65.0 → 55.0
+20.0
Bread Cutting
Dexterous Hands
95.0 → 80.0 → 20.0
100.0 → 80.0 → 35.0
100.0 → 95.0 → 70.0
+35.0
Lab. Fluid Manipulation
Bi-Gripper Nero
65.0 → 40.0 → 40.0
80.0 → 55.0 → 50.0
90.0 → 70.0 → 70.0
+20.0
Bread Toasting & Spreading
Bi-Gripper Piper
S1→S2→S3→S4→S5
S1→S2→S3→S4→S5
S1→S2→S3→S4→S5
90.0 → 20.0 → 20.0 → 0.0 → 0.0
95.0 → 25.0 → 25.0 → 10.0 → 5.0
95.0 → 45.0 → 45.0 → 30.0 → 20.0
+15.0
Table 5: Results of real-world experiments. We report the stage-wise success rate for each task, Sk denotes cumulative success through stage k . Best results are bolded .
Method
Suc. ↑
SPR. ↑
Latency (s) ↓
FastWAM
40.9
68.4
0.190
TaskWAM
69.4
87.1
0.209
Table 6: Performance-efficiency trade-off. TaskWAM achieves significant gains in multi-stage reliability with minimal inference overhead.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Setting
Backbone
Wan2.2-TI2V-5B + FastWAM Action DiT
Training GPUs
16× NVIDIA H100 80GB
Effective batch size
128
Optimizer updates
58,720
Optimizer
AdamW
Learning rate
1×10−4
Appendix
Table 7: Training configuration on kitchen manipulation tasks of StageBench.
ID
Task family
Execution requirements
Kitchen Manipulation
K1
Cooking & Cheese
Interleaved; Sequential
K2
Cooking & Juice
Interleaved; Sequential
K3
Cooking & Cleanup
Interleaved; Sequential
K4
Cooking & Breakfast
Interleaved; Sequential
Bimanual Manipulation
Appendix
Table 8: Task families and execution requirements in StageBench.
Figure 6: StageBench benchmark visualization. Representative task variants and their multi-stage execution processes are illustrated, highlighting diverse stage structures and instruction- dependent task requirements.
Figure 7: Execution orders in a LIBERO household task. Both variants share the mug-placement and heating prefix. The Interleaved route collects the bowl and butter during heating and then reopens the microwave. The Sequential route waits for heating to finish and reopens the microwave before collecting the objects.
Figure 8: Instruction-dependent cup placement in RoboTwin. From a shared initial state, the robot places the red (R), green (G), and blue (B) cups in four prescribed orders: RGB, RBG, BGR, and GRB. Stage labels identify cup-specific operations, whose execution order varies with the instruction.
Figure 9: Visualization of Sequential Cup Nesting.
Figure 10: Visualization of Bread Toasting & Spreading.
Figure 11: Visualization of Laboratory Fluid Manipulation.
Figure 13: Training convergence. TaskWAM converges faster and reaches a lower action loss than FastWAM under the same training setup.
Figure 14: Real-world stage confusion in Bread Toasting & Spreading. Stage 1 (loading bread) and Stage 4 (retrieving toasted bread) exhibit similar visual configurations, while Stage 3 introduces a long waiting period with redundant observations. Baseline WAMs may confuse the current progress with earlier stages, resulting in repeated or incorrect execution.
Generalist robot policies aim to map multimodal observations and linguistic task instructions to actions across diverse tasks. However, existing methods typically represent the future as a fixed, short video-action chunk. This short-term future captures local scene evolution for action execution, but it does not explicitly describe the stage-level future that specifies how a task should progress from its current stage to the next. We therefore distinguish two complementary futures for robot manipulation: a short-term physical future to capture local scene evolution and a stage-level semantic future to represent task progress. We introduce StageWAM, which augments a Motus-based World Action Model (WAM) with Stage-JEPA, a goal-conditioned Joint-Embedding Predictive Architecture (JEPA) predictor. Given the current observation and task instruction, Stage-JEPA uses a frozen V-JEPA2 encoder to extract the current-state representation and predicts the latent target of the next inferred stage. Across 50 RoboTwin 2.0 tasks in clean and randomized environments, StageWAM achieves 90.25% overall success and reduces the mean number of execution steps in successful rollouts by 5.97% relative to the strongest baseline.
Xiao Liu, Yuguang Yang, Xi Wang +6
Institute for AI Industry Research (AIR), Tsinghua University · School of Electronic Information Engineering, Beihang University · AIR Wuxi Innovation Center, Tsinghua University +3
Cross-task generalization remains difficult for world action models (WAMs), even when their predicted futures depict plausible manipulation. This gap can be attributed directly: we make this gap directly testable by decoupling a WAM into a world model and an inverse-dynamics model (IDM), then holding the IDM fixed while replacing the source of future images. We introduce Align Once, Evaluate Everywhere (AoE), a camera-space inverse-dynamics model pretrained on large-scale robot data, which provides this recovery interface for different future sources across tasks and environments. Through it, we find that actions recovered from recorded futures of unseen tasks execute almost as well as replayed demonstrations, whereas futures predicted by world models collapse across benchmarks. The failures are not generic: they concentrate on the gripper and interaction geometry while generation capacity is spent on task-irrelevant background. This diagnosis suggests that supervision should target exactly these regions. We therefore introduce the vision-to-action (VITA) loss, which passes generated futures through the frozen AoE and penalizes discrepancies in the recovered actions, redirecting generation toward action-relevant detail. Across two world models and three manipulation benchmarks, this simple change raises the execution success of generated futures from near zero to 47.0%. The same loss also transfers to WAMs and yields consistent gains in native control.
Chen Zhao, Zipeng Ye, Yihan Lin +2
Renmin University of China, Beijing 100872, China · Key Laboratory of Data Engineering and KnowledgeEngineering, MOE, China · Meituan, Beijing 100102, China
World Action Models (WAMs) jointly predict future visual observations and actions, but typically execute a fixed number of actions before replanning, regardless of whether the imagined future remains consistent with reality. We formulate adaptive WAM execution as future--reality verification and propose Future Forward Dynamics Causal Attention (FFDC), a lightweight verifier that reasons over predicted actions, predicted visual dynamics, real observations, and instruction-conditioned semantics. FFDC allows continued execution when a plan remains reliable and triggers replanning when its validity deteriorates. We combine verification with Mixture-of-Horizon Training to improve long-horizon trajectory coverage. On RoboTwin, FFDC-WAM achieves the highest average success rate among the evaluated methods in both clean and randomized settings. Under randomization, it improves success by 2.54 percentage points over Base-Motus while reducing WAM planning calls by 69.10% and completion time by 33.88%. Notably, across six real-world tasks on a Unitree G1D robot, FFDC-WAM improves average success by 9.17% over Base-Motus-LC16 while reducing calls by 74.06% and completion time by 51.31%.
Rui Wang, Yue Zhang, Canyang Chen +3
Southern University of Science and Technology, Shenzhen, China · The University of Hong Kong, Hong Kong, China