Organizations: College of Information Science and Electronic Engineering, Zhejiang University · Polytechnic Institute, Zhejiang University · Institute of Automation, Chinese Academy of Sciences · Zhongguancun Academy · School of Intelligent Robotics and Advanced Manufacturing Innovation, Fudan University · Zhongguancun Institute of Artificial Intelligence
World Action Models (WAMs) have emerged as an important paradigm in robot learning, jointly predicting future scene states and robot actions from current observations and instructions. However, future states entangle task-related changes with large amounts of task-irrelevant content, such as static backgrounds, which weakens the model's ability to capture changes relevant to task execution. To address this, we propose Difference World Action Model (DiffWAM), a Mixture-of-Experts (MoE) framework that enhances world representations by jointly learning future-state prediction and consecutive visual difference prediction. We define consecutive visual differences as differences between the latent representations of consecutive observations, providing an explicit learning signal for temporal variation. The framework comprises a World Expert for future frame prediction, an Action Expert for action generation, and a Diff Expert for predicting these latent differences along the trajectory. A structured unidirectional attention mask conditions the Diff, World, and Action Experts solely on the current-frame representation while blocking attention between experts. This design allows both the World and Diff Experts to be removed during inference, introducing no additional inference overhead. DiffWAM achieves state-of-the-art performance on the LIBERO and RoboTwin 2.0 benchmarks and substantially improves over FastWAM on real-world manipulation tasks. Specifically, it attains average success rates of 98.8%, 92.65%, and 88.3%, outperforming the state-of-the-art baseline (i.e., FastWAM) by 1.75, 1.55, and 15.0 percentage points, respectively.
Figures & tables
Figure 2: Framework of DiffWAM. The Observation Encoder provides current-observation representations to the World, Diff, and Action Experts. The World and Diff Experts receive the same noised future-frame latents, share their first N−M layers, and use independent final M layers for state generation through denoising and latent-difference prediction through regression, respectively. Structured unidirectional attention separates the expert pathways. At inference, both visual prediction experts are removed, retaining only the Observation Encoder and Action Expert without additional inference overhead.
Figure 3: Structured attention masks in DiffWAM. (a) Attention in the first N−M layers, where the World and Diff Experts share visual processing layers. (b) Attention in the last M layers, with independent state and diff prediction pathways. (c) Observation and action pathways retained at inference. Each pathway attends to observation representations and its own tokens, while cross-attention between expert pathways is blocked.
Figure 4: Training configurations and shared inference pathway of DiffWAM. DiffWAM-S freezes the independent LDB while retaining difference prediction during small-scale training. DiffWAM-L first trains the independent Diff Expert parameters with the remaining modules frozen, then jointly trains all modules. Orange denotes trainable modules, and blue with snowflake symbols denotes frozen modules; in DiffWAM-S, the shared World–Diff layers remain trainable. Both configurations retain only the Observation Encoder and Action Expert at inference.
Method
Spatial
Object
Goal
Long
Avg.
VLA
OpenVLA [ 5 ]
84.70
88.40
79.20
53.70
76.50
OpenVLA-O [ 6 ]
97.60
98.40
97.90
94.50
97.10
MemoryVLA [ 56 ]
98.40
98.40
96.40
93.40
96.65
π 0.5 [ 8 ]
98.80
98.20
98.00
92.40
96.85
GROOT-N1 [ 9 ]
94.40
97.60
93.00
90.60
93.90
Table 1: Main results on LIBERO. We use DiffWAM-S due to the small size of the LIBERO training dataset. † denotes our FastWAM baseline.
Method
Clean
Random
Avg.
VLA
π 0 [ 7 ]
65.92
58.80
62.36
X-VLA [ 59 ]
72.08
72.84
72.46
π 0.5 [ 8 ]
82.74
76.76
79.75
ABot-M0 [ 60 ]
86.60
85.08
85.84
QwenVLA [ 61 ]
86.10
87.20
86.65
Table 2: Main results on RoboTwin 2.0. We use DiffWAM-L due to the large size of the RoboTwin 2.0 training dataset. † denotes our FastWAM baseline.
Method
Embodied PT
Clean
Random
Avg.
With Embodied Pretraining
π0 [ 7 ]
Yes
46.4
16.3
31.35
X-VLA [ 59 ]
Yes
68.0
20.9
44.45
π0.5 [ 8 ]
Yes
70.7
46.0
58.35
Without Embodied Pretraining
StarVLA [ 62 ]
No
46.5
3.2
24.85
Table 3: Clean-to-random generalization on RoboTwin 2.0. Success rates (%) for policies trained on clean demonstrations and evaluated in clean and randomized environments without further adaptation. † denotes our FastWAM baseline.
Figure 5: Real-world setup, task execution sequences, and success rates. Left: the robot workspace and camera placement. Right: example executions of Place Pens, Store Block, and Stack Cups. Bottom: success rates of FastWAM and DiffWAM-S across the three tasks.
Setting
FastWAM
DiffWAM-L
DiffWAM-S
LIBERO
97.05
97.70
98.80
RoboTwin-Clean
88.47
87.72
93.13
RoboTwin-Full
91.88
93.28
92.60
Table 4: Success rates (%) of DiffWAM-L and DiffWAM-S. DiffWAM-L updates the LDB together with the remaining modules during its joint training stage; DiffWAM-S freezes the independent LDB parameters while retaining the diff loss. RoboTwin-Clean and RoboTwin-Full use clean-scene and full training data, respectively; both are evaluated in the clean setting.
Figure 6: Training performance comparison on LIBERO.
Architecture
Spatial
Object
Goal
Long
Avg.
World Expert
98.20
99.80
97.80
92.40
97.05
World Expert + Diff Loss
97.80
100.00
97.40
95.80
97.75
World Expert + Diff Expert
98.00
99.80
99.00
98.40
98.80
Table 5: Architecture ablation on LIBERO. World Expert uses the measured FastWAM results, World Expert + Diff Loss adds difference supervision without a separate Diff Expert, and World Expert + Diff Expert uses the DiffWAM-S results. The Observation Encoder and Action Expert are retained in all configurations.
Method
Spatial
Object
Goal
Long
Avg.
FastWAM [ 15 ]
98.20
99.80
97.80
92.40
97.05
DiffWAM ( M=5 )
99.60
100.00
97.20
96.00
98.20
DiffWAM ( M=10 )
98.00
99.80
99.00
98.40
98.80
DiffWAM ( M=15 )
98.20
99.80
97.40
97.00
98.10
DiffWAM ( M=20 )
97.00
100.00
98.40
98.00
98.35
Table 6: Ablation study of LDB layers M on LIBERO. All DiffWAM entries use the DiffWAM-S configuration.
Visual Experts
Spatial
Object
Goal
Long
Avg.
Only World Expert
98.20
99.80
97.80
92.40
97.05
Only Diff Expert
99.20
100.00
95.80
89.80
96.20
World + Diff Expert
98.00
99.80
99.00
98.40
98.80
Table 7: Ablation study of visual expert configurations on LIBERO.
Figure 7: Attention visualizations. Arrows indicate target regions specified by the language instruction.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Run 1
Run 2
Run 3
Mean ± Std.
RoboTwin 2.0 Full-data
92.49
92.65
92.43
92.52±0.11
LIBERO
98.60
98.80
98.60
98.67±0.12
RoboTwin 2.0 Clean-only
37.33
37.40
37.47
37.40±0.07
Appendix
Table 10: Independent training runs of DiffWAM. Success rates (%) for three independently trained and evaluated models; mean ± sample standard deviation ( n=3 , denominator n−1 ). RoboTwin scores average Clean and Random success; full-data and clean-only specify training settings. The clean-only average is not Random-only OOD success. LIBERO averages its four suites.
Figure 8: Supplementary per-suite training improvements of DiffWAM over the FastWAM baseline on LIBERO at each evaluation checkpoint.
Figure 9: Qualitative visualization of future-state and visual difference prediction by the World and Diff Experts on representative manipulation tasks.
World Action Models (WAMs) extend robot policy learning by incorporating future prediction as an additional training objective, encouraging the policy to encode task-relevant temporal structure in its representations. Current WAMs often rely on large-scale generative architectures that incur high training costs and inference latency, making them difficult to deploy as efficient closed-loop policies. We propose Light-WAM, a lightweight World Action Model for efficient robot manipulation. Specifically, it is built with a compact video backbone and performs future-video supervision in a downsampled latent space, reducing the cost of video co-training while retaining its benefits for representation learning. For action prediction, Light-WAM introduces the StateFusionActionExpert, which reads adapted states from multiple backbone layers, fuses them through learned-query pooling, and directly predicts action chunks in a single forward pass. This design provides an efficient interface between video backbone representations and robot actions, avoiding the need for heavy generative action experts. Experiments demonstrate that Light-WAM maintains strong performance on LIBERO and achieves usable multi-task performance on RoboTwin 2.0, while using only 0.44B trainable parameters. It also achieves 72.03ms inference latency with 4.1GiB peak GPU memory and improved training throughput.
Ziang Li, Dongzhou Cheng, Yibin Wang +5
Wuhan University · Shanghai Innovation Institute · Southeast University +2
Cascade world-action models (WAMs) use predicted visual futures to condition robot actions, but inherit costly iterative inference from diffusion-pretrained experts. We ask what a single-step predicted future must preserve to support accurate single-step action prediction. In the evaluated WAM, resampling video noise has little effect on downstream actions, yet naively truncating both streams to one step substantially reduces task success. Video-space supervision only partially closes this gap, and a local sensitivity analysis shows that vision prediction errors have highly unequal consequences for action accuracy. These findings motivate DirectWAM, which converts pretrained video and action experts into single-step predictors without stochastic sampling through action-sufficient distillation, while retaining the explicit future-to-action cascade. After learning a one-step action model, we train the future predictor by decoding actions from its own predicted futures and backpropagating demonstration action error through the expert, with an optional future-latent reconstruction loss. The resulting objective evaluates the student-generated future through its downstream control consequences rather than relying on representation matching alone. DirectWAM achieves 97.1% average success across four LIBERO suites and 85.4% on RoboTwin 2.0, outperforming Flash-WAM at the same one-evaluation-per-stream budget. This advantage has also extended to three real-world tasks that we designed. Compared with the full iterative teacher, it reduces inference latency by 28.0×.
World-Action Models (WAMs) offer visual foresight for robotic manipulation, but pixel-space models repeatedly reconstruct entire future scenes, incurring high computational cost and spatio-temporal redundancy. In physical manipulation, consecutive frames often share most of their visual context; the changes between them are what an action policy needs to anticipate. We introduce DeltaWAM, a change-centric WAM that makes a compact delta token the unit of future prediction. Each token is a single vector encoding changes between consecutive dense DINO feature maps. DeltaWAM builds on DeltaWorld, a latent world model pretrained on large-scale videos, to autoregressively predict one delta token per future frame. A flow-matching action expert then conditions on the predicted transitions and current DINO features, which serve as spatial anchors, to generate action chunks. Trained for 256 GPU hours on two H100 GPUs, DeltaWAM has 0.725B parameters and achieves 92.8% average success on LIBERO. It also shows robust generalization under procedural perturbations on LIBERO-Pro. Inference takes 142.1 ms per action chunk with 3.86 GB peak memory.
Tianyun Jiang, Wenrui Bao, Bingxin Xu +2
Fudan University · University of Central Florida · University of Southern California