Organizations: College of Information Science and Electronic Engineering, Zhejiang University · Polytechnic Institute, Zhejiang University · Institute of Automation, Chinese Academy of Sciences · Zhongguancun Academy · School of Intelligent Robotics and Advanced Manufacturing Innovation, Fudan University · Zhongguancun Institute of Artificial Intelligence
World Action Models (WAMs) have emerged as an important paradigm in robot learning, jointly predicting future scene states and robot actions from current observations and instructions. However, future states entangle task-related changes with large amounts of task-irrelevant content, such as static backgrounds, which weakens the model's ability to capture changes relevant to task execution. To address this, we propose Difference World Action Model (DiffWAM), a Mixture-of-Experts (MoE) framework that enhances world representations by jointly learning future-state prediction and consecutive visual difference prediction. We define consecutive visual differences as differences between the latent representations of consecutive observations, providing an explicit learning signal for temporal variation. The framework comprises a World Expert for future frame prediction, an Action Expert for action generation, and a Diff Expert for predicting these latent differences along the trajectory. A structured unidirectional attention mask conditions the Diff, World, and Action Experts solely on the current-frame representation while blocking attention between experts. This design allows both the World and Diff Experts to be removed during inference, introducing no additional inference overhead. DiffWAM achieves state-of-the-art performance on the LIBERO and RoboTwin 2.0 benchmarks and substantially improves over FastWAM on real-world manipulation tasks. Specifically, it attains average success rates of 98.8%, 92.65%, and 88.3%, outperforming the state-of-the-art baseline (i.e., FastWAM) by 1.75, 1.55, and 15.0 percentage points, respectively.
Figures & tables
Figure 2: Framework of DiffWAM. The Observation Encoder provides current-observation representations to the World, Diff, and Action Experts. The World and Diff Experts receive the same noised future-frame latents, share their first N−M layers, and use independent final M layers for state generation through denoising and latent-difference prediction through regression, respectively. Structured unidirectional attention separates the expert pathways. At inference, both visual prediction experts are removed, retaining only the Observation Encoder and Action Expert without additional inference overhead.
Figure 3: Structured attention masks in DiffWAM. (a) Attention in the first N−M layers, where the World and Diff Experts share visual processing layers. (b) Attention in the last M layers, with independent state and diff prediction pathways. (c) Observation and action pathways retained at inference. Each pathway attends to observation representations and its own tokens, while cross-attention between expert pathways is blocked.
Figure 4: Training configurations and shared inference pathway of DiffWAM. DiffWAM-S freezes the independent LDB while retaining difference prediction during small-scale training. DiffWAM-L first trains the independent Diff Expert parameters with the remaining modules frozen, then jointly trains all modules. Orange denotes trainable modules, and blue with snowflake symbols denotes frozen modules; in DiffWAM-S, the shared World–Diff layers remain trainable. Both configurations retain only the Observation Encoder and Action Expert at inference.
Method
Spatial
Object
Goal
Long
Avg.
VLA
OpenVLA [ 5 ]
84.70
88.40
79.20
53.70
76.50
OpenVLA-O [ 6 ]
97.60
98.40
97.90
94.50
97.10
MemoryVLA [ 56 ]
98.40
98.40
96.40
93.40
96.65
π 0.5 [ 8 ]
98.80
98.20
98.00
92.40
96.85
GROOT-N1 [ 9 ]
94.40
97.60
93.00
90.60
93.90
Table 1: Main results on LIBERO. We use DiffWAM-S due to the small size of the LIBERO training dataset. † denotes our FastWAM baseline.
Method
Clean
Random
Avg.
VLA
π 0 [ 7 ]
65.92
58.80
62.36
X-VLA [ 59 ]
72.08
72.84
72.46
π 0.5 [ 8 ]
82.74
76.76
79.75
ABot-M0 [ 60 ]
86.60
85.08
85.84
QwenVLA [ 61 ]
86.10
87.20
86.65
Table 2: Main results on RoboTwin 2.0. We use DiffWAM-L due to the large size of the RoboTwin 2.0 training dataset. † denotes our FastWAM baseline.
Method
Embodied PT
Clean
Random
Avg.
With Embodied Pretraining
π0 [ 7 ]
Yes
46.4
16.3
31.35
X-VLA [ 59 ]
Yes
68.0
20.9
44.45
π0.5 [ 8 ]
Yes
70.7
46.0
58.35
Without Embodied Pretraining
StarVLA [ 62 ]
No
46.5
3.2
24.85
Table 3: Clean-to-random generalization on RoboTwin 2.0. Success rates (%) for policies trained on clean demonstrations and evaluated in clean and randomized environments without further adaptation. † denotes our FastWAM baseline.
Figure 5: Real-world setup, task execution sequences, and success rates. Left: the robot workspace and camera placement. Right: example executions of Place Pens, Store Block, and Stack Cups. Bottom: success rates of FastWAM and DiffWAM-S across the three tasks.
Setting
FastWAM
DiffWAM-L
DiffWAM-S
LIBERO
97.05
97.70
98.80
RoboTwin-Clean
88.47
87.72
93.13
RoboTwin-Full
91.88
93.28
92.60
Table 4: Success rates (%) of DiffWAM-L and DiffWAM-S. DiffWAM-L updates the LDB together with the remaining modules during its joint training stage; DiffWAM-S freezes the independent LDB parameters while retaining the diff loss. RoboTwin-Clean and RoboTwin-Full use clean-scene and full training data, respectively; both are evaluated in the clean setting.
Figure 6: Training performance comparison on LIBERO.
Architecture
Spatial
Object
Goal
Long
Avg.
World Expert
98.20
99.80
97.80
92.40
97.05
World Expert + Diff Loss
97.80
100.00
97.40
95.80
97.75
World Expert + Diff Expert
98.00
99.80
99.00
98.40
98.80
Table 5: Architecture ablation on LIBERO. World Expert uses the measured FastWAM results, World Expert + Diff Loss adds difference supervision without a separate Diff Expert, and World Expert + Diff Expert uses the DiffWAM-S results. The Observation Encoder and Action Expert are retained in all configurations.
Method
Spatial
Object
Goal
Long
Avg.
FastWAM [ 15 ]
98.20
99.80
97.80
92.40
97.05
DiffWAM ( M=5 )
99.60
100.00
97.20
96.00
98.20
DiffWAM ( M=10 )
98.00
99.80
99.00
98.40
98.80
DiffWAM ( M=15 )
98.20
99.80
97.40
97.00
98.10
DiffWAM ( M=20 )
97.00
100.00
98.40
98.00
98.35
Table 6: Ablation study of LDB layers M on LIBERO. All DiffWAM entries use the DiffWAM-S configuration.
Visual Experts
Spatial
Object
Goal
Long
Avg.
Only World Expert
98.20
99.80
97.80
92.40
97.05
Only Diff Expert
99.20
100.00
95.80
89.80
96.20
World + Diff Expert
98.00
99.80
99.00
98.40
98.80
Table 7: Ablation study of visual expert configurations on LIBERO.
Figure 7: Attention visualizations. Arrows indicate target regions specified by the language instruction.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Run 1
Run 2
Run 3
Mean ± Std.
RoboTwin 2.0 Full-data
92.49
92.65
92.43
92.52±0.11
LIBERO
98.60
98.80
98.60
98.67±0.12
RoboTwin 2.0 Clean-only
37.33
37.40
37.47
37.40±0.07
Appendix
Table 10: Independent training runs of DiffWAM. Success rates (%) for three independently trained and evaluated models; mean ± sample standard deviation ( n=3 , denominator n−1 ). RoboTwin scores average Clean and Random success; full-data and clean-only specify training settings. The clean-only average is not Random-only OOD success. LIBERO averages its four suites.
Figure 8: Supplementary per-suite training improvements of DiffWAM over the FastWAM baseline on LIBERO at each evaluation checkpoint.
Figure 9: Qualitative visualization of future-state and visual difference prediction by the World and Diff Experts on representative manipulation tasks.