cs.RONov 15, 2026

DiffWAM: Learning World Action Model from Consecutive Visual Difference

Authors: Xinheng Zhang, Haoyu Xiong, Chao Jing, Xiaohan Zhang, Linshan Li, Hongbang Fan, Xingyu Chen

Organizations: College of Information Science and Electronic Engineering, Zhejiang University · Polytechnic Institute, Zhejiang University · Institute of Automation, Chinese Academy of Sciences · Zhongguancun Academy · 

Abstract

World Action Models (WAMs) have emerged as an important paradigm in robot learning, jointly predicting future scene states and robot actions from current observations and instructions. However, future states entangle task-related changes with large amounts of task-irrelevant content, such as static backgrounds, which weakens the model's ability to capture changes relevant to task execution. To address this, we propose Difference World Action Model (DiffWAM), a Mixture-of-Experts (MoE) framework that enhances world representations by jointly learning future-state prediction and consecutive visual difference prediction. We define consecutive visual differences as differences between the latent representations of consecutive observations, providing an explicit learning signal for temporal variation. The framework comprises a World Expert for future frame prediction, an Action Expert for action generation, and a Diff Expert for predicting these latent differences along the trajectory. A structured unidirectional attention mask conditions the Diff, World, and Action Experts solely on the current-frame representation while blocking attention between experts. This design allows both the World and Diff Experts to be removed during inference, introducing no additional inference overhead. DiffWAM achieves state-of-the-art performance on the LIBERO and RoboTwin 2.0 benchmarks and substantially improves over FastWAM on real-world manipulation tasks. Specifically, it attains average success rates of 98.8%, 92.65%, and 88.3%, outperforming the state-of-the-art baseline (i.e., FastWAM) by 1.75, 1.55, and 15.0 percentage points, respectively.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Light-WAM: Efficient World Action Models with State-Fusion Action Decoding

    Jun 6, 2026Ziang Li, Dongzhou Cheng, Yibin Wang +5Robot Policy LearningWorld Models for Robotics

  2. De-Diffusing WAMs: Action-Sufficient Direct Prediction for Robot Control

    Oct 20, 2026Shaoyi Zheng, Zhongtao Jiang, Yue Peng +4Efficient Inference for World Action ModelsWorld Models for VLAs

  3. DeltaWAM: Change-Centric Visual Foresight via Delta Tokens for an Efficient World-Action Model

    Sep 27, 2026Tianyun Jiang, Wenrui Bao, Bingxin Xu +2World Model LearningRobotic Manipulation