cs.ROSep 27, 2026

DeltaWAM: Change-Centric Visual Foresight via Delta Tokens for an Efficient World-Action Model

Authors: Tianyun Jiang, Wenrui Bao, Bingxin Xu, Yu Tian, Yuzhang Shang

Organizations: Fudan University · University of Central Florida · University of Southern California

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

World-Action Models (WAMs) offer visual foresight for robotic manipulation, but pixel-space models repeatedly reconstruct entire future scenes, incurring high computational cost and spatio-temporal redundancy. In physical manipulation, consecutive frames often share most of their visual context; the changes between them are what an action policy needs to anticipate. We introduce DeltaWAM, a change-centric WAM that makes a compact delta token the unit of future prediction. Each token is a single vector encoding changes between consecutive dense DINO feature maps. DeltaWAM builds on DeltaWorld, a latent world model pretrained on large-scale videos, to autoregressively predict one delta token per future frame. A flow-matching action expert then conditions on the predicted transitions and current DINO features, which serve as spatial anchors, to generate action chunks. Trained for 256 GPU hours on two H100 GPUs, DeltaWAM has 0.725B parameters and achieves 92.8% average success on LIBERO. It also shows robust generalization under procedural perturbations on LIBERO-Pro. Inference takes 142.1 ms per action chunk with 3.86 GB peak memory.

Figures & tables

Figure 1: DeltaWAM overview. (a) Existing policies represent an imagined future as a single subgoal or a sequence of dense frames or latent states, whereas DeltaWAM predicts one compact transition token per future step. (b) DeltaWAM couples a latent transition-centric world model DeltaWorld with an Action DiT, preserving efficient test-time visual foresight. (c) DeltaWAM is lightweight and has low latency compared with representative World-Action models.
Figure 2: Detailed DeltaWAM architecture. DeltaWAM couples the change-centric DeltaWorld with a flow-matching Action DiT. The pretrained DeltaWorld is first finetuned on demonstrations; then Action DiT is trained on robot data with the finetuned DeltaWorld frozen. DeltaWorld first rolls out predicted delta tokens autoregressively. Afterwards, both current DINO latents (the spatial anchor) and predicted delta tokens (the dynamic subgoal) are fed to Action DiT.
Figure 3: Language steers predicted futures toward the task goal. Wrist-view rollouts for LIBERO-Spatial task 9, which asks the robot to pick up the black bowl on the wooden cabinet and place it on the plate. In DeltaWorld’s unconditional predictions, the robot approaches the stove instead of the cabinet. This is misleading since the bowl is on the cabinet.
Figure 4: DeltaWAM training and inference. During the second training stage, Action DiT learns from oracle-selected or randomly sampled future trajectories produced by frozen DeltaWorld. At inference, one sampled future trajectory conditions action generation.
MethodSizeEmb. PTLatency (ms)SpatialObjectGoalLongAverage
Reactive Policies
Diffusion Policy [ 11 ]263M×\times–78.392.568.350.572.4
OpenVLA [ 27 ]7B✓\checkmark–84.788.479.253.776.5
π0\pi_{0} [ 6 ]3.5B✓\checkmark22098.096.894.488.494.4
π0.5\pi_{0.5} [ 5 ]3.5B✓\checkmark22098.898.298.092.496.9
GR00T-N1.6 [ 37 ]3.3B✓\checkmark25997.798.597.594.497.0
Table 1: Standard LIBERO success rate (%), model size, and model-only latency per action-chunk query. Emb. PT denotes embodied action pretraining. Baseline sizes and latency values follow their source papers and the comparison protocol of LaWAM [ 10 ] .
MethodSpatialObjectGoalLongAverage
OpenVLA [ 27 ]0.00.00.00.00.0
π0\pi_{0} [ 6 ]0.00.00.00.00.0
π0.5\pi_{0.5} [ 5 ]1.01.00.01.00.75
MolmoAct [ 28 ]0.00.00.06.01.5
NORA [ 24 ]0.00.00.00.00.0
x-VLA [ 55 ]0.08.09.010.06.75
Table 2: Success rate (%) under the LIBERO-Pro task perturbation, broken down by the four standard task suites. Official benchmark baselines follow Zhou et al. [57] . We evaluated the results for Fast-WAM and Lingbot-VA ourselves.
Figure 5: Delta-token structure under LIBERO-Pro task variations. The two t-SNE projections compare predicted delta tokens from a perturbed task (green triangles) with tokens from its nominal original task (blue) and an original task whose task requirements match the perturbation (orange).
Figure 6: Training and Test Ablations. (a) Success rates of three models across the four task suites on LIBERO. Current DINO only, decoded future DINO, and delta tokens use 2,048, 14,336, and 2,060 visual tokens in total for one action chunk, respectively. (b) Comparison of success rates on LIBERO as the number of predicted future steps per camera view varies. Each step contributes one delta token per view, and the dashed line marks the default six-step horizon.
Figure 7: Real-world deployment We conduct real-world experiments on a Piper robotic arm. The task is to pick up the bottle/can and place it on the plate. DeltaWorld first predicts the delta tokens, then the Action DiT predicts the action chunk for execution. The frozen delta tokenizer decoder and DINO-to-RGB decoder is applied for DeltaWorld’s predictions.
Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

SettingDeltaWorld adaptationAction DiT training
OptimizerAdamWAdamW
Learning rate10−410^{-4}5×10−55\times 10^{-5}
Weight decay0.40.40.010.01
Warmup steps1,000500
Batch size / GPU322
Gradient accumulation322
Appendix
Table 3: Training Hyperparameters.
Scope / MethodCameraRobotLightLayout
OpenVLA [ 27 ]0.83.58.128.5
NORA [ 24 ]2.237.045.762.1
WorldVLA [ 9 ]0.127.943.738.0
UniVLA [ 7 ]1.846.269.031.9
π0\pi_{0} [ 6 ]13.86.085.068.9
π0\pi_{0} -FAST [ 42 ]65.121.673.268.8
Appendix
Table 4: Success rates (%) across selected LIBERO-Plus perturbation suites. Published baseline results are taken from the official benchmark leaderboard and the WAM robustness study [ 15 , 54 ] .
Figure 8: Delta-token structure under LIBERO-Plus lighting variation. LIBERO-Goal task embeddings under original light (circles) and a changed light condition (triangles).
Future stepsSpatialObjectGoalLongAverage
Default (6)92.697.095.885.892.8
Full horizon (8)88.895.494.681.290.0
Appendix
Table 5: LIBERO success rates (%) with six versus eight future steps.
Figure 14
Figure 11: Task structure in predicted delta tokens across standard LIBERO suites. Each point concatenates synchronized agent-view and wrist-view tokens at one future step, yielding one joint embedding per step. Samples come from middle-phase windows of collected rollouts.
Figure 12: Action DiT attention to predicted future delta tokens. Native cross-attention probabilities are averaged across Action DiT layers. Rows denote action positions within a generated chunk, columns denote predicted future steps, and agent and wrist tokens are shown separately. Probabilities remain normalized over the complete conditioning context, not only the displayed future tokens.

Explore similar work

CardsList
Sep 23, 2026

DeltaWAM: Delta World Action Models for Bimanual Manipulation

World-action models (WAMs) transfer visual and motion priors from pretrained video generators to robot control by jointly modeling visual dynamics and actions. Existing WAMs, however, predict dense future frames during training, repeatedly modeling largely unchanged content and coupling action-conditioned dynamics to nuisance appearance variations. At inference, processing each complete observation with the heavy video expert bottlenecks few-step action generation. Accordingly, we propose DeltaWAM, which jointly predicts visual deltas and actions using dense-anchor, sparse-delta, and action streams, with three architectures that differ in representation and computation sharing. We further develop Streaming Delta Memory (SDM), which updates cached anchor context with compact observed deltas, reducing heavy video-expert processing. On RoboTwin, DeltaWAM with SDM improves average success over Fast-WAM from 81.3% to 85.4% in the clean setting and from 75.8% to 83.9% under visual randomization. The three architectures reduce training FLOPs by 17.78-23.77%, while SDM reduces one-step inference latency and FLOPs by 36.57% and 31.55%, respectively; real-world evaluations further show the highest overall success rate and normalized progress among the evaluated policies. Code: https://github.com/AIGeeksGroup/DeltaWAM. Website: https://aigeeksgroup.github.io/DeltaWAM.

Han Yan, Zishang Xiang, Haokai Jiang +6

School of Computer Science, Peking University · AI2 Robotics

Video Diffusion ModelsBimanual Robotic ManipulationAction-Conditioned World ModelsEfficient Inference for World Action ModelsWorld Action Models
AskPDF
Aug 5, 2026

DreamWAM: Beyond RGB Future Prediction for World Action Models

World Action Models (WAMs) learn action-relevant representations by predicting how the observed world will evolve. Most existing WAMs define this future in RGB space, where task-relevant state transitions are entangled with nuisance variations in texture, illumination, background, and viewpoint. We argue that WAMs should explicitly predict action-relevant future state rather than relying on RGB prediction alone. We introduce DreamWAM, which reformulates future prediction as structured world modeling beyond RGB, representing future states through complementary views of appearance, motion, geometry, and semantics. During training, DreamWAM combines joint latent denoising of RGB and motion with lightweight gated residual branches for geometry and semantics. Shared attention between VideoDiT and ActionDiT allows the action branch to learn from these future-state predictions, while all beyond-RGB supervision branches are disabled at inference and deployment remains RGB-only. Across both no-rollout and joint video-action inference, DreamWAM consistently improves the matched RGB-only baselines on LIBERO, from 97.30% to 98.40% and from 98.00% to 98.90%, respectively. The gains become larger under unseen LIBERO-Plus perturbations, from 51.36% to 63.44% and from 69.16% to 75.47%. The same robustness extends to real-world manipulation, where DreamWAM attains an average success rate of 74.4% across unseen changes in lighting, background, and object layout, compared with 55.6% for Fast-WAM-Joint. These results show that robust world-action learning depends not only on predicting the future, but on representing it in a form that matters for action. The code and models are publicly released at https://github.com/hustvl/DreamWAM.

Shanglin Yuan, Weiheng Zhao, Xin Shi +6

Huazhong University of Science and Technology · D-Robotics · Wuhan University +1

World Model LearningAction-Conditioned World ModelsRobotic ManipulationWorld Action Models
AskPDF
Aug 12, 2026

Foresight Without Seeing: Latent Futures for World Action Models

World Action Models (WAMs) connect visual prediction with robot control, but supplying predictive context often requires expensive future-video generation. Direct policies avoid this cost but lack an explicit interface for accessing future-indexed predictive information. We introduce ForeWAM, a World Action Model that separates forecasting from rendering to expose and shape latent predictive context for efficient control. Its core mechanism, Future-KV, performs a single Video DiT prefill over the current visual latent and noise-initialized future slots, then reuses the resulting key-value states throughout action denoising. To make this context relevant to control, we introduce dynamics registers supervised by latent actions from a frozen teacher during training, encouraging representations of interaction-induced transitions. This reusable context supports a lightweight, single-layer action decoder. We evaluate ForeWAM on LIBERO, LIBERO-Plus, RoboCasa, and real-world manipulation tasks. Without additional policy-level embodied pretraining, ForeWAM improves RoboCasa success by 9.7 percentage points over Fast-WAM at the same budget of 50 demonstrations per task, reaching 59.2%. With a single-layer decoder, it achieves 77.6% success on LIBERO-Plus and reduces policy-query latency to 88.7 ms on an NVIDIA A800, delivering a 6.27-fold speedup over Fast-WAM. These results show that latent predictive computation provides useful foresight for robust, efficient control without explicit future-video generation.

Jiakai Huang, Zhongbo Wu, Siyu Xu +5

Shanghai Jiao Tong University · ACE Robotics · Nanyang Technological University

World Models for RoboticsVisuomotor Policy LearningWorld Action ModelsLatent World Models
AskPDF