ΔWAM: Distilling Action Tangent Fields into World Action Models

Ke Wu; Hanwen Huang; Bo Gu; Kaizhao Zhang; Xiangting Meng; Yupeng Zheng; Zijun Xu; Jieru Zhao

cs.ROOct 7, 2026

ΔWAM: Distilling Action Tangent Fields into World Action Models

Authors: Ke Wu, Hanwen Huang, Bo Gu, Kaizhao Zhang, Xiangting Meng, Yupeng Zheng, Zijun Xu, Jieru Zhao, +1 more

Organizations: Fudan University · TARS Robotics · ShanghaiTech University · Institute of Automation, Chinese Academy of Sciences · Shanghai Jiao Tong University

Abstract

World Action Models (WAM) improve robot policies by augmenting sparse action supervision with dense future prediction. However, much of the predictable future is dominated by appearance and scene persistence rather than action-dependent dynamics. We observe that several recent WAM designs, including optical flow, motion-centric representations, and latent actions, can be understood from a common perspective in which world supervision becomes more efficient as it contains a higher proportion of action-relevant variation. Based on this insight, we introduce Action Tangent Fields, which reformulate world supervision through a local Taylor expansion of how actions induce changes in future dynamics. We represent future dynamics in Residual-VAE space, where the future latent remains recoverable from the current latent and its residual, and use a strong action-conditioned world model (ACWM) to probe the local correspondence between action variations and residual-world variations. This local first-order structure is distilled into the WAM to guide its denoising supervision toward dynamics that are more tightly coupled to action, rather than merely predictable from appearance. Across LIBERO-Plus, RoboTwin, and RoboTwin2.0-Plus, our method consistently improves robustness to lighting, background, camera, layout, and other environmental perturbations. Despite using no large-scale embodied pretraining, it achieves stronger robustness under several distribution shifts than pretrained policies. We further distill multi-step VideoDiT denoising into a single step for efficient inference. Our results suggest that effective WAM supervision should remain information-rich while concentrating its predictive capacity on the directions along which actions change the future.

Figures & tables

Figure 1: $\Delta$ WAM under distribution shift . Left: success rate by perturbation type on LIBERO-Plus and RoboTwin 2.0-Plus, with the average over perturbations shown for each method. Right: the corresponding shifts, camera noise, novel viewpoints and layouts, language, lighting, and unseen backgrounds.

Figure 2: Overview of $\Delta$ WAM . A Wan VideoDiT-5B is mid-trained to predict the Residual-VAE future $R_{t+1:t+H}$ . An embodiment-specific ACWM is then queried at $K$ perturbed actions, and its normalized response differences are orthonormalized into a cached basis $G_{t}$ of locally action-sensitive directions. Post-training keeps the real residual as the target and penalizes its projection onto $G_{t}$ , one VideoDiT pass at $t_{v}=0$ conditions four ActionDiT Euler steps at inference.

Model	Original	Camera	Robot	Lang.	Light	BG	Noise	Layout	Total
VLAs
$\pi_{0}$ ( 2025b )	94.2	13.8	6.0	58.8	85.0	81.4	79.0	68.9	53.6
$\pi_{0}$ rerun ( 2025b )	91.3	61.0	40.8	63.5	89.3	84.1	80.1	76.4	69.4
$\pi_{0}$ -FAST ( 2025 )	85.5	65.1	21.6	61.0	73.2	73.2	74.4	68.8	61.6
$\pi_{0.5}$ ( 2025a )	96.9	75.4	77.5	85.6	96.9	94.6	89.7	85.7	85.7
OpenVLA-OFT m ( 2025 )	97.6	55.6	21.7	81.0	92.7	91.0	78.6	68.7	67.9

Table 1: LIBERO-Plus evaluation results. We report success rates (%) under the original setting and seven perturbation dimensions Zhang et al. (2026b) . The best result in each column is shown in bold , while the second-best result is underlined .

	RoboTwin				RoboTwin 2.0-Plus
Method	Embodied PT.	Clean	Rand.	Avg.	Original	Camera	Robot	Lang.	Light	BG	Noise	Layout	Total
$\pi_{0}$ ( 2025b )	$\checkmark$	65.92	58.40	62.2	–	–	–	–	–	–	–	–	–
$\pi_{0.5}$ ( 2025a )	$\checkmark$	82.74	76.76	79.8	78.4	45.6	27.6	74.4	49.6	71.7	64.9	56.8	58.6
X-VLA ( 2025 )	–	–	–	–	65.6	23.2	65.2	64.4	63.1	58.6	49.7	34.8	53.1
MOTUS ( 2025 )	$\checkmark$	88.66	87.02	87.8	87.0	21.6	85.0	83.2	84.6	84.4	43.1	82.8	71.5
LingBot-VA ( 2026c )	$\checkmark$	92.90	91.50	92.2	92.1	28.9	36.2	87.3	89.0	91.3	80.9	87.9	74.2

Table 2: Results on RoboTwin and RoboTwin 2.0-Plus. We report success rates (%) on RoboTwin under the Clean and Randomized settings, and on RoboTwin 2.0-Plus under the original setting and seven perturbation dimensions. For RoboTwin 2.0-Plus, all methods use a single unified model across all tasks. The best result in each column is highlighted in bold , while the second-best result is underlined . Embodied PT.: embodied pretraining. Lang.: Language. BG: Background.

Figure 3: Real-world rollouts across embodiments. Left: Tianji performs bimanual cooking with dexterous hands under a new background. Right: NERO inserts test tubes with a parallel gripper under new objects layout, numbered frames show the progression of each task.

7-DoF NERO Arm & Gripper
Task	Method	Standard	Layout 1	Layout 2	Red & Green Light	Average
Stack Bowls	$\pi_{0.5}$ ( 2025a )	0.33	0.33	0.17	0.50	0.33
	FastWAM ( 2026 )	0.50	0.50	0.17	0.17	0.33
	$\Delta$ WAM (Ours)	0.67	0.50	0.50	0.50	0.54
Insert Tubes	$\pi_{0.5}$ ( 2025a )	0.83	0.50	0.50	0.50	0.58
	FastWAM ( 2026 )	0.50	0.50	0.33	0.50	0.46

Table 3: Real-world evaluation on NERO and Tianji . Scores are averaged over three trials for each task and condition. The Average column is the mean across the four conditions.

Figure 4: Ablations on LIBERO-Plus. (a) Average success and inference time for different VideoDiT denoising steps. (b) Average success for $K=0,4,8$ local action perturbations.

Explore similar work

MoWAM: Explicit Future Motion Prediction for Efficient World Action Models
Sep 17, 2026Jiayu Wang, Bin Zhu, Yue Yu +1Efficient World-Action Model World Models
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · Singapore Management University, Singapore · Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China
DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models
Jul 28, 2026Haoyuan Ji, Lingxiang Fan, Shang Su +4Faster-Wam Photometric Supervision
Tsinghua University · Dense-AI · University of Michigan
DreamWAM: Beyond RGB Future Prediction for World Action Models
Aug 5, 2026Shanglin Yuan, Weiheng Zhao, Xin Shi +6Efficient World-Action Model World Models
Huazhong University of Science and Technology · D-Robotics · Wuhan University +1

Sep 17, 2026, cs.ROJ/K move · Enter open · S save

MoWAM: Explicit Future Motion Prediction for Efficient World Action Models

Jiayu Wang, Bin Zhu, Yue Yu, Jingjing Chen

College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · Singapore Management University, Singapore · Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China

Efficient World-Action Model World Models Robot Policies Transformer Architectures

Open paper AskPDF arXiv

World Action Models (WAMs) improve robot policy learning by incorporating future dynamics, yet explicitly generating future videos at inference introduces substantial computational overhead. Removing future generation improves efficiency, but leaves future dynamics only implicitly encoded in observation features, which can limit robustness under distribution shifts. We propose MoWAM, an efficient WAM that replaces future video generation with explicit future motion prediction. Instead of reconstructing the complete future scene, MoWAM models structured robot motion as a compact abstraction of the future, capturing how the robot is expected to evolve under the current scene and interaction constraints. A Mixture-of-Transformer architecture learns future visual dynamics during training while jointly predicting motion and action, allowing video generation to be removed entirely at inference while retaining an explicit representation of the future. The compact motion representation further enables efficient inference-time scaling by sampling multiple candidates of motion and action pairs and selecting among them with a motion-aware task-progress verifier. Experiments on LIBERO, LIBERO-Plus, and real-world manipulation tasks demonstrate that MoWAM achieves strong in-distribution performance, improved out-of-distribution robustness, and higher average real-world success than representative WAM baselines. In addition, performance improves as more candidates are explored, demonstrating that explicit future motion provides an effective and efficient basis for inference-time scaling.

Abstract

Explore similar work

MoWAM: Explicit Future Motion Prediction for Efficient World Action Models

DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models

DreamWAM: Beyond RGB Future Prediction for World Action Models