InternW0-$Δ$: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data

Xingyu Miao; Zizun Li; Baole Fang; Kaiwen Song; Tenghui Wang; Hanxue Zhang; Yating Wang; Xudong Li

cs.ROSep 25, 2026

InternW0- $Δ$ : A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data

Authors: Xingyu Miao, Zizun Li, Baole Fang, Kaiwen Song, Tenghui Wang, Hanxue Zhang, Yating Wang, Xudong Li, +40 more

Abstract

World Action Models (WAMs) jointly model visual dynamics and action generation for generalist robot manipulation. A central challenge is to integrate priors from large-scale pretrained models---including visual dynamics, scene semantics, geometry, and motion---into a unified framework for robot action generation. We introduce InternW0- $Δ$ , a unified WAM pretrained on a heterogeneous corpus that outperforms prior methods across simulation benchmarks and real-robot platforms. InternW0- $Δ$ combines pretrained visual dynamics, scene-level semantics, 4D geometric and motion priors, and action generation within a Mixture-of-Transformers (MoT) framework. A pretrained video expert and an action expert interact under semantic guidance from a frozen VLM, while a pretrained 4D foundation model injects geometric and motion priors through training-only distillation. We further introduce Causal Imprint, which learns future-relevant scene changes from training-only future supervision and provides predictive representations directly to the action expert without future-video rollout at inference. For large-scale joint training, we construct a heterogeneous corpus of robot demonstrations, UMI data, egocentric human demonstrations, and Ego2Robot data, curated and aligned under a common state-action representation. The resulting corpus contains over 20K hours of processed training data, to our knowledge the largest open-source corpus of its kind. We pretrain InternW0- $Δ$ on this corpus and demonstrate strong performance across simulation benchmarks and real-robot platforms. We will open source the training code, model weights, infrastructure, data-processing pipeline, and processed data where licenses permit. Project page: https://internrobotics.github.io/InternW0-Delta/

Figures & tables

Figure 1: Architecture overview of InternW0- $\Delta$ . A frozen Wan VAE encodes sparse visual memory comprising anchor (A), recent (R), and current (C) observations. The pretrained Wan2.2-TI2V-5B video expert and the ActionDiT action expert are coupled through 30 directed Mixture-of-Transformers (MoT) blocks. T5 instruction embeddings condition the video expert, while a frozen VLM provides task-conditioned scene semantics to the action expert; current proprioception is independently projected for both experts. Causal Imprint tokens encode change-oriented predictive features from recent and current observations to guide action generation.

Figure 2: 4D-aware representation distillation. A query-based student branch aggregates clean A/R/C features from the video expert and aligns them with cached Track4World descriptors via an auxiliary MSE loss, transferring geometric and motion priors without additional policy inference cost.

Figure 3: World-Action MoT layer and attention mask. (a) The video stream contains anchor (A), recent (R), current (C), future (F), and Causal Imprint ( $\Delta$ ) tokens, while action tokens (Act) form a separate stream. (b) The two experts compute separate query, key, and value projections for joint masked self-attention, followed by expert-specific cross-attention, feed-forward, and residual pathways. (c) Rows and columns index query and key token groups, respectively.

Figure 4: Causal Imprint supervision. Causal Imprint is trained with two complementary objectives. For future-feature alignment (left), Causal Imprint features at block 8 are aligned with spatially corresponding features of the terminal future slice at block 20 using the cosine alignment loss. For direct Causal Imprint supervision (right), the final Causal Imprint features regress adjacent differences between clean ground-truth video latents using a squared-error loss.

Figure 5: Efficient inference with cached visual context. (1) Anchor (A), recent (R), and current (C) observations are encoded by the frozen Wan VAE. (2) Conditioned on the video context, the video expert processes the observation tokens and Causal Imprint tokens once, caching their key/value (K/V) projections at each MoT layer. Future-video tokens are omitted. (3) Starting from Gaussian noise, the action expert generates an action chunk through $N$ flow steps, reusing the cached A/R/C/ $\Delta$ features and the action-side context. Only the action stream is recomputed during denoising. (4) The predicted actions are executed, and the process repeats with an updated cache when a new observation is acquired. Each observation-to-action cycle requires one video-expert prefill and $N$ action-expert updates, without future-video sampling or decoding.

Arm-specific slots
Component	Slot range(s)	Dim.
Arm joints	$[0,7),\ [40,47)$	$2\times 7$
End-effector	$[7,16),\ [47,56)$	$2\times 9$
Gripper	$[16,17),\ [56,57)$	$2\times 1$
Hand	$[17,29),\ [57,69)$	$2\times 12$
Body and mobility slots

Table 1: Canonical layout of the 80-dimensional robot action representation. Slot ranges use zero-based, half-open indexing. Paired arm ranges are listed in left–right order.

Figure 6: Ego and ego-to-robot data processing pipeline. Ego2Robot data undergo all stages, whereas egocentric data undergo only action alignment and action speed alignment. Action alignment maps hand motion to end-effector targets and gripper commands. Kinematic alignment couples base search with trajectory IK to obtain base poses and joint trajectories. Visual alignment combines the rendered robot with the human-removed scene using depth-aware compositing.

Figure 7: Qualitative Ego2Robot synthesis results. After action and kinematic alignment, the robot is rendered from the original egocentric viewpoint and composited into the human-removed scene with depth-aware occlusion.

Setting	Value
Canonical state/action dimension	80
Video resolution	$384\times 256$
Video frames	33
Action horizon	32
Video-to-action frequency ratio	4
Recent-frame offset	32 action steps

Table 4: Pretraining settings.

Benchmark	Training split	Views	Resolution	Normalization	Epochs
EBench ( Gao et al., 2026 )	EBench train	3	$384\times 256$	Z-score	10
RoboDojo ( Chen et al., 2026b )	RoboDojo train	3	$384\times 256$	Z-score	10
LIBERO-Plus ( Fei et al., 2026 )	LIBERO ( Liu et al., 2023 ) train	2	$384\times 256$	Min–max	15
RoboTwin 2.0 ( Chen et al., 2026c )	Clean	3	$384\times 256$	Z-score	10

Table 5: Simulation post-training configurations. LIBERO-Plus is used only for evaluation. RoboTwin 2.0 is trained on the clean split and evaluated on the randomized.

Task	Episodes	Frames	Duration (min)	Epochs
AC-One (gripper)
Luminol reaction	162	433,715	240.95	10
Get a drink	137	244,440	135.80	10
Toast bread	304	311,631	173.13	10
MOF experiment	350	1,173,626	652.02	10
Arx5 (gripper)

Table 6: Real-robot datasets used for post-training and evaluation.

Setting	Value
Canonical state/action dimension	80
Video resolution	$384\times 256$
Video frames	33
Action horizon	32
Video-to-action frequency ratio	4
Recent-frame offset	32 action steps

Table 7: Post-training settings for real robots.

Figure 8: Feature caching for exploratory training. The Cache Manager serves VAE latents and VLM contexts to training through a common interface for online and offline materialization, reducing repeated feature extraction during model development.

Figure 9: Layerwise compilation of the two-stream MoT backbone. An eager orchestrator invokes independently compiled layers and groups three consecutive layers for external non-reentrant checkpointing when memory reduction is required. VLM contexts are padded before action cross-attention.

Method	Camera	Robot	Language	Light	Background	Noise	Layout	Total $\uparrow$
VLA
$\pi_{0}$ ( Black et al., 2025b )	13.8	6.0	58.8	85.0	81.4	79.0	68.9	53.6
$\pi_{0}$ -FAST ( Pertsch et al., 2025 )	65.1	21.6	61.0	73.2	73.2	74.4	68.8	61.6
RIPT-VLA ( Tan et al., 2025 )	55.2	31.2	77.6	88.4	91.6	73.5	74.2	68.4
OpenVLA-OFT ( Kim et al., 2025 )	56.4	31.9	79.5	88.7	93.3	75.8	74.2	69.6
StarVLA ( StarVLA Community, 2026 )	52.5	49.8	88.5	95.7	95.7	73.0	76.9	74.1

Table 9: Zero-shot robustness on LIBERO-Plus. All values are success rates (%).

VLA
Method	Clean2Clean $\uparrow$	Clean2Random $\uparrow$	Overall $\uparrow$
GR00T-N1.7 ( NVIDIA, 2026 )	43.6	20.7	32.2
StarVLA ( StarVLA Community, 2026 )	58.1	10.6	34.4
X-VLA ( Zheng et al., 2026 )	68.0	20.9	44.5
Spatial Forcing ( Li et al., 2026a )	77.2	26.7	52.0
ABot-M0 ( Yang et al., 2026c )	70.7	36.0	53.4

Table 10: Evaluation results on RoboTwin 2.0 ( Chen et al., 2026c ) Clean2Random under clean-only training. Success rates (%) are reported on Clean2Clean and Clean2Random.

VLA
Method	Table Top		Simple PnP		Long Horizon		Overall
Method	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$
StarVLA-OFT ( StarVLA Community, 2026 )	–	–	–	–	–	–	0.0	0.2
$\pi_{0}$ ( Black et al., 2025b )	15.7	30.0	35.0	39.0	17.0	41.0	23.6	37.0
X-VLA ( Zheng et al., 2026 )	8.6	24.0	50.0	54.0	6.2	25.0	23.7	36.0
InternVLA-A1 ( Cai et al., 2026b )	4.3	11.0	43.0	47.0	17.9	46.0	23.9	36.0

Table 11: Evaluation results on EBench ( Gao et al., 2026 )

VLA
Method	Gen-Std		Gen-Rand		Precision		Long-Horizon		Memory		Open		Avg
Method	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$
StarVLA- $\alpha$ ( StarVLA Community, 2026 )	5.00	7.54	0.00	0.33	4.33	9.90	6.50	14.15	2.44	3.34	0.58	0.68	3.24	6.40
X-VLA ( Zheng et al., 2026 )	12.00	17.90	1.00	3.04	12.00	18.32	9.75	16.53	3.56	4.76	0.50	0.55	6.52	10.13
$\pi_{0.5}$ ( Black et al., 2025a )	15.00	20.93	1.00	5.82	5.50	12.40	14.67	23.54	4.56	5.78	1.67	1.98	6.91	11.41
Spatial Forcing ( Li et al., 2026a )	15.00	21.25	4.00	6.98	10.58	17.33	14.58	23.26	4.11	5.43	1.58	1.78	8.04	12.38

Table 12: Evaluation results on RoboDojo ( Chen et al., 2026b ) across six evaluation categories.

Figure 10: Real-robot task execution. Selected video frames illustrate eight tasks, with each row ordered chronologically from left to right. From top to bottom: get a drink , toast bread , Luminol reaction , MOF experiment , place tube , pour water , stack cups , and use dropper . The final frame of the Luminol sequence shows a close-up of the reaction outcome.

Figure 11: Generalization to different target water levels. For the AC-One get a drink task, the rightmost cup shows the marked target level used throughout post-training data collection. The three cups on the left show the outcomes of three successful executions at different marked target levels. Black markings indicate the target levels.

Task	Without pretraining	With pretraining
Toast bread	4/20 (20%)	19/20 (95%)
Luminol reaction	0/20 (0%)	19/20 (95%)

Table 13: Real-robot performance with and without our pretraining stage, followed by task-specific post-training. Each entry reports successful trials out of 20 and the corresponding success rate.

Figure 12: Action jetting visualization. (a)

\ell_{2}

changes between consecutive published 14-dimensional action commands, aligned to the inference request at

t=0

, with

t

measured in published control steps. Hard prefix and training-time RTC are shown as mean curves with chunk switches at

t=10

; VJP is shown as an individual trace with its chunk switch at

t=16

. The gray dashed line at

t=0

also marks the previous hard-prefix plan’s prefix-to-suffix boundary. The blue dashed line at

t=11

marks the training-time prefix-to-suffix boundary. The red dashed line at

t=16

marks both the hard-prefix prefix-to-suffix boundary and the VJP chunk boundary. (b) Mean boundary command change divided by the mean consecutive-command change within the executed new-plan segments, excluding the evaluated boundary. The evaluated boundary is the chunk boundary for VJP and the prefix-to-suffix boundary for hard prefix and training-time RTC. The dashed horizontal line denotes a ratio of one.

Method	Success rate (%) $\uparrow$
Baseline	49.59
Baseline + SMC	53.47
Baseline + SMC + Qwen3.5-2B	60.37
Baseline + SMC + RynnBrain1.1-2B	69.08
Baseline + SMC + RynnBrain1.1-2B + CI	70.80
Baseline + SMC + RynnBrain1.1-2B + CI + $\mathcal{L}_{\mathrm{align}}$	76.45

Table 14: Cumulative ablation study on LIBERO-Plus ( Fei et al., 2026 ) . All variants are independently trained from scratch under the same training and evaluation settings. Starting from the baseline, we progressively introduce Sparse Memory Context (SMC), vision-language conditioning, Causal Imprint (CI), the additional CI alignment objective

\mathcal{L}_{\mathrm{align}}

, and 4D-aware representation distillation

\mathcal{L}_{\mathrm{4D}}

. Success rate denotes the overall success rate (%).

Teacher	Action injection	Success rate (%) $\uparrow$
None	No	76.45
CoWTracker	No	76.15
Pi3X	No	73.63
Track4World	Yes	77.78
Track4World	No	78.37

Table 15: Ablation of distillation design on LIBERO-Plus ( Fei et al., 2026 ) . All variants include SMC, RynnBrain1.1-2B ( Dang et al., 2026 ) , CI, and

\mathcal{L}_{\mathrm{align}}

. Action injection indicates whether the student descriptor is additionally fed into the action expert.

Method	Camera	Robot	Language	Light	Background	Noise	Layout	Overall $\uparrow$
Baseline	60.10	76.77	88.68	95.01	81.23	75.58	78.69	78.59
Robot	74.36	68.39	93.69	96.67	81.97	93.07	80.85	83.73
Ego	65.60	74.26	91.93	94.83	84.20	79.01	78.82	80.45
Ego2Robot	63.10	79.81	93.69	96.76	82.62	82.70	79.15	81.86
UMI	65.92	80.26	96.49	97.20	79.83	84.82	81.38	83.24

Table 16: Pretraining data-source ablation on LIBERO-Plus ( Fei et al., 2026 ) . The baseline uses no pretraining, whereas Robot, Ego, Ego2Robot, and UMI are pretrained exclusively on their respective data sources. All values are success rates (%).

Method	Clean2Clean $\uparrow$	Clean2Random $\uparrow$	Overall $\uparrow$
Baseline-Joint	78.20	4.38	41.29
Robot-Joint	81.36	32.34	56.85
Ego2Robot-Joint	76.92	6.66	41.79
Baseline-EEF	61.58	2.80	32.08
Ego-EEF	61.49	3.13	32.31
UMI-EEF	66.05	13.90	39.98

Table 17: Pretraining data-source ablation on RoboTwin 2.0 ( Chen et al., 2026c ) . Baseline-Joint and Baseline-EEF use no pretraining, whereas Robot-Joint, Ego2Robot-Joint, Ego-EEF and UMI-EEF are pretrained exclusively on their respective data sources. All values are success rates (%).

Method	classify objects		press by number		solve equation		general pickup		arrange largest number		Average
Method	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$	SR $\uparrow$	Score $\uparrow$
Ours	14.00	22.80	4.00	4.00	0.00	0.00	36.00	36.00	0.00	6.80	10.80	13.92
Ours+Astra	80.00	88.00	20.00	20.00	20.00	20.00	76.00	76.00	40.00	56.00	47.20	52.00

Table 18: Comparison of Ours and Ours+Astra on five RoboDojo ( Chen et al., 2026b ) tasks.

Explore similar work

RepWAM: World Action Modeling with Representation Visual-Action Tokenizers
Jun 11, 2026Junke Wang, Qihang Zhang, Shuai Yang +5Latent Action Learning World Models for Robotics
Institute of Trustworthy Embodied AI, Fudan University · 2Robbyant, Ant Group · 3Hongkong University of Science and Technology
OpenWAM: An Open Framework for Composable World-Action Models
Oct 6, 2026Heng Yu, David D. Yuan, Juze Zhang +7World Models for Robotics Robot Dynamics Learning
Stanford University
World-Action Models for Robot Learning and Control: A Survey
Sep 13, 2026Zuxing Lu, Hongjia Zhai, Guanzhi Wang +13World Action Models Action-Conditioned World Models
MBZUAI · Caltech · Amazon FAR +4

Jun 11, 2026J/K move · Enter open · S save

RepWAM: World Action Modeling with Representation Visual-Action Tokenizers

Junke Wang, Qihang Zhang, Shuai Yang, Yiming Luo+4

Institute of Trustworthy Embodied AI, Fudan University · 2Robbyant, Ant Group · 3Hongkong University of Science and Technology

Latent Action Learning World Models for Robotics Language-Conditioned Robot Manipulation Visual Tokenization Robotic Manipulation Vision-Language-Action Models+1

Open paper AskPDF arXiv

This work presents RepWAM, a representation-centric world action model (WAM) built on representation visual-action tokenizers. Existing WAMs typically inherit reconstruction-oriented video tokenizers from pretrained video generation models. Although these tokenizers preserve visual fidelity, pixel reconstruction alone provides limited guidance for learning instruction-following dynamics that connect future prediction with robot control. To address this, we explore a semantic visual-action latent space for representation-centric world action modeling. Specifically, we train a representation visual-action tokenizer that maps visual inputs into aligned visual and latent action tokens. We then pretrain our WAM to jointly model future visual states and the latent actions that connect them under language instructions, followed by adaptation to real robot trajectories for closed-loop manipulation. Experiments on real-world manipulation tasks and simulation benchmarks show that RepWAM delivers strong performance across diverse manipulation settings, while ablations highlight the value of semantic visual-action tokenization over reconstruction-oriented alternatives. These results establish representation visual-action tokenization as a promising foundation for world action models and a step toward generalist robot policies. Code and weights will be available at https://github.com/wdrink/RepWAM.

Abstract

Explore similar work

RepWAM: World Action Modeling with Representation Visual-Action Tokenizers

OpenWAM: An Open Framework for Composable World-Action Models

World-Action Models for Robot Learning and Control: A Survey