Video diffusion models provide a source of interaction representations for robot control, but transferring these representations requires reconciling variable camera inputs with heterogeneous action interfaces and closed-loop execution. We present X-World, a framework that transfers a video prior through a fixed-length token sequence to a shared action expert. A residual cross-view attention branch extends a pretrained video backbone to one, two, or three views while retaining its original self-attention path. A layer-aligned Q-Former aggregates backbone features, and dataset-specific modules map between native state and action interfaces and the shared action expert. Training separates video adaptation from action-only pretraining by freezing the video backbone in the latter stage, followed by joint target-domain post-training. The policy extracts video features with a single backbone pass and uses asynchronous action chunking for execution. On LIBERO, X-World reaches 98.5% mean success; its LIBERO-trained checkpoint obtains 76.5% on LIBERO-Plus without further adaptation. The seven-shift breakdown reveals gains in robot, language, lighting, and layout perturbations, alongside weaknesses under camera, background, and noise shifts relative to OpenVLA-OFT. On six dual-arm real-world tasks, it achieves 81.7% mean success over 20 trials per task. Removing video adaptation or cross-view fusion produces the largest observed success drops among the tested removals.
Figures & tables
Figure 1: Overview of X-World . The video backbone is first adapted on robot and human videos with one to three camera views (left). A layer-aligned Q-Former then compresses its features into fixed-length tokens that condition the action DiT, while dataset-specific projections map native states and actions to and from the shared expert (right). Action pretraining learns this interface with the video backbone frozen; target-domain post-training updates all modules using video and action supervision.
Figure 2: Cross-view adaptation of the video transformer. (a) LTX-Video ( HaCohen et al., 2024 ) applies self-attention to a single view. (b) Genie-Envisioner ( Liao et al., 2025 ) fuses concatenated views through cross-view attention. (c) X-World retains pretrained per-view self-attention and learns cross-view fusion through a separate residual branch. Zero initialization preserves the pretrained computation at the start of adaptation while allowing cross-camera interactions to be learned. (d) Training-time masks for one, two, and three available views. Shaded entries allow attention; replicated slots are isolated from available views, supporting joint training across camera counts.
Stage
Data
Loss
Updated modules
1: Video adaptation
Robot/human videos
Lvid
Video backbone and view modules
2: Action pretraining
Robot trajectories
Lact
Q-Former, action expert, native projections
3: Post-training
Target trajectories
Lvid+Lact
All modules
Table 1: Three-stage training of X-World . Stage 1 adapts the video backbone and view modules through video prediction on robot and human data. Stage 2 freezes the backbone and trains the Q-Former, action expert, and native interfaces on action-labeled trajectories. Stage 3 jointly optimizes video and action objectives on LIBERO or real-world demonstrations, updating all modules.
Method
↑ Spatial ↑
↑ Object ↑
↑ Goal ↑
↑ Long ↑
Average
Diffusion Policy ( Chi et al., 2025 )
78.5
87.5
73.5
64.8
76.1
OpenVLA ( Kim et al., 2024 )
84.7
88.4
79.2
53.7
76.5
SpatialVLA ( Qu et al., 2025 )
88.2
89.9
78.6
55.5
78.1
CoT-VLA ( Zhao et al., 2025a )
87.5
91.6
87.6
69.0
83.9
π0 -Fast ( Pertsch et al., 2025 )
96.4
96.8
88.6
60.2
85.5
GR00T-N1 ( Bjorck et al., 2025 )
94.4
97.6
93.0
90.6
93.9
Table 2: LIBERO success rates (%) on the Spatial, Object, Goal, and Long suites. Average is the mean of the four suite scores. Best results are bold with pink backgrounds; second-best results are underlined with lavender backgrounds.
Method
Cam.
Robot
Lang.
Light
Bg.
Noise
Layout
Total
OpenVLA ( Kim et al., 2024 )
0.8
3.5
23.0
8.1
34.8
15.2
28.5
15.6
π0 ( Black et al., 2024 )
13.8
6.0
58.8
85.0
81.4
79.0
68.9
53.6
OpenVLA-OFT ( Kim et al., 2025 )
56.4
31.9
79.5
88.7
93.3
75.8
74.2
69.6
OpenVLA-OFT w ( Kim et al., 2025 )
10.4
38.7
70.5
76.8
93.6
49.9
69.9
55.8
NORA ( Hung et al., 2025 )
2.2
37.0
65.1
45.7
58.6
12.8
62.1
39.0
WorldVLA ( Cen et al., 2025 )
0.1
27.9
41.6
43.7
17.1
10.9
38.0
25.0
Table 3: Zero-shot success rates (%) across seven LIBERO-Plus distribution shifts ( Fei et al., 2025 ) . Cam., Lang., and Bg. denote camera, language, and background changes, respectively. The w and m variants use no wrist input and mixed-data supervised fine-tuning, respectively. OpenVLA-OFT+ is excluded because it is trained on LIBERO-Plus. Best results are bold with pink backgrounds; second-best results are underlined with lavender backgrounds.
Figure 3: Dual-arm evaluation platform. Two UR5e arms use Robotiq 2F85 grippers and wrist-mounted RealSense D405 cameras; a shared D455 camera supplies the third-person view.
Figure 4: Real-world manipulation examples. Each row corresponds to a separate rollout of the language instruction above it, with three third-person snapshots arranged in execution order from left to right.
Figure 5: Real-world task success. Bars show success fractions over 20 trials per policy and task on six dual-arm UR5e tasks under a shared evaluation protocol and identical initial placements. X-World achieves the highest observed success on every task.
Setting
↑ Spatial ↑
↑ Object ↑
↑ Goal ↑
↑ Long ↑
Average
w/o Stage 1
95.4
97.6
92.4
72.4
89.5
w/o Stage 2
99.2
98.8
97.2
94.2
97.4
w/o Stage 1 and Stage 2
97.4
96.2
93.4
81.2
92.1
Shared projections
98.6
98.4
98.2
94.4
97.4
w/o Q-Former
99.8
98.6
98.6
94.6
97.9
w/o video loss
99.8
100.0
97.2
96.2
98.3
Table 4: LIBERO success rates (%) for ablations of training stages, model components, and video loss. Stage 1 is video adaptation and Stage 2 is action pretraining; Average is the mean across the four suites. Best results are bold with pink backgrounds; second-best results are underlined with lavender backgrounds.
Model timing
Deployment timing
T5 encoder (first call)
13
Cached model response
46
VAE
7
Network transfer
32
Video diffusion
12
Data processing
7
Q-Former
5
Control-side overhead
6
Action expert
8
End-to-end refresh
91
Table 5: Single-H100 inference latency (ms). Component timings exclude 14 ms of model-side overhead included in the 46 ms cached response; T5 encoding adds 13 ms on the first call. The 91 ms deployed refresh additionally includes network transfer, data processing, and control-side overhead.
Figure 6: Action-prefix and view-availability analyses. The left panel compares inference without real-time chunking (w/o RTC) with prefix lengths P=2,4,6,8 . The right panel uses the same checkpoint to compare one, two, and three camera views and a shuffled three-view order. Bars report success fractions.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Views
Summary
RoboTwin 2.0 ( Chen et al., 2025 )
3
Domain-randomized dual-arm benchmark designed for robust bimanual manipulation and transfer.
CALVIN ( Mees et al., 2022 )
2
Long-horizon language benchmark built around static and gripper observations for sequential manipulation.
AgiBotWorld ( Bu et al., 2025a )
3
Quality-controlled manipulation data collected in realistic settings with broad skill coverage.
Behavior-1K ( Li et al., 2024 )
3
Household simulation benchmark emphasizing realistic environments and long-horizon daily activities.
RoboCOIN ( Wu et al., 2025 )
3
Bimanual multi-embodiment data with hierarchical annotations for coordinated manipulation skills.
Galaxea ( Jiang et al., 2025 )
3
Open-world robot data from everyday scenes with fine-grained subtask language annotations.
Appendix
Table 6: Datasets for Stage 1 video adaptation. Robot and human videos supply future-frame supervision, while Views gives the number of available camera streams per training example.
Dataset
Views
Summary
RoboTwin 2.0 ( Chen et al., 2025 )
3
Domain-randomized dual-arm trajectories for robust bimanual manipulation and transfer.
CALVIN ( Mees et al., 2022 )
2
Long-horizon language-conditioned trajectories with static and gripper observations.
AgiBotWorld ( Bu et al., 2025a )
3
Large-scale manipulation trajectories collected in realistic settings with diverse skills.
Behavior-1K ( Li et al., 2024 )
3
Household embodied benchmark providing long-horizon trajectories in realistic simulation.
RoboCOIN ( Wu et al., 2025 )
3
Bimanual multi-embodiment trajectories with hierarchical annotations for coordinated skills.
Galaxea ( Jiang et al., 2025 )
3
Open-world manipulation trajectories with fine-grained subtask language supervision.
Appendix
Table 7: Datasets for Stage 2 action pretraining. Robot trajectories provide action supervision, while Views gives the available camera count per example; dataset-specific projections accommodate native state and action formats.
Figure 7: Future-video predictions after Stage 1 adaptation. The three examples use one, two, and three input camera views. Within each example, the first column contains observed frames and the remaining columns show predictions at frames 8, 16, and 24 of the 24-frame horizon; multiple rows correspond to synchronized views of the same sequence.
Figure 8: Future-video predictions after real-world post-training. Table clearing (top) and block stacking (bottom) each show three synchronized camera views. Observed frames occupy the first column, followed by predictions at frames 8, 16, and 24, so each row traces the predicted evolution of one camera view.
Many recent robot policies pursue stronger control by using large pretrained vision-language models (VLMs) as the action backbone. We introduce WorldDiT, a unified diffusion transformer architecture that couples action generation with visual world modeling and achieves strong performance without a large pretrained VLM action backbone. During training, a single diffusion transformer generates continuous action chunks and predicts normalized RGB patch targets from future camera frames. Across four LIBERO simulation suites, WorldDiT lies on the reported Pareto frontier for total model parameters and mean success among methods reporting all four suites. These results provide a strong sub-billion-parameter baseline for future scaling studies.
State-of-the-art Vision-Language-Action (VLA) models excel at semantic generalization but struggle to generalize to unseen physical motions in novel environments. We introduce DreamZero, a World Action Model (WAM) built upon a pretrained video diffusion backbone. Unlike VLAs, WAMs learn physical dynamics by predicting future world states and actions, using video as a dense representation of how the world evolves. By jointly modeling video and action, DreamZero learns diverse skills effectively from heterogeneous robot data without relying on repetitive demonstrations. This results in over 2x improvement in generalization to new tasks and environments compared to state-of-the-art VLAs in real robot experiments. Crucially, through model and system optimizations, we enable a 14B autoregressive video diffusion model to perform real-time closed-loop control at 7Hz. Finally, we demonstrate two forms of cross-embodiment transfer: video-only demonstrations from other robots or humans yield a relative improvement of over 42% on unseen task performance with just 10-20 minutes of data. More surprisingly, DreamZero enables few-shot embodiment adaptation, transferring to a new embodiment with only 30 minutes of play data while retaining zero-shot generalization.
World-action models inherit broad visual and physical priors from video pre-training. However, standard within-video next-frame prediction allows them to rely on appearance shared between the context and future, rather than learning interactions that transfer across scenes and embodiments. We introduce WATCH, a cross-video pre-training that learns from task-paired human demonstrations collected across diverse environments. Conditioned on the video and actions of one demonstration, the model jointly predicts another demonstration's actions and future frames from its current observation. Predicting one demonstration from another encourages the model to learn task-relevant interactions that transfer across scenes and demonstrators. Our pre-training shows that these capabilities can be learned from readily scalable human-human pairs alone, without requiring matched human-robot demonstrations. Analyses before robot post-training show improved motion alignment across embodiments and better action and video prediction on unseen robots, with representations closer to those learned from robot pairs. After robot post-training, WATCH improves MimicDroid in-context success from 40.9% to 68.3% with Cosmos Policy and from 59.8% to 77.7% with Cosmos3, with the largest gains on the unseen levels. Gains persist under context-free post-training across robot embodiments. WATCH outperforms within-video pre-training and co-training baselines, supporting cross-video prediction as a scalable approach to learning transferable world-action representations.