Achieving the broad task generalization of embodied models in physical environments remains a central challenge in robot learning. We study this challenge in World Action Models (WAMs) through two complementary capabilities: video generalization, which predicts the visual evolution of new tasks, and action generalization, which translates these predictions into executable behavior through a video-to-action bridge. By varying video and action supervision for tasks outside an initial training set, we examine their respective contributions to task transfer. On RoboTwin 2.0, using only in-distribution video-action supervision yields a 25.07% success rate on out-of-distribution tasks. Introducing videos of these tasks without their action labels raises success to 55.90%, while adding just one action-labeled demonstration per task further improves it to 70.70%. These findings suggest that broader video coverage enables task transfer by leveraging existing inverse-dynamics capabilities, while even sparse action supervision further strengthens those capabilities. Guided by this perspective, we introduce WorldBridge, a WAM with separate video and action experts connected through structured attention and trained with joint flow matching. WorldBridge incorporates visual memory at two temporal scales and action forcing, alongside a unified data processing pipeline that standardizes and filters heterogeneous robot trajectories and egocentric hand-derived supervision. Across multiple manipulation benchmarks, WorldBridge achieves state-of-the-art performance among the compared methods, including 96.23% on RoboTwin 2.0 full data setting and 80.9% on clean to random setting, 82.8% on RoboCasa GR1 TableTop, and 57.6% on DOMINO Level-1. Our study highlights the complementary roles of video prediction and action grounding in developing robots that generalize across tasks.
Figures & tables
Figure 1: Overview of our WorldBridge, a foundational WAM for task generalization. WorldBridge utilizes heterogeneous data sources and is trained with a simple but effective model architecture in two stages. The bottom panels report task-generalization results under different video and action supervision settings, benchmark performance, and real-world deployment results.
Figure 2: Task generalization under progressively expanded supervision. Compared with in-domain (ID) video-action training alone (25.07% OOD success rate), adding out-of-distribution (OOD) videos increases success to 55.90%, while adding a few paired OOD video-action examples further improves it to 70.70%.
Figure 3: Overview of the WorldBridge model architecture.
Figure 4: Attention mask design. (a) Attention mask for pretraining. (b) Attention mask for post-training.
Figure 5: Overview of the WorldBridge data processing pipeline.
Before curation
After curation
Retention
Dataset
Episodes
Hours
Episodes
Hours
(% hours)
Robot datasets
ABC-130K [ 2 ]
130,919
4,130.93
66,426
1,642.30
39.76
AgiBotWorld-Beta [ 7 ]
158,776
2,392.17
155,587
2,286.54
95.58
AgiBotWorld2026 [ 1 ]
22,436
456.85
18,233
390.66
85.51
DROID [ 27 ]
57,774
273.12
56,279
263.80
96.59
Table 1: Robot and ego corpus statistics before and after curation, excluding RoboCasa and RoboTwin. For robot data, hour retention includes subset selection, boundary trimming, length filtering, and quality filtering. For ego data, after-curation episode counts reflect episode-level annotation filtering, and hours count frames passing all seven configured quality checks within retained episodes. Durations are computed using repository-specific frame rates.
Exp. Index
# ID Videos
# ID Actions
# OOD Videos
# OOD Actions
ID SR (%)
OOD SR (%)
Overall SR (%)
1
550
550
0
0
91.85
25.07
71.81
2
550
550
550
0
91.96
55.90 (+30.83)
81.14
3
550
550
550
1
91.70
70.70 (+14.80)
85.40
4
550
550
550
5
92.30
84.67 (+13.97)
90.01
5
550
550
550
20
92.25
90.03 (+5.36)
91.58
6
550
550
550
50
90.22
92.27 (+2.24)
90.83
Table 2: Average success rates (%) on RoboTwin 2.0 under different amounts of OOD action supervision. OOD action counts denote the number of demonstrations per task.
Method
Training: Clean + Randomized
Training: Clean Only
Clean
Randomized
Average
Clean
Randomized
Average
GR00T-N1.7 [ 46 ]
–
–
–
43.6
20.7
32.2
StarVLA [ 53 ]
–
–
–
58.1
10.6
34.4
Xiaomi-Robotics-0 [ 9 ]
–
–
–
62.9
18.2
40.6
π0 [ 4 ]
65.92
58.40
62.16
–
–
–
X-VLA [ 74 ]
72.80
72.84
72.82
68.0
20.9
44.5
Table 3: Success rates (%) on RoboTwin 2.0 [ 12 ] under two training-data settings. Each setting is evaluated in Clean and Randomized scenes, with their average reported. Clean-only training corresponds to the Clean2Random protocol.
Method
RoboCasa GR1
RMBench
DOMINO
SimplerEnv-WidowX
SR
SR
SR
MS
Spoon
Carrot
Cube
Eggplant
Avg.
VLA
OpenVLA [ 28 ]
–
–
1.5
6.1
4.2
0.0
0.0
12.5
4.2
X-VLA [ 74 , 13 ]
–
9.8
–
–
100.0
91.7
95.8
95.8
95.8
π0 [ 4 ]
13.6
–
8.2
24.0
29.1
0.0
16.6
62.5
27.1
PUMA [ 18 ]
–
–
17.2
35.0
–
–
–
–
–
Table 4: Results on RoboCasa GR1 TableTop, RMBench, DOMINO Level-1, and SimplerEnv-WidowX. We report success rates (%) and the DOMINO manipulation score (MS)
Model
Gen-Std
Gen-Rand
Precision
Long
Memory
Open
Average
Score
SR
Score
SR
Score
SR
Score
SR
Score
SR
Score
SR
Score
SR
VLA
GR00T-N1.7 [ 46 ]
3.97
2.44
0.35
0.00
2.54
0.67
8.30
3.58
1.06
0.89
0.18
0.17
2.85
1.31
π0 [ 4 ]
7.18
4.89
0.71
0.22
3.56
0.75
6.19
2.00
3.47
2.11
0.25
0.25
3.48
1.53
ABot-M0 [ 67 ]
9.20
5.44
2.26
1.56
5.50
1.75
3.96
0.50
2.44
2.22
0.72
0.67
3.67
1.73
StarVLA [ 53 ]
7.54
4.67
0.33
0.00
9.90
4.33
14.15
6.50
3.34
2.44
0.67
0.58
6.40
3.24
Table 5: Performance comparison on the RoboDojo benchmark. Score and success rate (SR, %) are reported across six evaluation settings and averaged over the five capability dimensions. Additional entries are taken from the official leaderboard (October 9, 2026) [ 51 ] .
Exp. Index
Action Forcing
Short Memory
Long Memory
RoboTwin SR (%)
RoboCasa SR (%)
RoboDojo Score (%)
1
✗
✗
✗
91.28
67.50
14.50
2
✓
✗
✗
93.31
69.17
16.52
3
✓
✓
✗
93.98
74.33
16.90
4
✓
✓
✓
95.20
78.58
21.07
Table 6: Ablation study of action forcing and memory length on three robotic manipulation benchmarks.
Exp. Index
Pretraining
RoboTwin Clean-to-Random
Robot
Egocentric
Clean (%)
Rand (%)
Avg (%)
1
✗
✗
82.9
42.6
62.7
2
✓
✗
87.0
67.7
77.4
3
✓
✓
88.3
73.4
80.9
Table 7: Effect of robot and egocentric pretraining on RoboTwin Clean-to-Random.
Figure 6: Representative real-robot executions on six manipulation tasks. From top to bottom: arranging flowers, scanning drinks, separating LEGO bricks, packing books, sorting screws, and hanging cups. Arrows and circles highlight key operations, including sequential flower insertion, bimanual brick separation, and precise placement. The drink-scanning task probes three aspects of memory: remembering post-scan placement destinations, tracking previously scanned drinks, and processing the newly introduced drink last.
Figure 7: Task success rates across five methods.
Configuration
P50 (ms)
P95 (ms)
Speedup
Joint video/action, 10/10
957.2
1085.0
1.00×
Decoupled video/action, 1/10
269.7
290.5
3.55×
+ Inference mode and runtime caches
218.7
260.0
4.38×
+ QKV projection and KV buffers
204.1
212.1
4.69×
+ Attention switches (full eager profile)
227.2
232.0
4.21×
+ Compilation, without CUDA Graphs
126.6
134.7
7.56×
Table 8: Cumulative ablation of model-call latency for real-world deployment. Timings are measured after warmup with CUDA synchronization using BF16, recorded observations, precomputed T5 embeddings, and ten action denoising steps. Each configuration uses 60 calls across three measurement rounds. The checkpoint uses no historical observations. Timing includes conditioning VAE encoding and denoising, excluding JPEG processing, text encoding, communication, and robot execution.
Cross-task generalization remains difficult for world action models (WAMs), even when their predicted futures depict plausible manipulation. This gap can be attributed directly: we make this gap directly testable by decoupling a WAM into a world model and an inverse-dynamics model (IDM), then holding the IDM fixed while replacing the source of future images. We introduce Align Once, Evaluate Everywhere (AoE), a camera-space inverse-dynamics model pretrained on large-scale robot data, which provides this recovery interface for different future sources across tasks and environments. Through it, we find that actions recovered from recorded futures of unseen tasks execute almost as well as replayed demonstrations, whereas futures predicted by world models collapse across benchmarks. The failures are not generic: they concentrate on the gripper and interaction geometry while generation capacity is spent on task-irrelevant background. This diagnosis suggests that supervision should target exactly these regions. We therefore introduce the vision-to-action (VITA) loss, which passes generated futures through the frozen AoE and penalizes discrepancies in the recovered actions, redirecting generation toward action-relevant detail. Across two world models and three manipulation benchmarks, this simple change raises the execution success of generated futures from near zero to 47.0%. The same loss also transfers to WAMs and yields consistent gains in native control.
Chen Zhao, Zipeng Ye, Yihan Lin +2
Renmin University of China, Beijing 100872, China · Key Laboratory of Data Engineering and KnowledgeEngineering, MOE, China · Meituan, Beijing 100102, China
Achieving robust and generalizable manipulation across diverse environments remains a fundamental challenge in embodied robotics. Recent world action models achieve strong in-domain performance, yet their gains do not extend proportionally to out-of-distribution scenarios. We attribute this to a structural mismatch between visual and action modalities, whose intrinsically heterogeneous manifolds cause joint optimization to disproportionately degrade action robustness under distribution shift. To address this, we propose MV-WAM, a novel end-to-end framework that jointly models visual prediction, action generation, and value estimation designed to effectively leverage video priors during both training and inference for enhanced action generalization. Key to this unification is a cross-modality causal mask that hierarchically grounds actions in predicted video frames and value function tokens in both modalities. To further narrow the generalization gap, MV-WAM adopts a manifold-aware optimization scheme that explicitly accounts for the structural heterogeneity across modalities. Finally, MV-WAM introduces a progress-value regulation mechanism that estimates task completion and detects misalignment between predicted frames and generated actions, enabling the policy to autonomously identify execution deviations and recover through value-guided rollback. On the RoboTwin simulation, MV-WAM achieves a 55.7% mean success rate on random scenarios without any randomized action supervision, outperforming the strongest baseline by 29.3%. MV-WAM achieves a 77.5% mean success rate across four real-world tasks of varying difficulty on a dual-arm robot. Our results demonstrate that manifold-aware cross-modal alignment is essential for robust policy generalization, offering a path toward deployable robotic manipulation.
Jintao Chen, Peidong Jia, Qingpo Wuwu +13
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics
Improving robot policies on new tasks without collecting additional expert demonstrations remains a central challenge in robot learning. World action models (WAMs) use broad video priors to jointly predict future videos and actions, offering a potential source of supervision for adapting to new tasks. However, generated videos may fail to depict task completion, and even visually successful videos may be paired with inconsistent actions that lead to execution failure. We propose EVO-WAM, a framework that adapts WAMs to unseen tasks by learning from their own generated video-action trajectories, without executing candidate actions in an external environment. First, we augment WAM training with state prediction and anchored multi-frame context to enable complete autoregressive rollouts without external execution feedback. Second, we identify reliable training experience by selecting task-completing prefixes with a vision-language model and verifying their video-action consistency with an inverse dynamics model. Third, we iteratively train the WAM on verified prefixes and generate new rollouts with the updated model. On seven unseen RoboTwin 2.0 tasks, EVO-WAM increases average success rates from 26.9% to 68.0% for Cosmos3 and from 28.5% to 46.4% for DreamZero, reaching approximately 2.5× and 1.6× their initial success rates. On three unseen long-horizon composite tasks in the real world, it improves Cosmos3's average success rate from 20.0% to 76.7%, a gain of 56.7 percentage points. Project Page: https://evo-wam.github.io/.