Native Action-Prior Learning from Videos for World Action Models
Authors: Zhaochong An, Fei Zhang, Menglin Jia, Duncan Frost, Zijian Zhou, Yikai Wang, Xudong Wang, Aditya Patel, Belinda Zeng, Tao Xiang, Serge Belongie, Amir Bar, Sen He
Organizations: Meta AI · University of Copenhagen · Physical Intelligence · Imperial College London
World action models integrate future visual dynamics with robot action prediction, but their scalability remains limited by the need for action-annotated robot trajectories. Observation-only videos contain rich evidence about interaction dynamics, but existing approaches typically use them either to pretrain visual representations that must later be adapted for control, or to infer latent actions that are subsequently grounded to robot commands. We present NAVA-WAM, which introduces native action-prior learning by directly pretraining the action policy from observation-only videos, avoiding indirect representation-to-control transfer or a separate latent-action model. Our training consists of two stages. First, we pretrain on observation-only videos, where future-video flow-matching supervision over visual transitions is propagated through transition-structured joint attention to optimize the Action-DiT and learn action-relevant priors. Second, we use action-labeled demonstrations to post-train the Action-DiT for robot control through joint video--action flow matching, while asymmetric attention decouples the visual branch from iterative action denoising and enables efficient action-only inference. Extensive experiments show that NAVA-WAM consistently outperforms prior approaches under both in-distribution and out-of-distribution settings, while demonstrating strong action-label efficiency and effective real-robot generalization. These results establish native action-prior learning as an effective approach to directly pretrain action policies from observation-only videos, providing a scalable path beyond action-labeled robot data.
Figures & tables
Figure 1: Paradigms of leveraging observation-only videos for robot policy learning. (1) Vision representation learning first learns visual representations from videos within a vision encoder and then trains an action policy on top, introducing an indirect representation-to-control transfer. (2) Latent-action learning extracts latent actions from visual transitions as intermediate supervision for policy learning, introducing an additional modeling stage and a potential representational bottleneck. (3) Native action-prior learning directly uses video dynamics to optimize the action policy, avoiding additional intermediate interfaces and representational bottlenecks.
Figure 2: Overview of NAVA-WAM . During native action-prior pre-training, the Action-DiT is optimized solely through future-video supervision, learning action-relevant priors directly from observation-only videos. During post-training, the Video-DiT and Action-DiT are jointly optimized on action-labeled robot demonstrations using video and action flow-matching objectives, adapting the pretrained Action-DiT to continuous robot control. At inference, NAVA-WAM denoises only robot actions without explicitly generating future videos, enabling efficient closed-loop control.
Method
LIBERO
LIBERO-Plus
Direct action policies
π0 ( 2024a )
94.1
53.6
π0 -FAST ( 2025 )
85.5
61.6
StarVLA- α ( 2026b )
96.5
77.0
π0.5 ( 2025 )
96.9
77.4
ABot-M0 ( 2026c )
98.6
80.5
Table 1: Results on LIBERO/LIBERO-Plus ( left ) and RoboTwin 2.0 ( right ). We report average success rate (%). LIBERO and RoboTwin Clean measure in-domain performance, while LIBERO-Plus and RoboTwin Random evaluate out-of-domain generalization. Best results are shown in bold .
Figure 3: Real-robot evaluation on a Franka FR3. Left : success rate over five trials per task and average success over all 15 trials. Right : NAVA-WAM rollouts for the three evaluation tasks, shown from third-person and wrist-camera views. NAVA-WAM achieves 93.3% average success and correctly follows the spatial relation in T1, where both baselines fail across all trials.
Table 2: Ablation studies on RoboTwin 2.0. (a) Comparison of observation-only pre-training methods under varying action-label budgets. (b) Action relevance of learned representations, measured by ridge-regression R2 to ground-truth actions under different probe-data budgets. (c) Inference compute per action chunk and Random-domain performance. Best results are shown in bold .
Figure 4: Attention of learned action representations. We visualize attention from action representations to visual tokens on held-out data, overlaid on the current and next input observations. Our pretrained Action-DiT attends to interaction-relevant regions and aligns with observed motion, while the CoMo latent-action baseline shows more diffuse attention over static regions.
Figure 5: Qualitative visualization of motion transfer across domains. Each example transfers the transition encoded by a source pair ( Source: current → Source: next ) to a different target observation. The source transition specifies the underlying motion cue (orange arrows), which is applied to Target: current to obtain Target: transfer . The Overlay visualizes the transferred state relative to the current target observation, while Target: real shows the corresponding real transition (blue arrows). Across simulation, real-world, and sim-to-real examples, the transferred states follow the source motion despite substantial changes in visual appearance and scene configuration.
Figure 6: Scaling observation-only pre-training data. Success across data scales at different action-label budgets.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Cold Start
Joint Post-training
Trainable
Video-DiT
Video-DiT + Action-DiT
Action horizon
—
16
Learning rate
4×10−5 (video)
1×10−4 (video), 2×10−4 (action)
Epochs
5
10
Scheduler
cosine, 5% warm-up
Global batch size
256
Appendix
Table 3: Downstream training hyperparameters on LIBERO. We first perform action-free target-domain cold start and subsequently jointly post-train the Video-DiT and Action-DiT using action-labeled demonstrations.
Cold Start
Joint Post-training
Trainable
Video-DiT
Video-DiT + Action-DiT
Action horizon
—
16
Learning rate
2×10−4 (video)
1×10−4 (video), 3×10−4 (action)
Epochs
1
5
Scheduler
cosine, 5% warm-up
Global batch size
1024
Appendix
Table 4: Downstream training hyperparameters on RoboTwin 2.0. We first perform action-free target-domain cold start and subsequently jointly post-train the Video-DiT and Action-DiT using Clean-domain demonstrations.
Figure 7: Transition-structured attention masks. The pre-training mask ( left ) and post-training mask ( right ) are shown for three consecutive transition segments S1 – S3 . Each segment contains a clean preceding visual state, a noised future state, and the corresponding Action-DiT tokens, with attention isolated across segments. Pre-training allows noised-future and action-token queries to attend to all tokens within their segment, whereas during post-training, noised-future video queries attend only to visual tokens.
Figure 8: Native action-prior pre-training dynamics. Training and held-out future-video flow-matching losses over pre-training. The training curve shows the raw per-step loss (faint) and its exponential moving average (solid), while the held-out loss is evaluated periodically without smoothing.
Figure 10: Additional qualitative visualizations of motion transfer across domains. Each example transfers the transition encoded by a source pair ( Source: current → Source: next ) to a different target observation. The source transition specifies the underlying motion cue (orange arrows), which is applied to Target: current to obtain Target: transfer . The Overlay visualizes the transferred state relative to the current target observation, while Target: real shows the corresponding real transition (blue arrows). Across simulation, real-world, and sim-to-real examples, the transferred states follow the source motion despite substantial changes in visual appearance and scene configuration.
Method
Spatial
Object
Goal
Long
Avg.
Direct action policies
π0 ( 2024a )
96.8
98.8
95.8
85.2
94.1
π0.5 ( 2025 )
98.8
98.2
98.0
92.4
96.9
World action models
Fast-WAM ( 2026 )
98.2
100.0
97.0
95.2
97.6
Image-WAM ( 2026e )
97.2
99.2
98.8
98.4
98.4
Appendix
Table 5: Per-suite success rates (%) on LIBERO. All methods are trained on 50 demonstrations per task and evaluated over 50 rollouts per task. Avg. denotes the macro-average across the four task suites. Baselines with available per-suite results are reported. The best result in each column is shown in bold .
Method
Camera
Robot
Language
Light
Background
Noise
Layout
Avg.
π0 ( 2024a )
13.8
6.0
58.8
85.0
81.4
79.0
68.9
53.6
π0 -FAST ( 2025 )
65.1
21.6
61.0
73.2
73.2
74.4
68.8
61.6
Fast-WAM ( 2026 )
16.4
44.5
68.9
78.2
53.7
37.7
60.7
51.5
Image-WAM ( 2026e )
80.8
50.3
91.4
98.1
85.5
93.8
80.5
83.1
NAVA-WAM
72.8
65.6
93.8
97.9
82.8
84.8
87.1
83.5
Appendix
Table 6: Per-perturbation success rates (%) on LIBERO-Plus. Each column averages over the four LIBERO task suites, and Avg. denotes the macro-average across the seven perturbation types. No LIBERO-Plus trajectories are used for training. The best result in each column is shown in bold .
Task
DP
RDT
π0
UP-VLA
BagelVLA
HALO
Fast-WAM
MV-WAM
Ours
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Clean
Rand.
Adjust Bottle
97.0
0.0
81.0
75.0
90.0
56.0
100.0
17.0
100.0
14.0
97.0
9.0
95.0
0.0
83.0
65.0
100.0
97.5
Beat Block Hammer
42.0
0.0
77.0
37.0
43.0
21.0
66.0
16.0
87.0
16.0
96.0
11.0
74.0
2.0
75.0
53.0
100.0
42.5
Blocks Ranking RGB
0.0
0.0
3.0
0.0
19.0
5.0
38.0
0.0
84.0
4.0
94.0
7.0
0.0
1.0
99.0
88.0
100.0
92.5
Blocks Ranking Size
1.0
0.0
0.0
0.0
7.0
1.0
21.0
0.0
45.0
2.0
58.0
2.0
38.0
0.0
63.0
43.0
70.0
60.0
Click Alarmclock
61.0
5.0
61.0
12.0
63.0
11.0
69.0
41.0
85.0
20.0
83.0
14.0
100.0
38.0
90.0
24.0
100.0
100.0
Appendix
Table 7: Per-task success rates (%) on RoboTwin 2.0. All methods are trained exclusively on Clean-domain demonstrations and evaluated in both the Clean and Random ( Rand. ) domains. Baseline per-task results are taken from MV-WAM ( Chen et al., 2026c ) . The final row reports the average across all 50 tasks. The best result for each task and domain is shown in bold .
Figure 11: Qualitative NAVA-WAM rollouts on RoboTwin 2.0. We show seven successful rollouts (top) spanning diverse manipulation skills and two representative failures (bottom). Each row contains four temporal keyframes, with the head-camera view on the left and the two wrist-camera views stacked on the right. The row header gives the RoboTwin task description, and the final keyframe is outlined in blue for success and red for failure. All examples are from the Random (OOD) domain.
Successful trials ↑
Model
ms / call ↓
T1
T2
T3
Avg (%) ↑
π0.5 ( Physical Intelligence et al., 2025 )
195.7
0/5
5/5
3/5
53.3
DreamZero ( Ye et al., 2026c )
3523.9
0/5
5/5
5/5
66.7
NAVA-WAM
379.2
4/5
5/5
5/5
93.3
Appendix
Table 8: Real-robot success rates and inference cost. We report successful trials out of five runs per task on a Franka FR3, with Avg. denoting the success rate across all 15 trials. T1: move the cube to the left side of the bowl ; T2: put the banana in the box ; T3: put the blue cube on the red cube . ms / call denotes the server-side wall-clock latency per policy call on a single H100. The best result in each column is shown in bold .
World Action Models (WAMs) improve robot policy learning by jointly modeling future visual dynamics and actions. However, their scalability and generalization remain constrained by their reliance on costly expert demonstrations. We challenge this by asking whether future supervision for WAMs must originate from target-task expert trajectories. In this paper, we propose Vid2WAM, an offline distillation framework that transfers visual diffusion priors from a large video foundation model into a compact WAM student. Given an observation and language instruction, Vid2WAM distills supervision through two complementary channels: task-conditioned future rollouts directly supervise the student's future prediction branch, while an inverse dynamics model recovers embodiment-specific pseudo-actions for action learning. To robustly integrate synthetic and real supervision, we introduce source-aware residual action adaptation that learns source-specific corrections around a shared action backbone and mitigates interference from noisy pseudo-actions. During inference, both the video teacher and inverse dynamics model are discarded, leaving only the WAM student for efficient deployment. Simulation and real-world experiments demonstrate that Vid2WAM improves novel-task generalization and data efficiency under limited expert demonstrations while preserving low-latency inference.
Chenhao Qiu, Ruixiang Wang, Runyi Zhao +7
Fudan University · The Chinese University of Hong Kong, Shenzhen · Shanghai Innovation Institute
World Action Models (WAMs) connect visual prediction with robot control, but supplying predictive context often requires expensive future-video generation. Direct policies avoid this cost but lack an explicit interface for accessing future-indexed predictive information. We introduce ForeWAM, a World Action Model that separates forecasting from rendering to expose and shape latent predictive context for efficient control. Its core mechanism, Future-KV, performs a single Video DiT prefill over the current visual latent and noise-initialized future slots, then reuses the resulting key-value states throughout action denoising. To make this context relevant to control, we introduce dynamics registers supervised by latent actions from a frozen teacher during training, encouraging representations of interaction-induced transitions. This reusable context supports a lightweight, single-layer action decoder. We evaluate ForeWAM on LIBERO, LIBERO-Plus, RoboCasa, and real-world manipulation tasks. Without additional policy-level embodied pretraining, ForeWAM improves RoboCasa success by 9.7 percentage points over Fast-WAM at the same budget of 50 demonstrations per task, reaching 59.2%. With a single-layer decoder, it achieves 77.6% success on LIBERO-Plus and reduces policy-query latency to 88.7 ms on an NVIDIA A800, delivering a 6.27-fold speedup over Fast-WAM. These results show that latent predictive computation provides useful foresight for robust, efficient control without explicit future-video generation.
Jiakai Huang, Zhongbo Wu, Siyu Xu +5
Shanghai Jiao Tong University · ACE Robotics · Nanyang Technological University
World Action Models (WAMs) generate both future video and robot actions, offering two connected outputs for post-training supervision. How can a pretrained WAM learn from a stronger Teacher on the histories it encounters during execution? We present WAM-OPD, which collects Student rollout histories and queries a Teacher for paired video and action targets. The Student learns from both targets while retaining its one-step video and action generation at deployment. Across 12 RoboTwin 2.0 tasks, WAM-OPD improves average success from 33.8% to 65.7%; across four real-robot tasks, it improves average success from 51.4% to 64.6%. With the collected Student histories held fixed, joint video-action supervision achieves the highest observed success on all three ablation tasks, while either modality alone also improves performance. A separate comparison with Teacher-generated histories finds task-dependent differences between the two history sources. These results demonstrate the value of paired video-action supervision for improving WAM policies without increasing their deployed sampling budget.