When humans learn to manipulate an unfamiliar object, they may already be able to anticipate how its state will change while still being unable to control it skillfully. This gap between prediction and control motivates us to study how the priors acquired by video generation models can be used effectively by an action policy. We present ZERITH V2.5, a video-action model (VAM) in which the action policy reads intermediate features of a video generation model, and use it to study, under fixed demonstrations and a fixed total number of updates, what kind of predictive training improves subsequent action learning. We find that the way video predictive training is supervised affects subsequent action learning. In our experiments, adapting at many noise levels yields a large gain, while adapting only at the noise level used at deployment yields none. Weight-transfer experiments further show that the gain stays with the adapted weights when they remain in the video generation model and disappears when they are used only to initialize the action policy. Initial loss and gradient measurements at the start of joint training indicate that target-domain predictive training gives the video model a better starting state, making subsequent joint learning of prediction and action easier. Under matched data and training budgets, predictive adaptation improves control. We evaluate the complete policy on LIBERO, LIBERO-Plus, Meta-World MT50, and real-robot tasks.
Figures & tables
Figure 1: Adapt the visual world, then scale the action policy. (a) Bin-lining and SKU-sorting sequences. (Center) S1 uses 3k updates for video-model adaptation, followed by 27k joint video–action updates; deployment uses (W1,R0) . Robot-data pretraining precedes this target-task schedule. (b) LIBERO-Plus learning curves: direct (W0,R0) versus staged (W1,R1) training. (c) Weight-transfer controls at 30k total updates; LIBERO-Plus scores appear above standard LIBERO scores. (d) Initial S2 loss and gradient measurements with R0 fixed, normalized to values before S1. Subscripts 0 and 1 denote original Wan and S1-adapted weight sources, respectively.
Figure 2: Video–action architecture in Stage 2 and at deployment. Frozen encoders supply visual latents, language embeddings, and current-frame geometric features. Clean future frames (lower left) provide velocity targets during training only, whereas future latent slots contain pure noise during Stage 2 and at deployment. The action policy fuses exported video-model memory, static geometry tokens, and action queries through a single shared-softmax attention layer. Stage 1 adapts only the video generation model; Stage 2 attaches the action policy and geometry pathway. In the branch-initialization study, the video generation model and action policy are initialized independently from original or adapted weights.
Figure 3: Illustrative training-budget crossover. Normalized schematic of Equation 5 . A is the pretrained state, C the S1-adapted state, B={q≥q†} the target-performance region, and rAB,rAC,rCB the progress rates along the direct, S1, and subsequent S2 routes. τ is the S1 budget and T∗ the equal-progress crossing. At the target progress q† , TS and TD are the staged and direct budgets; the arrow marks the saving TD−TS .
Symbol
Meaning
S1
Stage 1: video-model-only visual adaptation on target-domain video across the full noise range ( Section 3 )
S2
Stage 2: joint training of the video generation model and action policy, with future slots at σw=1
Direct training
Joint training from the first update, using the full budget
W0 , W1
Original Wan video-model weights; S1-adapted video-model weights
R0 , R1
Policy blocks copied from W0 ; copied from W1
σw
Noise level of the future visual slots, from 0 (clean) to 1 (pure noise)
Table 1: Notation.
Question
Intervention
Starting point
Reporting unit
Section
How far does the system scale?
Pretraining duration
Robot-data pretraining
Single-seed downstream score
Section 5.3
Does staging help?
S1–S2 vs. S2; static geometry
Pretrained Wan
Three-seed mean; paired SD
Section 6.1
Which data route?
Target and mixed-data stages
Route-specific
Three-seed mean; paired SD
Section 6.2
Does staging help across operators?
Schedule × operator
Pretrained Wan
Three-seed mean; paired SD
Section 6.3
Which branch benefits?
(W0,W1)×(R0,R1)
Original / S1 weights
Single-run score
Section 7.1
Which training choices matter?
Visual noise; gradient isolation
Original / S1 weights
Single-run score
Section 7.2
Table 3: Overview of experimental protocols. Standard endpoint controls use 30k total downstream updates; target-domain S1 occupies the first 3k in the staged schedule. Learning curves also compare matched S2 updates, budget sweeps vary total updates, and extra pretraining is accounted for separately.
System
Backbone
LIBERO ↑
Plus ↑
Video inference
π0 [ 24 ]
PaliGemma 3B+0.3B
94.20
53.60
—
π0.5 [ 25 ]
PaliGemma+flow
96.90
85.70
—
OpenVLA-OFT [ 26 ]
OpenVLA 7B
97.10
69.60
—
Fast-WAM [ 7 ]
Wan2.2-5B+1B
97.60
49.14
Current-frame K/V
Cosmos Policy [ 27 ]
Cosmos-Predict2 2B
98.50
82.20
Joint future-state prediction
ZERITH V2.5 (Ours)
Wan2.2, 7B
98.95
93.70
Future-slot K/V
Table 4: Benchmark success (%). Baseline scores are listed for comparison. Both bold scores correspond to one deployed checkpoint. K/V denotes attention keys and values; the final column describes video-model inference treatment. For Cosmos Policy, joint future-state prediction refers to the direct-policy mode used for its reported standard LIBERO score; the predictions are discarded.
Figure 4: Reported benchmark scores. The systems in Table 4 , ordered by LIBERO-Plus score in both panels. (a) LIBERO-Plus success; bars start at zero. (b) Standard LIBERO success; the point-plot axis starts at 90%. Purple marks ZERITH V2.5. Baseline protocols differ; both ZERITH V2.5 scores use one deployed checkpoint.
System
Success (%) ↑
Aggregation / source
π0
47.9
Four difficulty groups, equal weight; SmolVLA [ 28 ]
π0.5
81.6
Four difficulty groups, equal weight; as reported in LA4VLA [ 29 ]
ZERITH V2.5
92.3
Pooled evaluation episodes; reported complete-policy result
Table 5: Meta-World MT50 success (%). Reference results retain their source-specific evaluation protocols. The π0 reference is the robot-pretrained variant in SmolVLA. We do not report whether the MT50 result uses the LIBERO deployment checkpoint.
Figure 5: Real-robot task examples. Each row contains a scene with wrist paths followed by three video keyframes. Paths connect manually annotated image-plane positions from the labeled intervals; colors identify wrists by their initial image-left or image-right positions. Frame labels give playback timestamps from the source recordings.
Pretraining
LIBERO (%) ↑
LIBERO-Plus (%) ↑
0 steps
94.70
57.21
10k
98.25
79.24
20k
98.10
80.50
30k
98.50
82.58
40k
98.82
87.56
1 epoch
98.95
93.70
Table 7: Effect of robot-data pretraining, aggregate success (%), single training runs. Every row uses 30k downstream S1–S2 updates. Zero steps omits robot-data pretraining while retaining the Wan initialization. Bold values identify the same deployed checkpoint.
Figure 6: Performance after additional pretraining. Aggregate LIBERO-Plus (solid circles) and standard LIBERO (dashed squares) success from single training runs. Both series evaluate the same downstream S1–S2 recipe for 30k updates. The zero-step reference omits robot-data pretraining. Checkpoints are ordered categories; aggregate values are listed in Table 7 .
Target training
LIBERO ↑
Plus ↑
Δ Plus (pp) ↑
S2 ( DL ; 30k), baseline
92.15
37.64
+0.00
S1 ( DL ; 3k) → S2 ( DL ; 27k)
95.30
56.80
+19.16
Table 8: S1–S2 versus direct S2 with static geometry. Both schedules use the LIBERO target dataset and 30k total updates. Success rates are three-seed means (%); Δ Plus is the mean percentage-point difference from direct S2.
Training sequence
LIBERO ↑
LIBERO-Plus ↑
S1 ( DL ) → S2 ( DL )
95.30
56.80
S1 ( DM ) → S2 ( DL )
96.15
55.56
S1 ( DM ) → S2 ( DM ) → S2 ( DL )
96.75
63.85
S1 ( DM ) → S1 ( DL ) → S2 ( DL )
95.92
59.73
Table 9: Training routes with mixed data DM and target data DL . Arrows show stage order. Every route receives 30k target-task updates; upstream mixed-data stages are additional. Success rates are three-seed means (%).
Figure 7: Dataset and training-stage ablations. Four routes under static-only geometry, three-seed means. Every route ends with target-task S2; mixed denotes ABC robot data plus LIBERO and target denotes LIBERO. Purple identifies the target-S1 recipe. Plus bars start at zero; the standard-LIBERO point axis starts at 94%.
Target training
F(g,d)
LIBERO ↑
Plus ↑
Δ Plus (pp) ↑
S2
0 (baseline)
92.15
37.64
+0.00
S2
g+d
91.70
38.70
+1.06
S2
g⊙d
91.80
39.93
+2.29
S1 → S2
0 (baseline)
95.30
56.80
+0.00
S1 → S2
g+d
95.55
56.63
−0.17
S1 → S2
g⊙d
95.65
57.48
+0.68
Table 10: Interaction between training schedule and geometric conditioning. Success rates are three-seed means (%). Δ Plus is the mean difference in pp from static geometry within the same schedule. All runs use the LIBERO target dataset and 30k total updates.
Figure 8: Schedule–operator interaction. (a) Three-seed mean LIBERO-Plus success for the six combinations. (b) Paired mean differences with error bars of ±1 sample SD across three training seeds. The upper group compares S1–S2 with direct S2 within each operator; the lower group compares product with static conditioning within each schedule. Differences are in pp.
Direct joint
Staged (3k S1)
S2 updates
Total
Plus ↑
Total
Plus ↑
Δ↑
500
500
0.24
3.500
0.79
+0.55
1.000
1.000
1.63
4.000
0.19
−1.44
2.000
2.000
4.17
5.000
4.22
+0.05
5.000
5.000
21.28
8.000
31.12
+9.84
10.000
10.000
30.98
13.000
39.86
+8.88
Table 11: Budget curve on LIBERO-Plus. “Total” is all updates consumed, so the staged column includes its 3,000 S1 updates. Percentages are over the 10,030-scenario manifest. The last two rows give the 30k-total endpoints. Here Δ is staged minus direct at matched S2 updates, in pp.
Figure 9: LIBERO-Plus learning checkpoints. (a) The recorded learning checkpoints against total updates, including 3k S1 updates for staged training. Lines connect recorded points as visual guides. (b) Registered staged-minus-direct differences at equal S2 updates; these comparisons give staged training 3k more total updates. Budget-sweep endpoints are shown separately in Section 6.5 ; exact values and unequal-budget comparisons remain in Tables 11 and 12 .
Direct checkpoint
First staged checkpoint above it
Staged budget ↓
Exposures ↑
Updates
Plus ↑
Updates
Plus ↑
(% of direct)
saved (M)
500
0.24
3.500
0.79
700.0
−0.96
1.000
1.63
5.000
4.22
500.0
−1.28
2.000
4.17
5.000
4.22
250.0
−0.96
5.000
21.28
8.000
31.12
160.0
−0.96
10.000
30.98
8.000
31.12
80.0
0.64
Table 12: Checkpoint comparisons at matched or exceeded success levels. For every recorded direct checkpoint, the first recorded staged checkpoint with higher LIBERO-Plus success. “Staged budget” is staged updates as a percentage of direct updates; exposures saved are the update difference times global batch 320. Values below 100% indicate fewer updates relative to the listed direct checkpoint.
Figure 10: Success at reported total training budgets. Purple circles denote staged and gray squares direct training. Vertical segments pair the two schedules’ endpoints at the same budget. Staged LIBERO-Plus at 20k is unreported. Exact scores and registered differences are retained in Table 13 .
Training / initialization
LIBERO (%) ↑
Plus (%) ↑
Branch initialization
Direct joint: (W0,R0)
92.35
37.54
Staged, video-model transfer: (W1,R0)
94.70
57.21
Policy transfer only: (W0,R1)
88.80
33.60
Staged, both transferred: (W1,R1)
95.35
56.70
Table 14: Branch-initialization ablations. Single-run success rates (%) at 30k total updates with multiplicative conditioning. R1 copies selected S1 video-model layers.
S1 noise schedule
S2 action-to-video-model gradient
LIBERO ↑
Plus ↑
No S1 (baseline)
Blocked → ramped → full
92.35
37.54
Broad noise
Blocked → ramped → full
95.35
56.70
Fixed σw=1
Blocked → ramped → full
91.10
35.90
No S1
Blocked throughout ( β=0 )
90.90
32.48
Table 15: S1 noise and S2 gradient-routing controls. All runs use LIBERO target data, 30k total updates, and multiplicative conditioning. Success rates (%) are single-run results. Both S1 variants copy the policy from their respective S1 video-model layers.
Figure 11: Branch initialization and training controls. Single-run success at 30k total updates. (a) LIBERO-Plus point estimates for the four (Wi,Rj) combinations; the adjacent numeric column gives standard LIBERO. Purple identifies adapted-video-model initialization, and the diamond marks the deployed (W1,R0) configuration. (b) Direct joint training, direct training with action-to-video-model gradients blocked throughout, staged broad-noise S1, and fixed-noise S1. Broad-noise S1 uses the (W1,R1) result for the noise comparison. Both benchmarks are retained in Tables 14 and 15 .
Variant
F(g,d)
LIBERO ↑
Plus ↑
Δ Plus (pp) ↑
Reference initialization
No geometry
—
92.50
38.64
−0.21
Static geometry (baseline)
0
90.30
38.85
+0.00
Bilinear
g⊙d
92.30
39.63
+0.78
Small-corpus robot-pretrained initialization
Static geometry (baseline)
0
97.78
67.92
+0.00
Table 16: Geometric-conditioning operator ablations. Success rates are three-seed means (%). Δ Plus is the mean difference in pp from the marked static-geometry baseline within the same initialization block.
Figure 12: Geometric-conditioning operator effects. Mean LIBERO-Plus changes from the static-geometry baseline within each initialization setting. Purple identifies the bilinear product. Points show mean differences only; no error bars are reported for these comparisons. g is projected static geometry and d is projected video-feature change.
Figure 13: Policy-interface choices. Standard LIBERO point estimates for source-layer selection and action indexing, grouped by intervention. Purple identifies the deployed options. The enlarged point-plot range makes the small reported differences visible.
Initialization
Lw
La
∥gw∥2
∥ga∥2
cos(gw,ga)
(W0,R0)
7.0856
1.6769
42.734
32.366
0.103
(W1,R0)
0.2089
1.6003
2.049
31.066
0.013
(W0,R1)
7.0856
1.6311
42.736
23.937
−0.070
(W1,R1)
0.2089
1.6000
2.049
38.166
−0.044
Table 17: Initialization probes on eight fixed LIBERO batches. gw=∇WLw and ga=∇WLa are gradients into the video generation model. The rows distinguish the four weight-source combinations; cross-camera interventions are reported separately in Table 18 .
Figure 14: Loss and gradient probes before S2. Visual loss, visual-gradient norm, and action-to-video-model gradient norm on eight fixed batches, each normalized to its own (W0,R0) value (100%). Purple shades identify W1 configurations. Labels give the remaining percentage. Absolute losses and gradients, and gradient cosines are retained in Table 17 .
Initialization
ΔcamLw
ΔcamLa
ΔcamL
(W0,R0)
0.2908
−0.1323
0.1585
(W1,R0)
0.0863
−0.0453
0.0410
(W0,R1)
0.2908
−0.0188
0.2720
(W1,R1)
0.0863
−0.0788
0.0074
Table 18: Loss changes after removing cross-camera attention. Measurements on eight fixed LIBERO batches for each initialization. Each entry is isolated loss minus full-attention loss; values are reported to four decimal places.
Figure 15: Effects of blocking cross-camera attention. Changes in visual, action, and total loss, ΔL=Lisolated−Loriginal , for four initialization pairs. Positive values indicate increased loss. All values are retained in Table 18 . Relative visual-loss changes remain in the text.
Attention statistic
Broad-noise S1
Fixed σw=1
Δ (broad − fixed)
Video generation model, visual attention
Normalized entropy
0.738
0.826
−0.088
Past-frame mass
0.565
0.477
+0.088
Same-frame mass
0.435
0.523
−0.088
Cross-camera mass
0.227
0.234
−0.007
Policy, action attention
Table 19: Attention statistics under broad-noise versus S1 with σw=1 , at 30k checkpoints with fixed inputs and matched action-denoising steps. Masses are fractions of the attention distribution.
Attention statistic
W1,R1
W1,R0
W0,R1
W0,R0
Fixed noise
Isolation
Video model, visual attention
Normalized entropy
0.7377
0.7370
0.7458
0.8419
0.8257
0.8424
Past-frame mass
0.5652
0.5659
0.4728
0.5246
0.4772
0.5358
Cross-camera mass
0.2267
0.2273
0.2220
0.2594
0.2339
0.2647
Policy, conditional action attention
Top-1 conditional mass
0.1749
0.1718
0.1617
0.1632
0.1622
0.1459
Table 20: Attention statistics across initialization conditions. All nine statistics for six configurations at 30k total updates. Fixed noise means S1 with σw=1 ; isolation means full action-gradient isolation. Entropies are normalized; masses are attention fractions. Top-1 and top-4 are conditional shares within action-token attention.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 16: Video- and action-token accounting. (a) Both schedules process video-model tokens for 30k updates; staging processes action tokens for 27k. Gray denotes video-model and purple action tokens. Labels give exposure relative to direct training; hatching marks omitted action work. (b) The analytical identity 0.1α for α=ta/(tw+ta) , where tw,ta count tokens per example. Values come from token counts.
Figure 17: Qualitative visual predictions. Offline frame-16 decoding, with two camera views. Columns: VAE target, original Wan with its native interface, original weights with the S1 interface, broad-noise S1, and fixed- σw=1 S1. Both S1 variants recover recognizable scene structure; the downstream noise control in Table 15 distinguishes their control performance.
Visuomotor policy learning trains robots to turn camera observations, often with language and proprioception, into actions for manipulation and navigation.