Cycle WAM: Coupling Action and Video through Cycle Consistency
Authors: Qinglin Yang, Jiepeng Wang, Yuxing Huang, Haibin Huang, Kai Xu, Chi Zhang, Ruizhen Hu, Xuelong Li
Organizations: Zhejiang University · Institute of Artificial Intelligence (TeleAI), China Telecom · National University of Defense Technology · Institute of AI for Industries, Chinese Academy of Sciences · Shenzhen University
Recent advances in video modeling have driven substantial progress in world action models (WAMs). Existing approaches exploit pretrained video priors by decoding actions from latent representations or inferring them from generated videos. The underlying link is causal: robot actions produce changes in the scene, and video observations of these changes provide evidence for inferring the actions that produced them. However, existing training formulations do not by themselves ensure that actions and their predicted visual consequences are consistent, potentially limiting the use of pretrained capabilities. We treat video and action as paired modalities linked by this causal relationship and learn their correspondence through a unified training framework designed to better exploit pretrained video priors. Cycle WAM first augments video-action co-training with bidirectional conversion to reinforce cross-modal correspondence and recover forward and inverse dynamics. It then introduces a differentiable cycle-reconstruction objective to further couple the two mappings. The resulting model supports both forward and inverse dynamics, with the potential to improve performance through more effective use of pretrained video priors. Cycle WAM achieves state-of-the-art performance among comparable methods on LIBERO-Pro and substantially improves DreamZero on DROID. Comparisons with UWM characterize its inverse-dynamics performance and show sharper visual detail in the displayed forward-dynamics predictions.
Figures & tables
Fig. 2: Overview of Cycle WAM. Left: co-training jointly models future video and robot actions using a video-pretrained WAM. Middle: Bidirectional Conversion learns video-to-action (green) and action-to-video (purple) mappings from paired data. Right: Cycle converts video to action and back to video, using reconstruction to reinforce the learned correspondence. The WAM blocks share parameters. Gray tokens denote noisy inputs; orange, blue, and green tokens denote video, action, and text, respectively. The diagram depicts the modality-level tasks; reconstruction is trained with the native video flow-matching objective.
Method
Spat-T
Spat-S
Goal-T
Goal-S
Avg.
OpenVLA [ 17 ]
0.0
0.0
0.0
0.0
0.0
π0 [ 6 ]
0.0
0.0
0.0
0.0
0.0
π0.5 [ 5 ]
1.0
20.0
2.0
38.0
15.3
MolmoAct [ 18 ]
0.0
0.0
0.0
0.0
0.0
NORA [ 15 ]
0.0
0.0
0.0
0.0
0.0
X-VLA [ 50 ]
0.0
0.0
9.0
1.0
2.5
TABLE I: Success (%) on four selected LIBERO-Pro conditions. T denotes instruction redirection and S position swaps. The final column averages only these four conditions. Baseline values are reported by LIBERO-Pro [ 53 ] , except for the † -marked LIBERO-130-finetuned checkpoints, which we evaluate alongside Cycle WAM with 100 episodes per condition.
Variant
Scene 1
Scene 2
Scene 3
Total
Cycle WAM
20/20
19/20
20/20
59/60 (98.3%)
Cycle WAM w/o Cycle
20/20
19/20
19/20
58/60 (96.7%)
Cycle WAM w/o BC and Cycle
20/20
17/20
15/20
52/60 (86.7%)
TABLE II: Cycle WAM variants on three DreamZero-DROID 14B simulator scenes [ 43 ] , with 20 episodes per scene. BC denotes Bidirectional Conversion. The w/o Cycle variant uses 5k steps of rank-32 LoRA BC; omitting both BC and Cycle gives the official weights. The two ablated runs use the Full20 protocol but are not paired by seed.
Model
MSE ↓
Rot. RMSE ↓
Motion cos. ↑
UWM [ 54 ]
0.06
0.23
0.83
DreamZero † [ 43 ]
0.63
0.63
0.35
Cycle WAM (ours)
0.10
0.20
0.85
TABLE III: Inverse-dynamics summary: total action error, rotation error, and motion-direction agreement. UWM and Cycle WAM use 100 paired windows. † marks LIBERO-130-finetuned DreamZero, a historical reference with different windows and horizon.
Fig. 3: Forward-dynamics examples on LIBERO: GT, UWM, and Cycle WAM (left to right), with two views per panel. Cycle WAM retains sharper boundaries, but object placement and view composition still differ from GT.
Model
MSE ↓
MAE ↓
PSNR ↑
SSIM ↑
Frame FID ↓
UWM
0.03
0.10
15.48
0.57
280.84
Cycle WAM
0.03
0.08
15.92
0.63
96.02
TABLE IV: Forward-dynamics prediction metrics, rounded to two decimals. Relative improvements use unrounded scores.
Variant
Spat-T
Spat-S
Goal-T
Goal-S
Avg.
Cycle WAM
24.0
25.0
19.0
3.0
17.8
w/o Cycle
25.0
23.0
15.0
5.0
17.0
w/o BC and Cycle
27.0
13.0
12.0
2.0
13.5
TABLE V: Cycle WAM control ablations on four selected LIBERO-Pro conditions. BC denotes Bidirectional Conversion; w/o labels indicate omitted training stages. Omitting both stages gives the LIBERO-finetuned DreamZero baseline [ 43 ] . Success (%) uses 100 episodes per selected condition. Budgets and update types differ; matched continuations are discussed in Section V-E .
Fig. 4: Real-robot placement of (a) a remote control and (b) a lemon into a basket. Ordered keyframes show approach, grasping, transport, and release. Dashed arrows indicate schematic motion directions; colored strips denote visually inferred gripper states.
Method
Completed scenes
DreamZero [ 43 ]
0/5
Cycle WAM
3/5
TABLE VI: Real-robot task completion across five evaluated scenes. Entries count scenes with observed task completion.