Cycle WAM: Coupling Action and Video through Cycle Consistency
Authors: Qinglin Yang, Jiepeng Wang, Yuxing Huang, Haibin Huang, Kai Xu, Chi Zhang, Ruizhen Hu, Xuelong Li
Organizations: Zhejiang University · Institute of Artificial Intelligence (TeleAI), China Telecom · National University of Defense Technology · Institute of AI for Industries, Chinese Academy of Sciences · Shenzhen University
Recent advances in video modeling have driven substantial progress in world action models (WAMs). Existing approaches exploit pretrained video priors by decoding actions from latent representations or inferring them from generated videos. The underlying link is causal: robot actions produce changes in the scene, and video observations of these changes provide evidence for inferring the actions that produced them. However, existing training formulations do not by themselves ensure that actions and their predicted visual consequences are consistent, potentially limiting the use of pretrained capabilities. We treat video and action as paired modalities linked by this causal relationship and learn their correspondence through a unified training framework designed to better exploit pretrained video priors. Cycle WAM first augments video-action co-training with bidirectional conversion to reinforce cross-modal correspondence and recover forward and inverse dynamics. It then introduces a differentiable cycle-reconstruction objective to further couple the two mappings. The resulting model supports both forward and inverse dynamics, with the potential to improve performance through more effective use of pretrained video priors. Cycle WAM achieves state-of-the-art performance among comparable methods on LIBERO-Pro and substantially improves DreamZero on DROID. Comparisons with UWM characterize its inverse-dynamics performance and show sharper visual detail in the displayed forward-dynamics predictions.
Figures & tables
Fig. 2: Overview of Cycle WAM. Left: co-training jointly models future video and robot actions using a video-pretrained WAM. Middle: Bidirectional Conversion learns video-to-action (green) and action-to-video (purple) mappings from paired data. Right: Cycle converts video to action and back to video, using reconstruction to reinforce the learned correspondence. The WAM blocks share parameters. Gray tokens denote noisy inputs; orange, blue, and green tokens denote video, action, and text, respectively. The diagram depicts the modality-level tasks; reconstruction is trained with the native video flow-matching objective.
Method
Spat-T
Spat-S
Goal-T
Goal-S
Avg.
OpenVLA [ 17 ]
0.0
0.0
0.0
0.0
0.0
π0 [ 6 ]
0.0
0.0
0.0
0.0
0.0
π0.5 [ 5 ]
1.0
20.0
2.0
38.0
15.3
MolmoAct [ 18 ]
0.0
0.0
0.0
0.0
0.0
NORA [ 15 ]
0.0
0.0
0.0
0.0
0.0
X-VLA [ 50 ]
0.0
0.0
9.0
1.0
2.5
TABLE I: Success (%) on four selected LIBERO-Pro conditions. T denotes instruction redirection and S position swaps. The final column averages only these four conditions. Baseline values are reported by LIBERO-Pro [ 53 ] , except for the † -marked LIBERO-130-finetuned checkpoints, which we evaluate alongside Cycle WAM with 100 episodes per condition.
Variant
Scene 1
Scene 2
Scene 3
Total
Cycle WAM
20/20
19/20
20/20
59/60 (98.3%)
Cycle WAM w/o Cycle
20/20
19/20
19/20
58/60 (96.7%)
Cycle WAM w/o BC and Cycle
20/20
17/20
15/20
52/60 (86.7%)
TABLE II: Cycle WAM variants on three DreamZero-DROID 14B simulator scenes [ 43 ] , with 20 episodes per scene. BC denotes Bidirectional Conversion. The w/o Cycle variant uses 5k steps of rank-32 LoRA BC; omitting both BC and Cycle gives the official weights. The two ablated runs use the Full20 protocol but are not paired by seed.
Model
MSE ↓
Rot. RMSE ↓
Motion cos. ↑
UWM [ 54 ]
0.06
0.23
0.83
DreamZero † [ 43 ]
0.63
0.63
0.35
Cycle WAM (ours)
0.10
0.20
0.85
TABLE III: Inverse-dynamics summary: total action error, rotation error, and motion-direction agreement. UWM and Cycle WAM use 100 paired windows. † marks LIBERO-130-finetuned DreamZero, a historical reference with different windows and horizon.
Fig. 3: Forward-dynamics examples on LIBERO: GT, UWM, and Cycle WAM (left to right), with two views per panel. Cycle WAM retains sharper boundaries, but object placement and view composition still differ from GT.
Model
MSE ↓
MAE ↓
PSNR ↑
SSIM ↑
Frame FID ↓
UWM
0.03
0.10
15.48
0.57
280.84
Cycle WAM
0.03
0.08
15.92
0.63
96.02
TABLE IV: Forward-dynamics prediction metrics, rounded to two decimals. Relative improvements use unrounded scores.
Variant
Spat-T
Spat-S
Goal-T
Goal-S
Avg.
Cycle WAM
24.0
25.0
19.0
3.0
17.8
w/o Cycle
25.0
23.0
15.0
5.0
17.0
w/o BC and Cycle
27.0
13.0
12.0
2.0
13.5
TABLE V: Cycle WAM control ablations on four selected LIBERO-Pro conditions. BC denotes Bidirectional Conversion; w/o labels indicate omitted training stages. Omitting both stages gives the LIBERO-finetuned DreamZero baseline [ 43 ] . Success (%) uses 100 episodes per selected condition. Budgets and update types differ; matched continuations are discussed in Section V-E .
Fig. 4: Real-robot placement of (a) a remote control and (b) a lemon into a basket. Ordered keyframes show approach, grasping, transport, and release. Dashed arrows indicate schematic motion directions; colored strips denote visually inferred gripper states.
Method
Completed scenes
DreamZero [ 43 ]
0/5
Cycle WAM
3/5
TABLE VI: Real-robot task completion across five evaluated scenes. Entries count scenes with observed task completion.
World Action Models (WAMs) improve robot policy learning by jointly modeling future visual dynamics and actions. However, their scalability and generalization remain constrained by their reliance on costly expert demonstrations. We challenge this by asking whether future supervision for WAMs must originate from target-task expert trajectories. In this paper, we propose Vid2WAM, an offline distillation framework that transfers visual diffusion priors from a large video foundation model into a compact WAM student. Given an observation and language instruction, Vid2WAM distills supervision through two complementary channels: task-conditioned future rollouts directly supervise the student's future prediction branch, while an inverse dynamics model recovers embodiment-specific pseudo-actions for action learning. To robustly integrate synthetic and real supervision, we introduce source-aware residual action adaptation that learns source-specific corrections around a shared action backbone and mitigates interference from noisy pseudo-actions. During inference, both the video teacher and inverse dynamics model are discarded, leaving only the WAM student for efficient deployment. Simulation and real-world experiments demonstrate that Vid2WAM improves novel-task generalization and data efficiency under limited expert demonstrations while preserving low-latency inference.
Chenhao Qiu, Ruixiang Wang, Runyi Zhao +7
Fudan University · The Chinese University of Hong Kong, Shenzhen · Shanghai Innovation Institute
World Action Models (WAMs) incorporate visual representations from video generation backbones to guide action prediction. Recent efficient WAMs adopt Mixture-of-Transformers (MoT) architectures and compute video representations once for reuse by the action expert. However, intra-expert iteration (\ie, multi-step action denoising) and inter-expert waiting (\ie, sequential execution of the video and action experts) still limit inference efficiency. To this end, we present RealtimeWAM, an extremely efficient WAM variant with one-step action generation and asynchronous inference, addressing these two bottlenecks. To reduce intra-expert iteration, we propose Teacher-Anchored Consistency Distillation (TACD) to address a local-global error gap: low local consistency error alone does not guarantee accurate final actions. TACD supplements local consistency with explicit supervision from the frozen teacher's multi-step rollout endpoint, enabling accurate one-step action generation. Additionally, we propose Cross-Expert Wavefront Pipelining (CEWP) to eliminate unnecessary expert-level waiting. It overlaps the two experts through block-wise sharing of the video KV cache, synchronizing only immediately before the corresponding action attention consumes it. Extensive experiments across diverse benchmarks (\eg, LIBERO, LIBERO-Plus and RoboTwin) and model variants (\eg, Fast-WAM and Faster-WAM) demonstrate the superiority of RealtimeWAM. Notably, RealtimeWAM maintains near-lossless performance (\ie, <1% drop) across these benchmarks while delivering significant end-to-end speedup (\eg, ∼25× on H100). Our code and checkpoints are available via this link.
Chengtao Lv, Jinyang Du, Shuyi Feng +7
Nanyang Technological University · Beihang University · Sensetime +1
World-action models (WAMs) couple future prediction with robot control, yet existing systems often vary the video backbone, interaction structure, supervision, and inference procedure simultaneously, making their design choices difficult to compare. We introduce OPENWAM, an open world-action modeling framework built around a common causal robot-video foundation and configurable video-action interaction. Starting from Wan2.2-5B, we perform causal robot-video pretraining on over 10,000 hours of video, then integrate an action expert through a shared Mixture-of-Transformers architecture that supports joint, video-then-action, action-then-video, and decoupled generation. OPENWAM achieves high success rates on four LIBERO suites and real-world bimanual tasks; robot-video training with causal adaptation improves VTA success on LIBERO-Long from 68.4% to 97.8%. The same configurable architecture naturally extends to inverse and forward dynamics, allowing us to study how counterfactual transitions improve independently trained dynamics models beyond demonstrations alone. When only the video predictor is adapted to a new task, a frozen local-context inverse dynamics model trained on counterfactual data and demonstrations achieves 84.0% mean success across four held-out LIBERO-90 tasks, compared with 47.0% for a full-context inverse model and 21.5% for a local-context model trained only on demonstrations. For forward dynamics, counterfactual supervision reduces RGB prediction error by 34.5% and raises outcome identification from 21.1% to 71.3% among 16 same-state outcomes. OPENWAM provides a common testbed for comparing WAM interaction designs and for studying dynamics learning from video data beyond successful demonstrations.