General-purpose world models promise scalable policy evaluation, optimization, and planning, yet achieving the required level of robustness remains challenging. Unlike policy learning which primarily focuses on optimal actions, a world model needs to be reliable over a vast space of suboptimal actions, which are often underrepresented in action-labeled robot interactions. To address this challenge, we propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve. The key idea is to decompose action-conditioned state prediction into two independently verifiable factors: state plausibility and action reachability. We show that verifying these factors is significantly more tractable than direct forward prediction due to two underlying asymmetries: the broader availability of action-free data and the lower dimensionality of action-relevant features. Leveraging these asymmetries, we augment a world model with (i) a diverse subgoal generator obtained from video corpora and (ii) a sparse inverse model that infers actions from a subset of state features. By enforcing cycle consistency among proposed subgoals, inferred actions, and forward rollouts, WAV provides an effective verification mechanism in under-explored regimes, where existing methods often fail. Across nine tasks spanning MiniGrid, RoboMimic, and ManiSkill, our method achieves 2x higher sample efficiency while improving downstream policy performance by over 22%.
Figures & tables
Figure 1: Overview of World Action Verifier, a framework that enables action-conditioned world models to verify their predictions and self-improve from an asymmetric forward-inverse cycle: (i) a diverse subgoal generator proposes plausible future states, (ii) a sparse inverse model infers actions from a relevant subset of state features, and (iii) a world model rolls forward and verifies consistency between its predicted state and the proposed state.
Figure 2
Figure 3: Robustness verification of WAV on MiniGrid. (Left) Sample efficiency comparison between Sparse IDM and the World Model with six objects. (Mid) Robustness to increasing state complexity. (Right) Robustness to growing environment stochasticity.
Figure 4: Evaluation of world model learning with WAV on MiniGrid. (Left) Prediction accuracy of Sparse IDM and Vanilla IDM. (Mid) Comparison of different exploration methods. (Right) Robustness of action following across different methods. Results are averaged across 5 seeds.
Figure 5: World-model learning on RoboMimic and ManiSkill. We report 32-frame prediction error (MSE) as the number of training trajectories increases over 3 seeds.
Figure 6: OOD adaptation results on RoboMimic. We evaluate world-model adaptation (normalized prediction error and downstream reward) under visual shifts and objects/interaction shifts. Error bars denote the standard error over 3 seeds.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Can
Lift
Square
PokeCube
PullCube
LiftPeg
RND
26.5±0.7
23.3±0.8
39.7±0.8
30.1±0.9
24.6±0.8
23.4±0.7
Uncertainty
22.9±0.6
20.7±0.6
33.0±0.7
25.2±0.7
19.9±0.5
19.4±0.6
Progress
22.0±0.6
19.6±0.6
35.1±0.7
27.3±0.7
21.1±0.6
20.5±0.8
Vanilla IDM
20.2±0.2
18.3±0.5
29.8±0.7
23.4±0.6
18.2±0.5
17.5±0.5
Ours
18.2±0.5†
16.4±0.4†
26.0±0.3†
22.4±0.5†
17.1±0.4†
16.8±0.5†
Appendix
Table 1: World model prediction error with 200 training samples. Results are reported as mean ± standard error. The best non-Oracle result for each task is highlighted in light yellow. † indicates that the best non-Oracle method is likely significantly better than the second-best non-Oracle/non-IDM method at the 5% level.
Figure 7: Downstream policy performance on RoboMimic and ManiSkill using learned world models. Error bars denote the standard error over 3 seeds.
Figure 8: Visualization of the OOD adaptation settings. Starting from the original RoboMimic Can environment, we construct two categories of target-domain shifts. The first introduces visual shifts, including modified backgrounds and embodiment colors, while preserving the same task dynamics. The second introduces more challenging object and interaction shifts, including scenes with multiple objects and demonstrations collected from policies with different levels of optimality. These settings test whether the learned world model can adapt to both visual nuisance changes and harder dynamics-relevant distribution shifts with limited target-domain data.
Figure 9: Robustness verification of WAV on Robomimic and ManiSkill. Correlation with Oracle ranking. We evaluate how well each method orders informative samples by computing Spearman rank correlations between the method’s assigned scores and Oracle scores on RoboMimic and ManiSkill environments. Higher correlation indicates closer agreement with the Oracle’s ranking.
Category
Count (Transitions)
Total Collected
66,641
Unlabeled Pre-training Set
28,000
Exploration Pool
28,273
Test Set (Action-balanced)
10,368
Appendix
Table 2: Statistics of the collected dataset for exploration. The data is split into an unlabeled pre-training set for learning the video model, an exploration pool for sample acquisition, and an action-balanced test set for evaluation.
Action \ Object
red key
blue key
red ball
blue ball
Pick up
×
✓
×
⋆
Drop
⋆
×
✓
×
Toggle
⋆
✓
✓
⋆
Swap (box for …)
⋆
×
✓
×
Appendix
Table 3: Action–object composition coverage in the training and OOS test sets. ✓ denotes compositions seen during training, ⋆ denotes OOS-only compositions evaluated at test time, and × denotes combinations absent from both sets.
Figure 10: Verification score vs. ground-truth error. Our method exhibits a strong monotonic alignment with true error, whereas baselines show scattered distributions and frequent misranking.
Figure 11: The distribution of world model’s prediction error on the data selected by different exploration methods in MiniGrid.
Figure 12: Visual description of all tasks used from RoboMimic (Row 1) and Maniskill (Row 2).
World Model
Replay capacity
1×105
Batch size
16
Batch length
32
Optimizer
Adam
Reconstruction loss scale
1.0
Learning rate
1×10−4
Appendix
Table 4: World model training hyperparameters.
Hyperparameter
Value
Num updates
500,000
Train action decoder every
2
Action decoder batch size
128
Action decoder loss weight
1
Action decoder hidden dim
[1024, 1024, 1024]
Action decoder embedding dim
512
Appendix
Table 5: Hyperparameters for inverse dynamics and action decoding.
Figure 13: Qualitative comparison of world-model rollouts under diverse interaction actions (Part I–II). Gold borders denote ground-truth next observations; green and red borders indicate correct and incorrect predictions, respectively. Across both task sets, our method better preserves action-dependent state changes—notably for structured interactions such as Toggle and Swap —than exploration-based baselines.
Figure 14: (Continued.) Task Set B. The Random baseline frequently collapses to predicting the most common primitive motions (e.g., Turn ), failing to model interaction-induced state changes (e.g., Toggle ). In contrast, models trained with data selected by our method capture these state transitions.
Figure 15: Qualitative comparison of world model predictions across different methods on Robomimic Lift.
Figure 16: Qualitative comparison of world model predictions across different methods on Robomimic Square.
Interactive video world models are essential for long-horizon planning and exploration, yet they suffer from compounding errors. Post-training methods such as reinforcement learning (RL) can improve these models, but they hit a verification bottleneck: for arbitrary action sequences, no ground-truth future state exists to measure long-term drift. Our key insight is that reversible action cycles make this verification possible: a sequence composed with its inverse must analytically return to the initial state, yielding annotation-free supervision on long-horizon correctness. Building on this, we introduce WorldCycle, a self-verifiable RL framework that constructs closed action cycles and their repeated executions from ordinary action sequences, and optimizes two complementary rewards: a spatial closure reward enforcing symmetry between mirrored forward and reverse segments, and a temporal consistency reward aligning states across repeated cycle executions. These rewards force the model to learn actions as consistent state operators rather than memorized temporal patterns, and extend naturally to out-of-distribution composite action cycles that the base model handles poorly. We further release CycleBench, a diagnostic benchmark for state-returning ability under complex action structures. WorldCycle reduces state returning drift by up to 44% and lifts composite-action accuracy nearly 4x over the base model, providing a vital foundation for physically grounded world models.
Bohai Gu, Yueyang Yuan, Taiyi Wu +9
1The Hong Kong University of Science and Technology · 2Wuhan University · 3AI Technology Center, Tencent Video, Tencent
World Action Models (WAMs) emerge as a promising foundation for policy refinement by predicting the consequences of sampled actions. However, visually plausible predictions can mislead policy refinement if they fail to reflect the input actions. To address this mismatch, we introduce R2-WAM, a two-stage repair-and-reject post-training framework that first improves the consistency of predicted futures with input actions, then uses these futures to select inferior action samples for negative fine-tuning. The repair stage grounds imagination in observed robot behavior through a kinematic alignment score that measures agreement between predicted and demonstrated motion, enabling the predicted video to faithfully reflect its input actions. Using the repaired video model, the rejection stage compares imagined outcomes of sampled and demonstrated actions, selectively applying negative fine-tuning to samples whose predicted task progress falls below the demonstrated reference by a prescribed margin. Together, the two stages extend video prediction from representation learning to consequence-based policy refinement without additional environment interaction or changes to the inference procedure. R2-WAM achieves 93.8% average success on RoboTwin 2.0 across clean and randomized settings. On the long-horizon real-world Fold Shirt task, it achieves 87.5% average success, compared with 0% for Fast-WAM. Our project page is available at https://r2-wam.github.io/.
Ruiyan Xu, Haisheng Su, Sixu Lin +4
The Chinese University of Hong Kong, Shenzhen · Manifold AI · Southeast University +1
Video world models are emerging as a scalable alternative for evaluating generalist robot policies, bypassing the physical constraints and engineering burdens of real-world deployment. However, evaluating policies with video world models remains challenging, as world-model errors can make generated rollouts unreliable and slow inference limits large-scale throughput. We introduce RoboWorld, an automated evaluation pipeline that pairs a fast autoregressive video world model with a task-progress-aware vision-language model scoring. To enable reliable long-horizon autoregressive world-model rollouts, we propose Step Forcing, which combines anchored and one-step self-forwarded contexts to reduce train-test mismatch while preserving action-observation dynamics. Together, these components enable RoboWorld to align strongly with real-world robot evaluation across tasks and environments, achieving Pearson's r = 0.989 and Spearman's ρ = 0.970.