World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

Yuejiang Liu; Fan Feng; Lingjing Kong; Weifeng Lu; Jinzhou Tang; Kun Zhang; Kevin Murphy; Chelsea Finn

cs.LGApr 2, 2026

World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

Authors: Yuejiang Liu, Fan Feng, Lingjing Kong, Weifeng Lu, Jinzhou Tang, Kun Zhang, Kevin Murphy, Chelsea Finn, +1 more

Organizations: Stanford University · UC San Diego · Carnegie Mellon University · Google DeepMind · Harvard University

Abstract

General-purpose world models promise scalable policy evaluation, optimization, and planning, yet achieving the required level of robustness remains challenging. Unlike policy learning which primarily focuses on optimal actions, a world model needs to be reliable over a vast space of suboptimal actions, which are often underrepresented in action-labeled robot interactions. To address this challenge, we propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve. The key idea is to decompose action-conditioned state prediction into two independently verifiable factors: state plausibility and action reachability. We show that verifying these factors is significantly more tractable than direct forward prediction due to two underlying asymmetries: the broader availability of action-free data and the lower dimensionality of action-relevant features. Leveraging these asymmetries, we augment a world model with (i) a diverse subgoal generator obtained from video corpora and (ii) a sparse inverse model that infers actions from a subset of state features. By enforcing cycle consistency among proposed subgoals, inferred actions, and forward rollouts, WAV provides an effective verification mechanism in under-explored regimes, where existing methods often fail. Across nine tasks spanning MiniGrid, RoboMimic, and ManiSkill, our method achieves 2x higher sample efficiency while improving downstream policy performance by over 22%.

Figures & tables

Figure 1: Overview of World Action Verifier, a framework that enables action-conditioned world models to verify their predictions and self-improve from an asymmetric forward-inverse cycle: (i) a diverse subgoal generator proposes plausible future states, (ii) a sparse inverse model infers actions from a relevant subset of state features, and (iii) a world model rolls forward and verifies consistency between its predicted state and the proposed state.

Figure 3: Robustness verification of WAV on MiniGrid. (Left) Sample efficiency comparison between Sparse IDM and the World Model with six objects. (Mid) Robustness to increasing state complexity. (Right) Robustness to growing environment stochasticity.

Figure 4: Evaluation of world model learning with WAV on MiniGrid. (Left) Prediction accuracy of Sparse IDM and Vanilla IDM. (Mid) Comparison of different exploration methods. (Right) Robustness of action following across different methods. Results are averaged across 5 seeds.

Figure 5: World-model learning on RoboMimic and ManiSkill. We report 32-frame prediction error (MSE) as the number of training trajectories increases over 3 seeds.

Figure 6: OOD adaptation results on RoboMimic. We evaluate world-model adaptation (normalized prediction error and downstream reward) under visual shifts and objects/interaction shifts. Error bars denote the standard error over 3 seeds.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Can	Lift	Square	PokeCube	PullCube	LiftPeg
RND	$26.5\pm 0.7$	$23.3\pm 0.8$	$39.7\pm 0.8$	$30.1\pm 0.9$	$24.6\pm 0.8$	$23.4\pm 0.7$
Uncertainty	$22.9\pm 0.6$	$20.7\pm 0.6$	$33.0\pm 0.7$	$25.2\pm 0.7$	$19.9\pm 0.5$	$19.4\pm 0.6$
Progress	$22.0\pm 0.6$	$19.6\pm 0.6$	$35.1\pm 0.7$	$27.3\pm 0.7$	$21.1\pm 0.6$	$20.5\pm 0.8$
Vanilla IDM	$20.2\pm 0.2$	$18.3\pm 0.5$	$29.8\pm 0.7$	$23.4\pm 0.6$	$18.2\pm 0.5$	$17.5\pm 0.5$
Ours	$18.2\pm 0.5^{\dagger}$	$16.4\pm 0.4^{\dagger}$	$26.0\pm 0.3^{\dagger}$	$22.4\pm 0.5^{\dagger}$	$17.1\pm 0.4^{\dagger}$	$16.8\pm 0.5^{\dagger}$

Appendix

Table 1: World model prediction error with 200 training samples. Results are reported as mean

\pm

standard error. The best non-Oracle result for each task is highlighted in light yellow. † indicates that the best non-Oracle method is likely significantly better than the second-best non-Oracle/non-IDM method at the 5% level.

Figure 7: Downstream policy performance on RoboMimic and ManiSkill using learned world models. Error bars denote the standard error over 3 seeds.

Figure 8: Visualization of the OOD adaptation settings. Starting from the original RoboMimic Can environment, we construct two categories of target-domain shifts. The first introduces visual shifts, including modified backgrounds and embodiment colors, while preserving the same task dynamics. The second introduces more challenging object and interaction shifts, including scenes with multiple objects and demonstrations collected from policies with different levels of optimality. These settings test whether the learned world model can adapt to both visual nuisance changes and harder dynamics-relevant distribution shifts with limited target-domain data.

Figure 9: Robustness verification of WAV on Robomimic and ManiSkill. Correlation with Oracle ranking. We evaluate how well each method orders informative samples by computing Spearman rank correlations between the method’s assigned scores and Oracle scores on RoboMimic and ManiSkill environments. Higher correlation indicates closer agreement with the Oracle’s ranking.

Category	Count (Transitions)
Total Collected	66,641
Unlabeled Pre-training Set	28,000
Exploration Pool	28,273
Test Set (Action-balanced)	10,368

Appendix

Table 2: Statistics of the collected dataset for exploration. The data is split into an unlabeled pre-training set for learning the video model, an exploration pool for sample acquisition, and an action-balanced test set for evaluation.

Action $\backslash$ Object	red key	blue key	red ball	blue ball
Pick up	$\times$	$\checkmark$	$\times$	$\star$
Drop	$\star$	$\times$	$\checkmark$	$\times$
Toggle	$\star$	$\checkmark$	$\checkmark$	$\star$
Swap (box for …)	$\star$	$\times$	$\checkmark$	$\times$

Appendix

Table 3: Action–object composition coverage in the training and OOS test sets.

\checkmark

denotes compositions seen during training,

\star

denotes OOS-only compositions evaluated at test time, and

\times

denotes combinations absent from both sets.

Figure 10: Verification score vs. ground-truth error. Our method exhibits a strong monotonic alignment with true error, whereas baselines show scattered distributions and frequent misranking.

Figure 11: The distribution of world model’s prediction error on the data selected by different exploration methods in MiniGrid.

Figure 12: Visual description of all tasks used from RoboMimic (Row 1) and Maniskill (Row 2).

World Model
Replay capacity	$1\times 10^{5}$
Batch size	16
Batch length	32
Optimizer	Adam
Reconstruction loss scale	1.0
Learning rate	$1\times 10^{-4}$

Appendix

Table 4: World model training hyperparameters.

Hyperparameter	Value
Num updates	500,000
Train action decoder every	2
Action decoder batch size	128
Action decoder loss weight	1
Action decoder hidden dim	[1024, 1024, 1024]
Action decoder embedding dim	512

Appendix

Table 5: Hyperparameters for inverse dynamics and action decoding.

Figure 13: Qualitative comparison of world-model rollouts under diverse interaction actions (Part I–II). Gold borders denote ground-truth next observations; green and red borders indicate correct and incorrect predictions, respectively. Across both task sets, our method better preserves action-dependent state changes—notably for structured interactions such as Toggle and Swap —than exploration-based baselines.

Figure 14: (Continued.) Task Set B. The Random baseline frequently collapses to predicting the most common primitive motions (e.g., Turn ), failing to model interaction-induced state changes (e.g., Toggle ). In contrast, models trained with data selected by our method capture these state transitions.

Figure 15: Qualitative comparison of world model predictions across different methods on Robomimic Lift.

Figure 16: Qualitative comparison of world model predictions across different methods on Robomimic Square.

Explore similar work

WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models
Aug 5, 2026Bohai Gu, Yueyang Yuan, Taiyi Wu +9World Model-Based Planning Action-Conditioned World Models
1The Hong Kong University of Science and Technology · 2Wuhan University · 3AI Technology Center, Tencent Video, Tencent
$R^2$ -WAM: Repair-and-Reject Post-Training for World Action Models
Oct 4, 2026Ruiyan Xu, Haisheng Su, Sixu Lin +4World Action Models Action-Conditioned World Models
The Chinese University of Hong Kong, Shenzhen · Manifold AI · Southeast University +1
RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation
Jul 1, 2026Byeongguk Jeon, Seonghyeon Ye, JaeHyeok Doo +4World Model Learning Robotic Policy Evaluation
1KAIST · 2Config

Aug 5, 2026J/K move · Enter open · S save

WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models

Bohai Gu, Yueyang Yuan, Taiyi Wu, Dazhao Du+8

1The Hong Kong University of Science and Technology · 2Wuhan University · 3AI Technology Center, Tencent Video, Tencent

World Model-Based Planning Action-Conditioned World Models Reinforcement Learning with Verifiable Rewards World Model Evaluation Interactive World Models Video World Models

Open paper AskPDF arXiv

Interactive video world models are essential for long-horizon planning and exploration, yet they suffer from compounding errors. Post-training methods such as reinforcement learning (RL) can improve these models, but they hit a verification bottleneck: for arbitrary action sequences, no ground-truth future state exists to measure long-term drift. Our key insight is that reversible action cycles make this verification possible: a sequence composed with its inverse must analytically return to the initial state, yielding annotation-free supervision on long-horizon correctness. Building on this, we introduce WorldCycle, a self-verifiable RL framework that constructs closed action cycles and their repeated executions from ordinary action sequences, and optimizes two complementary rewards: a spatial closure reward enforcing symmetry between mirrored forward and reverse segments, and a temporal consistency reward aligning states across repeated cycle executions. These rewards force the model to learn actions as consistent state operators rather than memorized temporal patterns, and extend naturally to out-of-distribution composite action cycles that the base model handles poorly. We further release CycleBench, a diagnostic benchmark for state-returning ability under complex action structures. WorldCycle reduces state returning drift by up to 44% and lifts composite-action accuracy nearly 4x over the base model, providing a vital foundation for physically grounded world models.

Abstract

Explore similar work

WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models

R2R^2R2-WAM: Repair-and-Reject Post-Training for World Action Models

RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation

$R^2$ -WAM: Repair-and-Reject Post-Training for World Action Models