cs.CVOct 24, 2026

WMEval: Evaluating Interactive World Models Grounded on Physical Simulations

Authors: Lanxiang Hu, Yixin Huang, Abhilash Shankarampeta, Zilin Dai, Yuyuan Wu, Kun Qian, Junze Liu, Arvind Srinivasan, 

Organizations: UC San Diego · Unity

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Interactive world models hold immense potential for high-stakes applications in robotics, game engines, and immersive digital twins, yet existing benchmarks primarily focus on perceptual similarity rather than the physical-causal coherence essential for real-world utility. We introduce WMEval, a comprehensive benchmark that evaluates interactive world models against Unity-based physical simulations. By leveraging Unity as a high-fidelity ground-truth oracle, we shift evaluation beyond perceptual visual assessment toward the objective state-based verification required for downstream applications. Spanning 9 diverse scenes, WMEval assesses action controllability, scene and character consistency, prompt adherence, kinematics, fluid mechanics, physical causality, and lighting. By executing matched action sequences in both the simulator and the world model, WMEval enables evaluation at both the perceptual level and the physical-causal level, using simulator-derived states as reference. Experiments on state-of-the-art models reveal significant room for improvement in dimensions that remain hidden by standard perceptual metrics but are vital for real-world deployments. Our code is available at: https://github.com/hao-ai-lab/WMEval.

Figures & tables

Figure 1: Overview of WMEval. A shared action sequence {ai}i=1T\{a_{i}\}_{i=1}^{T} , sampled from predefined or rule-based synthetic trajectories over actor and camera controls, is dispatched to two parallel flows: a Unity physical simulator yielding reference observations o^i\hat{o}_{i} and simulator-derived states s^i\hat{s}_{i} , and a world model under evaluation producing generated observations oio_{i} , from which state-like quantities sis_{i} are recovered via CV-based estimators.
Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Visual Gallery of the WMEval Benchmark Environments. The nine Unity scenes are architected to evaluate world models across six core dimensions defined in Section 3 . (a–c) Causal & Kinematic Interactions: BallRoller, BeamDomino, and BilliardGame assess action controllability, multi-body collision dynamics, and multi-stage causal chains. (d, e, i) Spatial & Identity Consistency: ChessBoard, MirrorRollingBall, and WanderingGarden test scene geometry preservation, object permanence, and character identity. (f–h) Specialized Continuum & Environmental Physics: LiquidContainer (fluid mechanics), RacingCar (non-linear vehicle kinematics), and SunRisesAndSets (temporal lighting/shadow kinematics) isolate complex physical phenomena.
SceneAct.Obs.CameraPrimary Evaluation Dimensions
BallRoller2C48Fixed-offset followControllability, cLPIPS, Consistency
BeamDomino2C92Fixed-offset followPhysical Causality, Controllability, cLPIPS
BilliardGame10C + 2D13Fixed overheadKinematics & Mechanics, cLPIPS
ChessBoard2C + 2D8Third-person followScene & Character Consistency
MirrorRollingBall2C297Fixed-offset followConsistency, Kinematics
LiquidContainer6C24Static sideFluid Mechanics
Appendix
Table 10: Summary of the nine WMEval Unity scenes. Act. = action-space dimensionality (C = continuous, D = discrete branches); Obs. = observation-vector length; Camera = camera regime used during evaluation.
Figure 3: BallRoller Scene Views.
Figure 4: BeamDomino Scene Views.
Figure 5: BilliardGame Scene Views.
Figure 6: ChessBoard Scene Views.
Figure 7: MirrorRollingBall Scene Views.
Figure 8: LiquidContainer Scene Views.
Figure 9: RacingCar Scene Views.
Figure 10: SunRisesAndSets Scene Views. (a–c) The environment across a full diurnal cycle, illustrating dynamic shadow kinematics and global illumination changes. (d) A bird’s-eye environmental layout showing the static objects used to evaluate shadow projection accuracy.
Figure 11: WanderingGarden Scene Views.
ModelVLM judgeHuman
Unity Reference0.9170.9170.9040.904
Project Genie0.8900.8900.7800.780
LingBot-World (Cam)0.8900.8900.8430.843
Matrix-Game 3.00.7320.7320.5050.505
HY-WorldPlay0.7780.7780.5220.522
Spearman ρ\rho ( ↑\uparrow )0.900.90
Appendix
Table 11: VLM judge vs. blinded human ratings on prompt-scene consistency. Scores are averaged over the six view ×\times scene cells; Human is the mean of three raters. ρ\rho and τ\tau are the Spearman and Kendall rank correlations between the model rankings induced by the judge and by the human ratings, computed from unrounded scores.
Table 12: Evaluator calibration and validity. Each learned evaluator is checked against simulator-derived quantities or human annotations.
Table 13: Finite-sample robustness. We test whether conclusions from five generations remain stable under three-run subsampling and statistical testing.
Figure 12: impulse_decay (brief vertical jump pulse, then brief lateral strafe pulse, idle otherwise; isolates an underdamped slosh response). Columns: pre-jump, post-jump ring-down, post-strafe lateral slosh, and late decay.
Figure 13: LiquidContainer : chirp_strafe (square-wave strafe sweep 0.3→3.00.3{\to}3.0 Hz; the only profile that excites the bottle’s natural slosh modes). Columns sample the low / mid-low / mid-high / high frequency regimes of the sweep. Unity shows the cola surface visibly raised against the bottle wall by the high-frequency excitation.
CategoryMetricRacing Car (3P)Billiard Game (1P)
KinematicsDTW ↓\downarrow0.940.940.010.01
FDE ↓\downarrow1.731.730.010.01
Mechanical LawsRobust Error (%) ↓\downarrow5.55.54.94.9
Appendix
Table 16: Unity reference for Table 4 : the simulator video scored against itself, giving the error floor of the trajectory and mechanics measurement.
Figure 14: BeamDomino : Stage-aligned keyframes on the canonical action trajectory.
Figure 15: BeamDomino : Stage-aligned keyframes on the direct_strike action trajectory.
Figure 16: BeamDomino : Stage-aligned keyframes on the glancing action trajectory.
Figure 17: Cross-model snapshots on SunRisesAndSets at four orbit fractions (one representative seed per model). Rows top-to-bottom: Project Genie , LingBot-World , Matrix-Game 2.0 , Matrix-Game 3.0 . Columns left-to-right: 0%0\% , 33%33\% , 66%66\% , 99%99\% of orbit completion.
Figure 18: Worst single-step Shadow Shape Drift events per model. Each triplet shows three consecutive frames covering the largest morphological discontinuity in the rollout. The Hu-moment shape descriptor is invariant to translation, rotation, and scale, so a smooth perspective sweep does not register; only genuine shape morphing does.
Figure 19: First scene drop-out per model, with ±8\pm 8 -frame context. tt is the first frame at which the anchor mask falls below the area threshold used for Scene Retention Rate.
More in this topic

World Model Evaluation

World Model Evaluation measures whether learned simulators predict consistent, physically plausible, action-responsive futures that help agents plan and act.

See all 212 papers

Explore similar work

CardsList

  1. WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation

    May 25, 2026Kaining Ying, Hengrui Hu, Siyu Ren +6World Model EvaluationInteractive World Models

    Fudan University · Meituan Longcat Team

  2. WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models

    Jun 30, 2026Ting-Bing Xu, Jiacheng Sui, Zhe Gao +12World Model EvaluationInteractive World Models

    AMAP CV Lab, Alibaba Group · Nanjing University · Tsinghua University +1

  3. PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

    Aug 13, 2026Kaixin Ding, Xi Chen, Minghong Cai +9Long-Horizon Agent EvaluationWorld Model Evaluation

    The University of Hong Kong · Kling Team, Kuaishou Technology · The Chinese University of Hong Kong +1