Organizations: The Chinese University of Hong Kong · Peking University · University of Science and Technology of China · Department of Computer Science and Engineering, CUHK · Shanghai Artificial Intelligence Laboratory · Nanyang Technological University
Video world models generate future visual observations from camera or action inputs, with the goal of synthesizing persistent 3D worlds that can be explored across time and space. Existing benchmarks typically evaluate short interactions or rely on camera trajectories fixed before generation. The performance of video world models under long-horizon exploration with extensive camera motion remains under-explored. We introduce WorldProbe, a diagnostic benchmark that uses a goal-conditioned multimodal agent to adapt camera actions online during generation. WorldProbe comprises 164 test cases across four probing families. Across 10 state-of-the-art models, we generate more than 1,600 rollouts totaling approximately 800 minutes, with horizons up to 180 seconds. Our evaluation across four dimensions reveals both the degradation of local measures and the emergence of long-range inconsistencies. We find that recent models sustain visual and content quality for longer, while long-range memory and evolution remain weak. These results expose a gap between locally stable generation and persistent world consistency that is largely missed by existing benchmarks. Code and benchmark resources are available at https://github.com/Untitled-W/WorldProbe.
Figures & tables
Figure 1: WorldProbe overview. A goal-conditioned agent adaptively probes an evolving video world through closed-loop camera control, enabling long-horizon evaluation across four task families and four evaluation dimensions.
Benchmark
Int.
Cplx.
Len.
N
WorldModelBench ( Li et al., 2025a )
×
Low (T)
∼ 10 s ∗
350
WorldScore ( Duan et al., 2025 )
×
Mid (CC)
≤ 10 s ∗
3000
WorldMark ( Xu et al., 2026b )
×
Mid (TC)
60 s
500
WBench ( Ying et al., 2026 )
×
Mid (CC)
≤ 9 tr.
289
LoopNav ( Lian et al., 2025 )
×
Mid (TC)
∼ 1200 fr.
144
Omni-WorldBench ( Wu et al., 2026a )
×
Mid (CC)
∼ 5.7 s ∗
1068
Table 1: Comparison with existing video world-model benchmarks. Int. interactive control. Cplx. probing complexity: T = text prompt; CC = case-by-case design; TC = template composition; AA = agent action. ∗ inferred from the dataset.
Figure 2: Four scene-adaptive probing tasks. Each task induces a distinct spatial exploration pattern: approaching a constrained target, traversing an open path, inspecting a scene across viewpoints, or successively exploring new regions.
Model
Latency
Chunk ∗
Avg. Len
Pass Rate
> 60s
(per chunk)
(frames/FPS)
Matrix-Game 2.0
1.469s
12/12
54.4s
73.2%
26.8%
WorldPlay (HY)
14.651s
16/24
59.6s
81.1%
24.2%
WorldPlay (Wan)
3.418s
16/16
42.4s
57.9%
9.8%
LingBot-World
6.681s
12/16
39.5s
48.8%
12.4%
Matrix-Game 3.0
8.178s
40/17
49.5s
62.2%
17.6%
Table 2: Video world model inference configuration. Per-chunk latency and rollout statistics for evaluated models.
Module
Latency
(per chunk)
Qwen reasoning
3.025s
SAM3.1 tracking
7.609s
DA3+ Voxel
0.5143s
Table 3: Additional processing latency.
Model
Visual
Content
Mean
Slope
Mean
Slope
Matrix-Game 2.0
0.565
−0.77
6.09
−0.061
WorldPlay (HY)
0.542
−3.30
6.08
−0.072
WorldPlay (Wan)
0.603
−2.91
6.29
−0.092
LingBot-World
0.532
−2.27
6.61
−0.026
Matrix-Game 3.0
0.556
+0.76
5.76
−0.051
Table 4: Local quality and temporal degradation across models. Mean measures average rollout-level quality, while Slope measures its change over time; more negative slopes indicate faster degradation. Visual scores are normalized to 0–1, and content scores retain their 0–10 scale. Visual slopes are reported in 10−3 /s; content slopes are in points/s.
Figure 3: Representative local-quality behaviors. Metric degradation aligns with visible quality loss, while some long rollouts sustain rich content without evident collapse.
Figure 4: Local quality degrades over long-horizon interaction. Per-rollout visual and content trajectories show stronger collapse in earlier models, while more recent models generally maintain a higher quality floor. Models are ordered by release time from left to right.
Figure 5: Extended closed-loop rollout. Sampled observations from a 367.9 s DreamX rollout driven by 491 agent actions.
Figure 6: Representative cases for spatial memory. Long-range revisits show strong inconsistencies across spatially separated viewpoints, while short-range revisits better preserve scene structure; memory scores align with visible consistency across revisitation.
Figure 7: Representative evolution behaviors. Low-scoring cases exhibit entity and scene drift, implausible dynamics, and action-inconsistent changes, while high-scoring cases preserve substantially more coherent evolution.
Memory
Evolution
Model
Scene
Layout
Entity
Texture
Overall
VP/seq.
Entity
Scene
Physics
Causality
Overall
Cxt/seq.
Matrix-Game 2.0
4.73
3.66
3.47
4.19
4.01
2.14
3.78
3.59
3.60
3.28
3.56
12.68
WorldPlay (HY)
6.14
4.99
4.96
5.37
5.37
1.74
3.56
3.86
3.37
3.49
3.57
13.62
WorldPlay (Wan)
4.90
3.61
3.56
4.57
4.16
1.88
2.72
2.40
2.48
2.43
2.51
12.03
LingBot-World
4.59
3.22
2.92
4.37
3.78
2.66
4.30
3.79
3.77
3.21
3.77
12.23
Matrix-Game 3.0
7.14
5.75
5.81
6.54
6.31
2.24
3.41
3.62
3.07
3.08
3.29
12.67
Table 5: Long-range memory and evolution fidelity across models. Memory evaluates scene, layout, entity, and texture consistency across revisited viewpoints; evolution evaluates entity, scene, physical, and causal consistency over extended rollouts.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Visual (normalized)
Content (0–10)
MUSIQ
MANIQA
Q-Align
Geometry
Object Validity
Scene Richness
Mean
Slope
Mean
Slope
Mean
Slope
Mean
Slope
Mean
Slope
Mean
Slope
Matrix-Game 2.0
0.667
−0.08
0.378
−0.22
0.649
−2.01
6.14
−0.060
6.07
−0.063
6.07
−0.059
WorldPlay (HY)
0.622
−1.87
0.346
−1.13
0.657
−6.89
5.97
−0.077
5.90
−0.082
6.38
−0.057
WorldPlay (Wan)
0.687
−0.24
0.400
−0.92
0.722
−7.57
6.06
−0.102
5.94
−0.117
6.87
−0.058
LingBot-World
0.606
−1.70
0.331
−2.21
0.660
−2.90
6.88
−0.021
6.66
−0.031
6.28
−0.026
Appendix
Table 6: Per-dimension frame-level visual-IQA and content scores across the 327 rollouts with complete scores in both groups. For each model and dimension, Mean is the equal-weighted average of rollout-level frame means; Slope is the equal-weighted average of per-rollout least-squares slopes against elapsed time in seconds. Visual scores are normalized to their theoretical ranges (MUSIQ: 0–100, MANIQA: 0–1, Q-Align: 1–5); their slopes are reported in 10−3 /s. Content dimensions retain their 0–10 scale and slopes are in points/s. Best mean and slope in each column are in bold.
Figure 8: WorldProbe’s service and data flow. The runner mediates the task agent, auxiliary perception modules, and WMI. Bidirectional arrows indicate request and response; the final artifact flows to rollout screening and metric evaluation.
Model / driver
GPUs
Configured chunk bound
Native step, cache, and adaptation
Matrix-Game 2.0
1
180
3 latents per action; a 4- or 6-latent visual-KV window by game; 32-slot causal-VAE feature cache; the token cursor advances by 2,640 per step.
WorldPlay (Hunyuan)
2
270
4 latents per action; geometry-aware history retrieval from up to 20 frames.
WorldPlay (Wan)
4
180
4 latents per action; a 16-latent visual context composed of 12 recent and 4 field-of-view-selected remote latents.
LingBot-World (v1 Fast)
2
240
3 latents per action; local attention 18 with 6 sink latents; four denoising evaluations per chunk.
Matrix-Game 3.0
2
77
Initial segment of 15 latents; subsequent segments add 10 latents with 4 preceding latents for image conditioning; field-of-view memory selects 5 latents.
Table 7: Code-derived streaming capacities and model-specific state.
Model
Largest recorded native run
Public-path control and memory behavior
Matrix-Game 2.0
10 blocks; 117 frames, 9.75 s
3-latent loop with rolling visual/action KV and persistent causal-VAE cache; each block waits for an action on stdin. The standalone entrypoint has no HTTP session API.
WorldPlay (Hunyuan)
10 chunks; 157 frames, 6.54 s
CLI takes the full pose and target length. Its 4-latent loop retains text KV, rebuilds visual KV from selected history, then decodes the accumulated latents.
WorldPlay (Wan)
10 chunks; 157 frames, 9.81 s
Default: 4 chunks (about 61 frames, 3.81 s). num_frames=961 preallocates 241-latent noise, but num_chunk sets generated chunks. Pose and decode remain inside one invocation.
LingBot-World (v1 Fast)
10 chunks; 117 frames, 7.31 s
The CLI takes a complete camera trajectory and owns the 3-latent loop. It commits clean-history KV per chunk, then assembles latents for whole-sequence VAE decoding.
Matrix-Game 3.0
10 segments; 417 frames, 24.53 s
Normal mode constructs a full action/pose trajectory from num_iterations ; interactive mode reads one stdin action per segment but fixes the total count. Optional async VAE uses a separate GPU id.
SANA-WM
40 chunks; 960 frames, 60.00 s
Native 24-frame callbacks use three CUDA streams for Stage-1, refiner, and decoder on one GPU. The 60-s run completed without OOM; the camera/action plan is supplied to the invocation.
Appendix
Table 8: Recorded horizons and control or memory behavior of the public inference paths. Durations are computed from output frames and the corresponding public-path frame rate.
Figure 9: Representative rollout trajectories with local, memory, and evolution diagnostics. Each row shows sampled rollout frames (left), Content and Visual metric curves, and Memory and Evolution comparison frames with dimension scores and failure explanations. Gray headers report the probing task, sample count, and frame layout.
Video world models simulate future states conditioned on current observations and user actions. Recent systems have demonstrated impressive video consistency and action controllability over long sequences. However, fairly comparing these interactive models remains challenging. In practice, a human player typically evaluates a world model by pursuing long-horizon objectives through interaction. For example, a user may turn around 360 degrees to see whether the environment remains consistent, or walk into the water and inspect whether realistic water ripples are generated. The action sequence required to achieve the same objective may vary substantially between models, making fixed action-conditioned evaluation unsuitable for cross-model comparison. To address this, we employ multi-modal Agent Players to interact with world models toward specified long-horizon objectives. Building on this paradigm, we introduce PlayWorld, a benchmark providing 171 scenarios, each with a specified objective. To evaluate performance thoroughly, we assess models along four core dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution. In addition, we incorporate basic ability metrics for video quality and controllability. Experiments across nine state-of-the-art world models reveal that current models remain unreliable on long-horizon interactive objectives, particularly in maintaining spatial consistency and persistent state evolution. Code and data are available at https://github.com/kxding/PlayWorld.
Kaixin Ding, Xi Chen, Minghong Cai +9
The University of Hong Kong · Kling Team, Kuaishou Technology · The Chinese University of Hong Kong +1
Video world models enable interactive exploration of dynamic environments, yet struggle to respect prior observations over long horizons and across viewpoints. We present WorldCrafter, a video world model that learns a camera-queryable implicit 3D-aware memory for this purpose. The key insight is to let the requested viewpoint shape how multi-view evidence is compressed into the video generator's limited token budget. Trained jointly with the video generator, a memory encoder and pose-conditioned readout module integrate historical observations into a fixed set of target view-specific tokens before denoising, without explicit depth-based correspondences. By combining this memory with recent temporal context and few-step distillation, WorldCrafter enables streaming scene exploration from a single input image or text prompt. Experiments across static and dynamic scenes show substantial gains in long-horizon consistency and camera-control accuracy while preserving visual quality during minute-scale exploration.
Evaluating world models requires assessing both the quality of the worlds they generate and their consistency and responsiveness under exploration, interaction, and modification. We introduce HappyWorld-Bench, a comprehensive benchmark that evaluates whether generated worlds remain reliable as agents interact with them. Our design is built on a hierarchical capability framework of six world capabilities (W1-W6), from generative construction to unified world modeling, instantiated across three independent evaluation tracks: video world models, spatial world models, and embodied world models. HappyWorld-Bench comprises 1,138 video prompts, 300 spatial scenes, and 254 embodied test cases. Across all three tracks, we build and operate HappyWorld-Arena to organize human A/B comparisons and derive model-level Elo ratings, which complement newly designed automated metrics that capture behavioral correctness. We evaluate 14 video world models, 9 spatial systems, and 8 embodied candidates under this unified framework. Results reveal remaining reliability gaps across all three tracks: video models exhibit reduced consistency during extended rollouts and revisits, spatial models achieve at best 70.14% placement accuracy and 73.33% edit execution, and embodied models struggle to preserve state across multi-step actions and respond precisely to altered action conditions and physical rules. These findings highlight the need to evaluate world models not only by visual quality, but also by state consistency and the correctness of their responses to actions and interventions.
Zhiqi Bai, Junai Cai, Yixin Chen +33
Alibaba Token Hub, Alibaba Group · State Key Laboratory of General Artificial Intelligence, BIGAI · Tsinghua University +2