StrideWorld: Benchmarking Scene Rigidity and Action Consistency in Interactive Video World Model
Organizations: Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Beihang University · Tianjin University ·
Abstract
Interactive video world models turn user commands into the exploration of a generated scene, which requires the scene to keep its 3D structure as the camera moves and each action to keep a stable effect. Existing benchmarks mostly evaluate these properties against references fitted to the video under test, either a scene reconstructed from its own frames or a camera trajectory aligned to the command path with a free scale. Such references can absorb gradual scene deformation and changes in pace and route length, so these errors can pass undetected. We therefore introduce StrideWorld, a benchmark whose references are fixed before the motion they judge. It evaluates scene rigidity against the ground-truth geometry of the start view, and action consistency against the stride of each action, the camera motion of one command. The stride is measured on the model's first steps and then used to test the steps that follow, so each model is judged only against its own steps. Its measurements are validated on rendered videos with known camera motion, and its scores agree with human judgments. Across \nvideos{} videos generated by \nmodels{} open models from start views, all models but one deform their scenes as the camera moves, with offsets up to times those of a rigid rendering along the same camera path. Most models keep a steady pace when a single action is held. Along composed routes, several models often stall, and most completed routes end away from the positions predicted by each model's own strides. Only the two models that scale commands by the estimated scene depth bring most routes close to these positions. Fine-tuning two models on a few rendered clips removes most of their mid-route stalls. Code, data, and a leaderboard are publicly available at https://github.com/chuangwang123/strideworld.
Figures & tables
| Benchmark | Interf. | Reference | Pace | Seq. | Geom. | 3D test | Hum. |
|---|---|---|---|---|---|---|---|
| WorldScore [ 9 ] | C T | trajectory | – | multi-scene | recon. | ||
| WorldMark [ 56 ] | K C T | direction | 2D | open/loops | recon. | ||
| iWorld-Bench [ 11 ] | K C T | traj./dir. | – | open/loops | poses/2D | ||
| WBench [ 61 ] | K C T | trajectory | open/loops | recon. | |||
| WorldExam [ 58 ] | K C T | traj./dir. | – | open/loops | recon. | ||
| WorldRoamBench [ 55 ] | K C T | adapt./dir. | – | open/loops | recon./GT |
| Rigidity | Pace | Composition | |||||||||
| Model | Resp. (%) | Offset ( ∘ ) | vs. rigid | Ratio ( ) | Score | Stall (%) | End err. (str.) | Swap gap (str.) | Near (%) | Score | MUSIQ |
| Discrete keys | |||||||||||
| Matrix-Game 3.0 | 99.6 | 0.39 | 3.9 | 0.96 | 77.1 | 0.7 | 0.67 | 0.41 | 42.5 | 59.3 | 72.1 |
| Matrix-Game 2.0 | 84.3 | 0.40 | 3.3 | 0.96 | 64.8 | 7.4 | 1.45 | 1.14 | 24.7 | 49.6 | 68.5 |
| SANA-WM | 99.4 | 0.27 | 2.1 | 1.09 | 76.2 | 36.5 | 2.47 | 1.95 | 7.2 | 28.3 | 69.6 |
| DreamX-World | 99.0 | 0.38 | 3.4 | 2.11 | 46.4 | 12.3 | 5.22 | 4.92 | 0.5 | 24.2 | 74.3 |
Appendix figures & tables39 assets
Supplementary material from the paper’s appendix.
Appendix
| Route family | Route 1 | Route 2 |
|---|---|---|
| G1: cancel | W A D W | W W S W |
| G2: swap | W W D | D W W |
| G3: swap after a turn | YR W D | YR D W |
| G4: swap across turns | YR W D YL S D | YR D W YL D S |
| Model | Frames/ unit | FPS | Resolution | Control and preprocessing |
|---|---|---|---|---|
| Matrix-Game 3.0 ∗ | 4 | 17 | 1280 704 | Keyboard 1; mouse 0.04; center crop |
| Matrix-Game 2.0 ∗ | 4 | 16 | 640 352 | Keyboard 1; mouse 0.035; center crop |
| SANA-WM ∗ | 4 | 16 | 1280 704 | Native action sequence; center crop |
| DreamX-World ∗ | 4 | 16 | 1280 704 | Action tokens; translation 0.075 and yaw per frame interval; stretch resize |
| HY-World 1.5 ∗ | 4 | 24 | 832 480 | Native w/s/a/d/left/right control tokens; translation 0.08 and yaw per action unit; native crop |
| HY-World 2.0 | 2 | 16 | 832 480 | Camera pose; translation 0.025 and yaw per action unit; stretch resize |
| Model | Direction (%) | Extra rotation median (%) |
|---|---|---|
| Matrix-Game 3.0 | 100.0 | 2.9 |
| Matrix-Game 2.0 | 100.0 | 3.9 |
| SANA-WM | 100.0 | 3.7 |
| DreamX-World | 100.0 | 6.9 |
| HY-World 1.5 | 100.0 | 5.1 |
| HY-World 2.0 | 99.1 | 2.4 |
| Translation (% of depth) | Recovered (%) | Steps | Turn ( ∘ ) | Recovered (%) | Steps |
|---|---|---|---|---|---|
| 0–10 | 99.9 | 153,432 | 0–5 | 100.0 | 48,155 |
| 10–25 | 99.6 | 108,677 | 5–10 | 99.9 | 35,542 |
| 25–50 | 98.4 | 68,710 | 10–20 | 99.7 | 55,468 |
| 50–75 | 95.4 | 20,465 | 20–30 | 99.7 | 30,613 |
| 75–100 | 94.5 | 6,662 | 30–45 | 95.8 | 12,291 |
| 100–150 | 93.3 | 3,692 | 45–60 | 93.5 | 1,974 |
| Response per action | Response | Pace score | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Model | W | S | A | D | YL | YR | ||||
| Matrix-Game 3.0 | 100.0 | 100.0 | 98.7 | 98.9 | 100.0 | 100.0 | 99.8 | 99.5 | 77.1 | 77.0 |
| Matrix-Game 2.0 | 100.0 | 100.0 | 8.0 | 98.0 | 100.0 | 100.0 | 84.6 | 84.2 | 64.9 | 64.7 |
| SANA-WM | 100.0 | 100.0 | 97.8 | 98.7 | 100.0 | 100.0 | 99.5 | 98.9 | 76.3 | 76.0 |
| DreamX-World | 100.0 | 100.0 | 97.1 | 97.1 | 100.0 | 100.0 | 99.5 | 98.7 | 46.6 | 46.3 |
| HY-World 1.5 | 100.0 | 99.6 | 99.1 | 99.1 | 100.0 | 100.0 | 99.9 | 99.2 | 77.8 | 77.5 |
| Offset by image speed ( ∘ /frame) | Ratio to Lyra 2.0 | ||||||||
| Model | – | – | – | (%) | pooled | equal | median | ||
| Matrix-Game 3.0 | 0.40 | 0.40 | 0.40 | 0.38 | 0.36 | 24 | 3.6 | 4.1 | 3.5 |
| Matrix-Game 2.0 | 0.40 | 0.41 | 0.41 | 0.39 | 0.41 | 13 | 3.7 | 4.2 | 3.7 |
| SANA-WM | 0.24 | 0.28 | 0.29 | 0.30 | 0.28 | 19 | 2.5 | 2.6 | 2.5 |
| DreamX-World | 0.37 | 0.37 | 0.38 | 0.40 | 0.58 | 5 | 3.6 | 4.4 | 3.8 |
| HY-World 1.5 | 0.35 | 0.38 | 0.33 | 0.34 | 0.22 | 10 | 3.1 | 3.2 | 3.6 |
| Model | Step 1 | Steps 13–16 | 13–16 / 2–5 |
|---|---|---|---|
| Matrix-Game 3.0 | 1.02 | 0.98 | 0.99 |
| Matrix-Game 2.0 | 1.00 | 0.95 | 0.98 |
| SANA-WM | 0.84 | 1.00 | 0.93 |
| DreamX-World | 0.53 | 1.96 | 1.49 |
| HY-World 1.5 | 0.99 | 0.95 | 0.95 |
| HY-World 2.0 | 0.99 | 1.00 | 1.00 |
| Pace ratio | Near (%) | ||||
|---|---|---|---|---|---|
| Model | K4 | M4 | M2–5 | K4 | M4 |
| Matrix-Game 3.0 | 0.96 | 0.98 | 0.99 | 42.5 | 42.0 |
| Matrix-Game 2.0 | 0.96 | 0.97 | 0.99 | 24.7 | 21.9 |
| SANA-WM | 1.09 | 1.04 | 0.97 | 7.2 | 8.7 |
| DreamX-World | 2.11 | 1.62 | 1.25 | 0.5 | 2.5 |
| HY-World 1.5 | 0.97 | 0.98 | 0.98 | 3.8 | 4.8 |
| UE5 | ETH3D | |||
|---|---|---|---|---|
| Model | Pace | Comp. | Pace | Comp. |
| Matrix-Game 3.0 | 80.4 | 62.9 | 63.7 | 48.9 |
| Matrix-Game 2.0 | 66.9 | 53.0 | 56.3 | 40.0 |
| SANA-WM | 78.0 | 28.8 | 69.3 | 26.9 |
| DreamX-World | 46.0 | 24.0 | 48.2 | 24.6 |
| HY-World 1.5 | 74.1 | 17.2 | 91.9 | 35.3 |
| After a switch | Held | Block | Route | Switch steps (%) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Routes | New | Carried | 1st | 1st | 2nd | 4th | 1–4 | 9–12 | Misdir. | Stall |
| Matrix-Game 3.0 | 1063 | 0.90 | -0.00 [-0.01, 0.01] | 0.99 | 0.99 | 1.03 | 0.92 | 1.00 | 0.94 | 1 (-0) | 0 (-0) |
| Matrix-Game 2.0 | 1098 | 0.90 | 0.15 [0.13, 0.17] | 1.00 | 0.92 | 1.01 | 0.99 | 0.96 | 1.02 | 13 (+5) | 1 (+0) |
| SANA-WM | 1108 | 0.62 | 0.43 [0.39, 0.49] | 0.85 | 0.80 | 1.21 | 1.09 | 0.80 | 1.02 | 17 (+14) | 7 (+5) |
| DreamX-World | 1120 | 0.36 | 1.33 [1.25, 1.43] | 0.70 | 0.33 | 0.72 | 2.03 | 0.60 | 0.92 | 46 (+35) | 1 (+0) |
| HY-World 1.5 | 1120 | 0.46 | 0.10 [0.07, 0.13] | 0.98 | 0.55 | 0.82 | 0.81 | 0.60 | 0.70 | 23 (+17) | 11 (+4) |
| End error | Swap gap | ||||
|---|---|---|---|---|---|
| Model | observed | regen. | ratio | observed | expected |
| Matrix-Game 3.0 | 0.67 | 0.34 | 1.9 | 0.41 | 0.33 |
| Matrix-Game 2.0 | 1.45 | 0.34 | 4.3 | 1.14 | 0.36 |
| SANA-WM | 2.47 | 0.94 | 2.6 | 1.95 | 1.04 |
| DreamX-World | 5.22 | 1.83 | 2.8 | 4.92 | 1.68 |
| HY-World 1.5 | 2.42 | 1.77 | 1.4 | 1.56 | 1.65 |
| Model | Seed | Stall (%) | Near (%) | End err. (str.) | Swap gap (str.) | Composition score |
|---|---|---|---|---|---|---|
| Matrix-Game 3.0 | 0 | 0.8 | 50.4 | 0.60 | 0.41 | 60.3 |
| 1 | 0.0 | 59.6 | 0.61 | 0.38 | 63.8 | |
| 2 | 0.4 | 50.8 | 0.69 | 0.37 | 62.8 | |
| Matrix-Game 2.0 | 0 | 9.6 | 25.8 | 1.40 | 1.14 | 49.7 |
| 1 | 8.8 | 23.8 | 1.27 | 1.05 | 49.6 | |
| 2 | 7.9 | 21.2 | 1.46 | 1.16 | 49.5 |
| Spread | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | Med. | P90 | Err. | Ratio | Regen. | Corr. | Agree | ||
| Matrix-Game 3.0 | 221 | 0.30 | 0.71 | 0.59 | 2.10 | 0.34 | 0.79–0.82 | 59 | 66 |
| Matrix-Game 2.0 | 190 | 0.51 | 1.58 | 1.27 | 2.44 | 0.26 | 0.82–0.85 | 78 | 59 |
| SANA-WM § | 109 | 1.36 | 3.36 | 2.77 | 1.75 | 0.88 | 0.75–0.79 | 62 | 61 |
| DreamX-World § | 181 | 2.26 | 4.63 | 4.50 | 2.13 | 1.79 | 0.93–0.94 | 78 | 74 |
| HY-World 1.5 | 49 | 0.90 | 2.16 | 2.00 | 1.91 | 1.50 | 0.27–0.59 | 49 | 53 |
| Model | End err. | Scale-aligned | Fitted scale | Distance ratio |
|---|---|---|---|---|
| Matrix-Game 3.0 | 0.67 | 0.28 | 0.92 | 0.95 |
| Matrix-Game 2.0 | 1.45 | 0.79 | 0.96 | 1.02 |
| SANA-WM | 2.47 | 1.29 | 1.08 | 1.22 |
| DreamX-World | 5.22 | 2.36 | 1.89 | 2.31 |
| HY-World 1.5 | 2.42 | 1.16 | 0.68 | 0.79 |
| HY-World 2.0 | 0.30 | 0.14 | 1.00 | 1.01 |
| End err. | Composition score | ||||
|---|---|---|---|---|---|
| Model | Stride | Stride | |||
| Matrix-Game 3.0 | 0.67 | 0.67 | 0.65 | 59.3 | 60.4 |
| Matrix-Game 2.0 | 1.45 | 1.46 | 1.43 | 49.6 | 49.7 |
| SANA-WM | 2.47 | 2.42 | 2.39 | 28.3 | 28.6 |
| DreamX-World | 5.22 | 4.92 | 4.48 | 24.2 | 24.9 |
| HY-World 1.5 | 2.42 | 2.38 | 2.03 | 21.8 | 22.4 |
| Route ATE | Held ATE | Rated accurate | |||||
|---|---|---|---|---|---|---|---|
| Model | Resp. | Scale | Sim(3) | Scale | Sim(3) | Stalled | End err. |
| Matrix-Game 3.0 | 99.6 | 4.1 | 2.7 | 10.1 | 4.8 | 6/8 | 62/103 |
| Matrix-Game 2.0 | 84.3 | 9.5 | 5.6 | 19.7 | 10.8 | 38/83 | 78/309 |
| SANA-WM | 99.3 | 11.0 | 7.4 | 11.5 | 6.0 | 160/409 | 131/430 |
| DreamX-World | 99.5 | 13.5 | 9.4 | 14.9 | 7.0 | 21/138 | 135/849 |
| HY-World 1.5 | 99.8 | 14.4 | 8.6 | 7.7 | 3.4 | 125/605 | 49/322 |
| Condition | Tracking | Translation | Rotation |
|---|---|---|---|
| Unperturbed | 0.117 / 0.176 | 1.480 / 3.983 | 0.306 / 0.675 |
| Blur px | 0.113 / 0.181 | 1.580 / 3.188 | 0.335 / 0.647 |
| Blur px | 0.117 / 0.179 | 1.500 / 3.796 | 0.356 / 0.653 |
| H.264 CRF 30 | 0.119 / 0.187 | 1.461 / 3.204 | 0.390 / 0.692 |
| H.264 CRF 42 | 0.139 / 0.240 | 1.597 / 5.637 | 0.413 / 0.703 |
| Flicker | 0.116 / 0.176 | 1.475 / 3.971 | 0.305 / 0.676 |
| Per-frame mean | Median | Free depth | Joint fit | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Frames | Rigid | Gen. | Ratio | int. | Ratio | Focal | Ratio | int. | Ratio | int. |
| Matrix-Game 3.0 | 961 | 0.113 | 0.437 | 3.9 | [3.15, 4.72] | 3.9 | – | 4.2 | [2.80, 5.98] | 2.6 | [1.73, 3.44] |
| Matrix-Game 2.0 | 977 | 0.113 | 0.374 | 3.3 | [2.65, 4.11] | 4.4 | – | 4.6 | [2.91, 5.50] | 3.1 | [2.34, 3.58] |
| SANA-WM | 1062 | 0.097 | 0.208 | 2.1 | [1.85, 2.73] | 2.8 | – | 2.4 | [1.69, 2.91] | 1.6 | [1.43, 1.96] |
| DreamX-World | 832 | 0.112 | 0.382 | 3.4 | [2.87, 4.18] | 4.0 | – | 3.9 | [2.82, 5.30] | 2.5 | [1.97, 3.18] |
| HY-World 1.5 | 1017 | 0.054 | 0.391 | 7.3 | [5.59, 9.04] | 9.2 | – | 9.5 | [6.07, 11.77] | 4.3 | [3.32, 5.02] |
| Stride-based metrics | Existing control metrics | ||||||
| Change | Near (%) | Stall (%) | End err. (str.) | Swap gap (str.) | Dir. (%) | ATE scale (%) | ATE Sim(3) (%) |
| Ideal (M0) | 95.5 | 0.0 | 0.12 | 0.04 | 99.0 | 0.8 | 0.5 |
| M1, stride by step | 95.5 | 0.0 | 0.12 | 0.04 | 99.0 | 0.9 | 0.6 |
| M1, stride | 95.5 | 0.0 | 0.13 | 0.04 | 98.9 | 1.0 | 0.7 |
| M1, stride | 97.3 | 0.0 | 0.19 | 0.04 | 98.7 | 1.4 | 1.0 |
| M2, carry-over | 88.4 | 0.0 | 0.39 | 0.33 | 98.4 | 2.8 | 2.0 |
| Camera | |||||
| Change | Offset | Median | P90 | Anchors | VGGT |
| Rigid counterpart | 0.062 | 0.08 | 0.14 | 85.6 | 0.34 |
| Smooth deformation | |||||
| 0.109 | 0.22 | 0.36 | 85.8 | 0.41 | |
| 0.179 | 0.36 | 0.72 | 85.0 | 0.64 | |
| 0.325 | 0.64 | 1.49 | 83.6 | 1.44 | |
| Pace ratio | Near (%) | Stall (%) | End err. (str.) | |||||||||||||
| Model | P | P1 | P2 | Q | P | P1 | P2 | Q | P | P1 | P2 | Q | P | P1 | P2 | Q |
| Discrete keys | ||||||||||||||||
| Matrix-Game 3.0 | 0.97 | 0.97 | 0.98 | 0.97 | 42.5 | 42.0 | 40.5 | 43.0 | 0.7 | 0.4 | 1.1 | 0.6 | 0.67 | 0.68 | 0.67 | 0.66 |
| Matrix-Game 2.0 | 0.96 | 0.96 | 0.96 | 0.96 | 24.7 | 23.1 | 24.0 | 23.4 | 7.4 | 7.0 | 7.6 | 6.8 | 1.45 | 1.47 | 1.45 | 1.46 |
| SANA-WM | 1.10 | 1.10 | 1.10 | 1.10 | 7.2 | 7.9 | 7.7 | 7.9 | 36.5 | 36.0 | 36.0 | 36.5 | 2.47 | 2.45 | 2.54 | 2.44 |
| DreamX-World | 2.47 | 2.47 | 2.46 | 2.45 | 0.5 | 0.5 | 0.7 | 0.5 | 12.3 | 11.3 | 11.6 | 11.0 | 5.22 | 5.10 | 5.24 | 5.18 |
| Model | Response (%) | Pace score | Compl. (%) | Composition score | Repeat (%) | Views (%) | Pace score rank | Composition score rank |
|---|---|---|---|---|---|---|---|---|
| Matrix-Game 3.0 | [99.1, 100.0] | [73.0, 80.7] | [87.8, 98.6] | [54.7, 62.7] | [4.1, 5.7] | [19.6, 32.0] | 6–10 | 4–6 |
| Matrix-Game 2.0 | [83.2, 85.8] | [62.0, 67.4] | [87.8, 93.1] | [46.3, 53.0] | [13.2, 18.1] | [36.7, 47.1] | 11–12 | 6–7 |
| SANA-WM | [98.8, 99.9] | [73.8, 78.3] | [57.4, 67.4] | [25.2, 31.5] | [12.2, 16.9] | [32.0, 41.9] | 6–9 | 10–11 |
| DreamX-World | [98.5, 99.5] | [44.4, 48.3] | [84.4, 91.0] | [22.3, 26.2] | [22.3, 25.4] | [31.7, 46.4] | 13 | 11–13 |
| HY-World 1.5 | [99.1, 100.0] | [74.7, 80.8] | [38.4, 54.0] | [17.1, 27.0] | [40.6, 51.9] | [41.2, 54.4] | 5–8 | 11–13 |
| HY-World 2.0 | [98.7, 99.7] | [83.1, 89.7] | [98.1, 99.8] | [80.4, 85.4] | [1.7, 3.5] | [31.0, 46.3] | 1–3 | 1 |
| Model | Interface | Frames/step | FPS | Seconds/step | Pace ratio | Rigidity offset | Rigid-counterpart ratio |
|---|---|---|---|---|---|---|---|
| Matrix-Game 3.0 | Keyboard | 4 | 17 | 0.235 | 0.96 | 0.39 | 3.85 |
| Matrix-Game 2.0 | Keyboard | 4 | 16 | 0.250 | 0.96 | 0.40 | 3.30 |
| SANA-WM | Keyboard | 4 | 16 | 0.250 | 1.09 | 0.27 | 2.15 |
| DreamX-World | Keyboard | 4 | 16 | 0.250 | 2.11 | 0.38 | 3.40 |
| HY-World 1.5 | Keyboard | 4 | 24 | 0.167 | 0.97 | 0.33 | 7.30 |
| HY-World 2.0 | Camera pose | 2 | 16 | 0.125 | 1.00 | 0.35 | 4.01 |
| Near (%) | Stall (%) | End err. (str.) | |||||
|---|---|---|---|---|---|---|---|
| Model | All | Matched | All | Matched | All | Matched | |
| Matrix-Game 3.0 | 42.5 | 40.7 | 0.7 | 0.5 | 0.67 | 0.78 | 393 |
| Matrix-Game 2.0 | 24.7 | 29.5 | 7.4 | 7.2 | 1.45 | 1.40 | 488 |
| SANA-WM | 7.2 | 5.7 | 36.5 | 33.3 | 2.47 | 2.51 | 441 |
| DreamX-World | 0.5 | 0.6 | 12.3 | 10.8 | 5.22 | 4.22 | 351 |
| HY-World 1.5 | 3.8 | 3.2 | 54.0 | 52.5 | 2.42 | 2.44 | 558 |
| Repeat | Views | |||
|---|---|---|---|---|
| Model | Transl. | Turn | Transl. | Turn |
| Matrix-Game 3.0 | 4.8 | 14.4 | 25.3 | 13.6 |
| Matrix-Game 2.0 | 15.6 | 1.8 | 42.0 | 9.0 |
| SANA-WM | 14.4 | 3.1 | 36.8 | 5.3 |
| DreamX-World | 23.8 | 33.0 | 38.6 | 19.3 |
| HY-World 1.5 | 46.2 | 6.6 | 47.6 | 9.0 |
| Trajectory error | Per-step | ||||||
|---|---|---|---|---|---|---|---|
| Videos | StrideWorld | Direction | Scale fit | Sim(3) fit | Fixed | error | |
| All | 149 | 0.88 | 0.53 | 0.65 | 0.70 | 0.62 | 0.71 |
| Held actions | 70 | 0.86 | 0.40 | 0.63 | 0.64 | 0.69 | 0.54 |
| Routes | 79 | 0.84 | 0.72 | 0.82 | 0.84 | 0.78 | 0.84 |
| Completed routes | 62 | 0.71 | 0.73 | 0.80 | 0.85 | 0.77 | 0.82 |
| Swap pairs (Part B) | 139 | 0.83 | 0.71 | 0.68 | 0.67 | 0.71 | 0.66 |
| Part, level | Subset / score | Agreement | |
| A, video | All sampled videos (primary) | 160 | AUC 0.88 [0.82, 0.93] |
| A, video | single-action videos (Pace score) | 80 | AUC 0.87 [0.73, 0.98] |
| A, video | route videos (route score) | 80 | AUC 0.85 [0.76, 0.93] |
| A, video | completed routes only | 62 | AUC 0.71 [0.57, 0.84] |
| A, video | route completion alone (binary) | 80 | AUC 0.75 [0.67, 0.82] |
| A, video | gain of the route score over completion alone | 80 | 0.10 [0.04, 0.16] |
| HY-World 1.5 | SANA-WM | |||||
| Base | SFT | Rel. | Base | SFT | Rel. | |
| Stall (%) | 72.0 | 11.2 | 54.0 | 57.9 | 23.1 | 36.5 |
| Completed (%) | 27.5 | 88.8 | 46.0 | 41.1 | 76.8 | 62.4 |
| End err. (str.) | 4.07 | 2.33 | 2.42 | 3.66 | 1.46 | 2.47 |
| Swap gap (str.) | 3.60 | 2.48 | 1.56 | 1.65 | 0.71 | 1.95 |
| Near (%) | 0.7 | 11.4 | 3.8 | 1.9 | 24.1 | 7.2 |
| Construction | Route a | Route b | Steps |
|---|---|---|---|
| Turn–translate order | A 8 YR 8 | YR 8 A 8 | 16 |
| Segment order | A 3 D 3 S 3 W 3 | S 3 W 3 A 3 D 3 | 12 |
| Turn and counter-turn | YR 2 A 4 YL 2 | YL 2 A 4 YR 2 | 8 |
| Two-turn order | YR 3 A 3 YL 3 S 3 | YR 3 S 3 YL 3 A 3 | 12 |
| ETH3D (real photos) | UE5 (renders) | |||||||
|---|---|---|---|---|---|---|---|---|
| Metric | Released | N0 | SFT (H) | SFT (F) | Released | N0 | SFT (H) | SFT (F) |
| Response (%) | 100.0 | 99.5 | 100.0 | 100.0 | 99.6 | 95.0 | 98.7 | 99.5 |
| Pace score | 91.9 | 88.1 | 88.9 | 90.2 | 74.1 | 70.6 | 89.4 | 90.1 |
| Completed (%) | 61.1 | 43.1 | 87.2 | 83.3 | 40.7 | 22.1 | 87.7 | 90.6 |
| Route / Pair | 36.2 / 34.4 | 21.9 / 18.0 | 48.7 / 48.6 | 47.8 / 46.3 | 20.7 / 13.6 | 8.2 / 5.2 | 47.4 / 46.1 | 49.9 / 48.7 |
| Composition score | 35.3 | 20.0 | 48.6 | 47.1 | 17.2 | 6.7 | 46.8 | 49.3 |
| N0 | N0, | SFT | Perturbed | |
|---|---|---|---|---|
| Forward stride (rel. N0) | 1.00 | 0.97 | 0.52 | 0.53 |
| Turn stride ( ∘ ) | 1.25 | 1.27 | 1.75 | 1.57 |
| Response (%) | 96.1 | 95.6 | 100.0 | 100.0 |
| Pace score | 73.6 | 71.6 | 92.5 | 87.0 |
| Pace ratio | 1.01 | 1.01 | 1.03 | 1.00 |
| Composition score | 11.4 | 9.6 | 48.3 | 39.7 |
World Model Evaluation
World Model Evaluation measures whether learned simulators predict consistent, physically plausible, action-responsive futures that help agents plan and act.
See all 250 papers