Every Camera Grows Up: Camera-Height Regression in Interactive Video World Models
Organizations: State Key Laboratory for Novel Software Technology, Nanjing University
Abstract
Interactive video world models serve as simulators for embodied agents: from a starting image and an action trajectory, they generate the views the agent would see. The starting image fixes how high the camera is, for example 0.15 m for a floor robot or 2.2 m for an overhead camera, and a faithful simulator must keep this height as the camera moves. We show that six recent open-weight models fail to do so: as motion replaces the starting view with generated content, they move the camera toward human eye level, a failure we call camera-height regression. Our benchmark, EyeHeight, pairs six rendered elevations (0.15 m to 2.2 m) and four calibrated real-world rig heights with scale-free multi-view height estimation. All six models lose their initial height during a turn-and-walk rollout: initialized at 0.15 m, only 0-20% of clips end within of the starting height as inferred viewpoints converge to human heights of 1.1 m to 1.5 m, whereas ground-truth renders remain within a few percent. In Matrix-Game models, homography-tracked floor motion stays consistent with constant elevation during rotation even as newly outpainted room geometry yields a higher reconstructed viewpoint. Regression grows with rotation angle, begins while the start frame remains in context, and largely reverses in SANA-WM when a turn restores the initial view. We trace this pull to human-centric video priors in which floor-level views are nearly absent: in SANA-WM, textual height cues and dense floor-disparity maps do not prevent it, whereas classifier-free text guidance during the first half of denoising strengthens it. Combining unguided sampling with height-balanced fine-tuning recovers more retention but costs 5 MUSIQ points, a loss fixed during those same early steps. Viewpoint drift alone shifts the paths predicted by visual navigation policies.
Figures & tables
| Synthetic starts, turn | Forward | Real starts, turn | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | Control | Yaw | at | Kept | at 90 ° | at | , lowest | ||||
| Ground truth | – | 180 ° – 182 ° | 0.95 [0.90, 0.99] | – | 1.05 | 10/10 | 1.01 | 0.97 | 0.99 | – | – |
| SANA-WM [ 50 ] | poses | 167 ° – 209 ° | 0.18 [0.04, 0.32] | 1.32 | 5.47 | 0/21 | 4.49 ( 90 ° ) | 0.63 | 0.99 | 0.36 [0.27, 0.46] | 1.85 |
| Matrix-Game 2.0 [ 12 ] | keys, mouse | 171 ° – 174 ° | 0.64 [0.54, 0.75] | 1.51 | 2.47 | 2/18 | 1.02 ( 75 ° ) | 0.81 | 0.83 | 0.62 [0.45, 0.78] | 1.32 |
| Matrix-Game 3.0 [ 41 ] | keys, mouse | 176 ° – 205 ° | 0.67 [0.60, 0.73] | 1.14 | 2.03 | 4/20 | 1.07 ( 92 ° ) | 0.85 | 0.66 | 0.79 [0.70, 0.88] | 1.15 |
| LingBot-World [ 7 ] | poses | 97 ° – 111 ° | 0.64 [0.55, 0.74] | 1.14 | 2.47 | 2/21 | 1.58 ( 94 ° ) | 0.77 | 0.48 | 0.56 [0.45, 0.69] | 1.43 |
| Real starts, turn | Synth. | |||||
| Model | Guid. | Kept | MUSIQ | |||
| SANA-WM | 5 | 0.36 [0.27, 0.46] | 1.85 | 48% | 47.6 | 0.18 |
| none | 0.56 [0.47, 0.64] | 1.42 | 60% | 47.9 | 0.29 | |
| late | 0.51 [0.43, 0.58] | 1.63 | 60% | 48.4 | 0.23 | |
| + FT | 5 | 0.40 [0.31, 0.51] | 1.54 | 54% | 48.4 | 0.35 |
| none | 0.68 [0.58, 0.77] | 1.12 | 79% | 42.4 | 0.56 | |
Interactive World Models
Interactive world models predict and generate how environments change in response to actions, supporting simulation, planning, and control.
See all 131 papers