Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control. However, they implicitly assume that the world only evolves within the observer's field of view. Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime. In this work, we identify and formalize this overlooked limitation as the "out-of-sight dynamics" problem, which impedes video world models from representing a continuously evolving world. To address this issue, we propose LiveWorld, a novel framework that extends video world models to support persistent world evolution. Instead of treating the world as static observational memory, LiveWorld models a persistent global state composed of a static 3D background and dynamic entities that continue evolving even when unobserved. To maintain these unseen dynamics, LiveWorld introduces a monitor-based mechanism that autonomously simulates the temporal progression of active entities and synchronizes their evolved states upon revisiting, ensuring spatially coherent rendering. For evaluation, we further introduce LiveBench, a dedicated benchmark for the task of maintaining out-of-sight dynamics. Extensive experiments show that LiveWorld enables persistent event evolution and long-term scene consistency, bridging the gap between existing 2D observation-based memory and true 4D dynamic world simulation. The baseline and benchmark will be publicly available at https://zichengduan.github.io/LiveWorld/index.html.
Figures & tables
Figure 1: LiveWorld enables persistent out-of-sight dynamics. Instead of freezing unobserved regions, our framework explicitly decouples world evolution from observation rendering. We register stationary Monitors to autonomously fast-forward the temporal progression of active entities (e.g., the dog and the person) in the background. As the observer explores the scene along the target trajectory (green cameras), our state-aware renderer projects the continuously evolved world states to synthesize the final observation. This ensures that dynamic events progress naturally, accurately reflecting the elapsed time even when entities are completely out of the observer’s view.
Figure 2: World State Formulation. We approximate the intractable 4D world state Wt by decoupling it into two trackable representations: a temporally-invariant static 3D environment Mstatic via T -axis projection, and 2D video sequences of dynamic entities Mdyn,t via Z -axis projection.
Figure 3: LiveWorld overview. Our system explicitly decouples world modeling into two processes. (1) Static Accumulation (Blue): Temporally-invariant backgrounds are fused into a static 3D point cloud via SLAM. (2) Dynamic Evolution (Green): Stationary monitors use the Evolution Engine Gθevo to fast-forward the out-of-sight progression of active entities, lifting them into 4D point clouds. (3) State-aware Rendering (Purple): Both representations are projected onto the target camera trajectory. This geometric projection, alongside appearance references, guides the renderer Gθrender to synthesize coherent observations reflecting the elapsed dynamics.
Figure 4: Given one or multiple preceding frames from the previous round, we first detect if the scene visited by the observer contains active dynamic entities, using off-the-shelf VLMs and segmentors. Following a positive detection, we further validate if the entity and scene are already registered by existing monitors.
Same Pose Revisit
Different Pose Revisit
PSNRbg↑
SSIMbg↑
LPIPSbg↓
CDfg↓
DINOfg↑
VQA-Acc ↑
DINOfg↑
VQA-Acc ↑
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
MG-2
16.321
16.131
0.512
0.502
0.565
0.629
6.631
7.429
0.335
0.198
7.737
5.012
0.230
0.122
5.111
4.132
GC-1
17.637
16.012
0.571
0.523
0.421
0.572
2.107
6.236
0.527
0.262
20.125
10.273
0.475
0.191
18.799
8.397
Spatia †
20.132
19.020
0.672
0.649
0.297
0.310
4.031
5.122
0.440
0.416
19.205
14.655
0.392
0.363
18.723
13.212
Ours
20.071
19.983
0.679
0.650
0.301
0.330
0.068
0.135
0.760
0.721
59.063
54.620
0.691
0.632
52.829
49.478
Table 1: Quantitative comparison on LiveBench. Background (bg) metrics measure spatial memory against the first scene frame, foreground (fg) metrics measure entity identity preservation, and VQA-Acc evaluates the success of out-of-sight event progression guided by the text prompt. Columns in blue highlight the second revisit performance. † denotes our implementation.
Figure 5: A comparison result with the latest state-of-the-art methods on LiveBench. With the camera view repeatedly moving rightwards and backwards, our methods stand out alone to successfully maintain long-horizon (260 frames) out-of-sight dynamics, while others fail. Different colors correspond to different evolving prompts of the event.
Method
Presence
Id. fg
Consistency
Event Succ.
Full Succ.
E1
E2
E1
E2
E1
E2
E1
E2
w/o Event Evo
40
39
32
31
20
15
2
3
0
Ours
92
70
80
61
67
53
42
35
26
Table 2: User study on late-appearing event revisit success rate, higher is better (%).
Figure 6: A demonstration of the late-appearing event revisiting. While we have the initial image containing the kid, we allow the renderer to generate the corgi following the text prompt. The monitor for the corgi is registered after the camera has overlap under the threshold with the monitor of the initial frame. The results showcase a perfect synchronization between the renderer and each monitor.
Method
Same Pose Revisit
Different Pose Revisit
PSNRbg↑
SSIMbg↑
LPIPSbg↓
CDfg↓
DINOfg↑
VQA-Acc ↑
DINOfg↑
VQA-Acc ↑
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
1st
2nd
w/o Event Evo.
20.005
18.995
0.671
0.647
0.299
0.315
4.215
5.431
0.425
0.401
18.512
13.985
0.380
0.350
18.105
12.855
w/o Spa. Mem.
17.512
16.895
0.550
0.531
0.495
0.550
5.820
6.915
0.395
0.285
12.350
8.510
0.315
0.185
10.125
6.850
w/o Ref. Frame
18.520
17.105
0.610
0.545
0.385
0.490
1.520
4.850
0.615
0.410
38.510
22.150
0.550
0.320
32.105
18.550
Full
20.071
18.983
0.679
0.650
0.301
0.330
0.068
0.135
0.760
0.721
59.063
54.620
0.691
0.632
52.829
49.478
Table 3: Ablation studies of our proposed components. We validate the necessity of the evolution engine, spatial memory, and reference frames. The column in blue demonstrates the performance of a second revisit .
Figure 7: Typical revisit cases in our proposed LiveBench. More can be seen here .
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Method
VQA-Acc (%) ↑
CLIPF↑
DINOfg↑
PSNRbg↑
R1
R4
R1
R4
R1
R4
R1
R4
Wan-I2V [ 43 ]
70.983
42.505
0.988
0.932
0.882
0.695
27.122
20.231
Ours ( Gθevo )
61.774
50.541
0.977
0.972
0.811
0.794
26.481
22.639
Appendix
Table 4: Evaluation of the Evolution Engine ( Gθevo ) over long horizons. We compare round 1 (R1) and round 4 (R4) generations to demonstrate the robustness of our monitor-driven local evolution in maintaining script alignment, temporal coherence, identity, and background stability. The blue columns highlight the R4 performance.
Figure 8: Demonsration of the ground truth frame of Mstatic and Mdyn . The state adapter (in Sec. 3.2 and Fig. 3 ) takes the channel-wise concatenation of the Mstatic and Mdyn latents as input during training.
How can a world model continuously observe regions beyond the actor's current view? Video world models simulate how an environment evolves from an agent's actions, yet remain actor-centric. Once an object leaves the actor's view, they lose direct evidence of its evolution, often failing to preserve its state and dynamics upon re-entry. To address this, we introduce World Observer, which decouples observing from acting by jointly generating a perspective actor for the agent-centric view with one or more panoramic observers that watch selected world regions. This allows objects that leave the actor's view to remain visually evolving in an observer, so their updated states are reflected when they re-enter. We ground the actor and observers by warping from a shared panoramic source for explicit geometric correspondence, and introduce an Observer Sink of high-resolution perspective references to restore fine appearance upon re-entry. Since the observers are decoupled from the actor, they can be placed freely across the scene, extended to multiple locations for broader coverage, and driven by control signals to steer out-of-view evolution. To evaluate out-of-view evolution, we further introduce world-space metrics and a benchmark spanning real and synthetic scenes. World Observer substantially improves out-of-view dynamics while remaining competitive in visual fidelity, camera control, and 3D adherence.
We present WorldDirector, a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouples semantic motion orchestration from visual generation. By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experimental results demonstrate that our method supports the synthesis of complex and extended events with unprecedented controllability and persistent dynamic object memory. Project Page: https://worlddirector.github.io/
Evolutions in the world, such as water pouring or ice melting, happen regardless of being observed. Video world models generate "worlds" via 2D frame observations. Can these generated "worlds" evolve regardless of observation? To probe this question, we design a benchmark to evaluate whether video world models can decouple state evolution from observation. Our benchmark, STEVO-Bench, applies observation control to evolving processes via instructions of occluder insertion, turning off the light, or specifying camera "lookaway" trajectories. By evaluating video models with and without camera control for a diverse set of naturally-occurring evolutions, we expose their limitations in decoupling state evolution from observation. STEVO-Bench proposes an evaluation protocol to automatically detect and disentangle failure modes of video world models across key aspects of natural state evolution. Analysis of STEVO-Bench results provide new insight into potential data and architecture bias of present-day video world models. Project website: https://glab-caltech.github.io/STEVOBench/. Blog: https://ziqi-ma.github.io/blog/2026/outofsight/