cs.CVOct 1, 2026
SaveWorld Observer: Joint Actor-Observer Generation for Persistent World Modeling
Organizations: KAIST AI
Abstract
How can a world model continuously observe regions beyond the actor's current view? Video world models simulate how an environment evolves from an agent's actions, yet remain actor-centric. Once an object leaves the actor's view, they lose direct evidence of its evolution, often failing to preserve its state and dynamics upon re-entry. To address this, we introduce World Observer, which decouples observing from acting by jointly generating a perspective actor for the agent-centric view with one or more panoramic observers that watch selected world regions. This allows objects that leave the actor's view to remain visually evolving in an observer, so their updated states are reflected when they re-enter. We ground the actor and observers by warping from a shared panoramic source for explicit geometric correspondence, and introduce an Observer Sink of high-resolution perspective references to restore fine appearance upon re-entry. Since the observers are decoupled from the actor, they can be placed freely across the scene, extended to multiple locations for broader coverage, and driven by control signals to steer out-of-view evolution. To evaluate out-of-view evolution, we further introduce world-space metrics and a benchmark spanning real and synthetic scenes. World Observer substantially improves out-of-view dynamics while remaining competitive in visual fidelity, camera control, and 3D adherence.
Figures & tables
Figure 1: Teaser. World Observer jointly generates panoramic observers with the actor. (a) Decoupled observer enables control over actor’s unseen regions. (b) Multiple observers can be placed flexibly to track occluded regions.
Figure 2: Motivation. When dynamic objects leave the actor’s view, existing models exhibit four failures: (a) frame-locked, (b) lost, (c) frozen, and (d) impostor state. These failures show the difficulty of maintaining and evolving states without direct observations during out-of-view intervals.
Figure 3: Dataset Overview. Real-world (top) and synthetic (bottom) examples with moving and two-fixed-observer configurations, respectively. Colored boxes show cross-view correspondences, while the rightmost diagrams show camera configurations and trajectories.
Figure 4: Model Overview. (a) World Observer jointly generates actor and panoramic observer streams with a shared DiT. (b) Observer Sink converts the high-resolution initial panorama into four perspective references for fine-grained appearance.
Figure 5: Visualization of self-attention. A query on a re-entering object attends to the observer region holding its out-of-view state, while the object fails to re-enter after out-of-view interval in non-joint model (Baseline).
Figure 6: Qualitative results. World Observer supports (a) state memory, (b) observer-prompted control, (c) long-horizon generation, and (d) multi-observer. Insets show the observer views.
Figure 7: Qualitative comparisons with other models. Red boxes mark target objects before exit and after re-entry. While existing models often lose, freeze, or inconsistently reconstruct them, World Observer better preserves object identity and state evolution across out-of-view intervals.
| Visual & Temporal Fidelity | Camera Control | 3D Adherence | Out-of-View Dynamics | ||||||||
| Method | #Param | FID | FVD | Img.Q. | RotErr | TransErr | mPSNR | mLPIPS | OOV-F | OOV-D | OOV-D |
| Panorama World Models | |||||||||||
| OmniRoam ( Liu et al., 2026 ) | 1.3B | 42.13 / 31.56 | 404.7 / 418.9 | 0.387 / 0.390 | - / - | 0.616 / 0.072 | 18.523 / 15.934 | 0.317 / 0.388 | 0.563 / 0.875 | 0.055 / 0.107 | 0.063 / 0.211 |
| PanoWorld ( Li et al., 2026b ) | 5B | 62.48 / 55.73 | 677.4 / 557.2 | 0.397 / 0.377 | - / - | 0.659 / 0.090 | 17.055 / 15.426 | 0.383 / 0.418 | 0.476 / 0.289 | 0.177 / 0.292 | 0.142 / 0.216 |
| Video World Models | |||||||||||
| HyDRA ( Chen et al., 2026b ) | 1.3B | 50.99 / 67.94 | 615.9 / 944.4 | 0.637 / 0.656 | 0.140 / 0.164 | 0.561 / 0.510 | 14.859 / 13.455 | 0.407 / 0.490 | 0.416 / 0.258 | 0.274 / 0.266 | 0.292 / 0.253 |
Table 1: Quantitative comparison with existing methods. Results are reported as Real-OOV-Bench / Synthetic-OOV-Bench. RotErr is (-) for panorama outputs cropped with the ground-truth rotation. World Observer (Multi) is evaluated only on Synthetic-OOV-Bench, as multi-observer conditions are scarce in the real world.
| Visual & Temporal Fidelity | Camera Control | 3D Adherence | Out-of-View Dynamics | |||||||
| Variant | FID | FVD | Img.Q. | RotErr | TransErr | mPSNR | mLPIPS | OOV-F | OOV-D | OOV-D |
| Backbone (raymap only) | 36.59 | 450.5 | 0.674 | 0.595 | 0.134 | 12.463 | 0.543 | 0.070 | 0.332 | 0.414 |
| w/o Observer generation | 24.82 | 243.1 | 0.659 | 0.057 | 0.073 | 16.997 | 0.337 | 0.674 | 0.387 | 0.430 |
| w/o Actor generation | 21.38 | 259.9 | 0.433 | - | 0.048 | 18.591 | 0.315 | 0.554 | 0.471 | 0.499 |
| w/o Observer Sink | 20.33 | 214.0 | 0.646 | 0.026 | 0.031 | 18.221 | 0.303 | 0.545 | 0.506 | 0.470 |
| World Observer (Single) | 19.65 | 196.2 | 0.641 | 0.022 | 0.029 | 18.893 | 0.276 | 0.580 | 0.531 | 0.526 |
Table 2: Ablation study on Synthetic-OOV-Bench. We evaluate visual and temporal fidelity, camera-following accuracy, 3D adherence, and out-of-view dynamics. For fairness, all variants share the full model’s conditioning. RotErr is (-) for w/o Actor generation, whose panoramas are cropped with the ground-truth rotation.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Out-of-view dynamics evaluation protocol. We track dynamic objects in 3D to evaluate their exit and re-entry (OOV-F) and motion consistency during out-of-view intervals (OOV-D and OOV-D ).
Figure 9: Comparison with existing Out-Of-View metrics. VLM-based metrics fail to identify frozen and impostor states as inconsistent, while our 3D displacement-based metrics quantitatively capture these failures.
Figure 10: Additional qualitative comparisons with other models. We provide additional comparisons of World Observer with existing world models on real and synthetic scenes involving out-of-view dynamics. For synthetic scenes, we additionally include results from World Observer (Multi). Red boxes highlight the target objects before exit and after re-entry. Across diverse objects, motions, and scenes, existing models exhibit out-of-view failures such as lost, frozen, and impostor states, whereas World Observer consistently preserves object identity and state evolution throughout the out-of-view interval.
Figure 11: Qualitative ablation results. We show the effect of removing each component on generation quality. Without the observer, the model fails to maintain out-of-view dynamics, causing the dynamic object to disappear when it re-enters the view. Without the actor, panorama-based generation followed by cropping leads to reduced visual fidelity. World Observer preserves evolving out-of-view dynamics while maintaining high-fidelity generation.
Figure 12: Qualitative ablation results in AR generation. We compare ground truth with variants without the observer and Observer Sink under a long camera trajectory with a full sweep. Without the observer, out-of-view object dynamics are not maintained, while removing the Observer Sink degrades fine appearance under large viewpoint changes. Our full model preserves both dynamic objects (red/yellow) and appearance details (green) upon re-entry.
Figure 13: Qualitative prompt ablation results. We compare (a) actor-concentrated, (b) unified, and (c) decoupled actor-observer prompting. Actor-concentrated prompting misplaces off-view events in the actor view, while unified prompting can hallucinate duplicate subjects in the observer. Decoupled prompting instead produces a consistent event across both streams.