Video world models should maintain a world that keeps evolving as the viewpoint changes. Yet when the camera turns away and returns, current models often freeze the unobserved scene, lose objects, or change their appearance, because content outside the view has no representation that keeps changing. We present PanoEvo, which uses a Panoramic Evolver to evolve the world state in the latent space of a panoramic video diffusion model, where every viewing direction is present at every time step, so the scene evolves whether or not it is observed. Existing panoramic video models are not trained to evolve a scene from a single image or the objects within it, so we build PanoEvo-Data, an event-centric dataset from real 360° videos, and train the Panoramic Evolver coarse-to-fine as an image-to-panoramic-video model. Since each view covers only a small part of the low-resolution panorama, directly projected views are blurry and often show low-quality details or structural errors. A Perspective Renderer, built on a perspective video diffusion model, reads this state via latent-space projection and direction-restricted cross-attention to render sharp, high-quality views. PanoEvo achieves the best overall score among 23 publicly evaluated models on WRBench and the highest total score on STEVO-Bench. Code, models, and data will be released.
Figures & tables
Figure 2: Overview of PanoEvo. (a) E turns an image I0 and optional text y into an evolving panoramic latent W , the world state, from which R renders a video V along any trajectory C . (b) E is fine-tuned in two stages and writes observed cells back from zref . (c) R reads W via projected latent maps and frustum cross-attention, where each video token attends to panoramic tokens within ρ of its ray.
Figure 3: Construction of PanoEvo-Data. A shared quality gate feeds three branches, two for the stages of E and one for R .
Coarse
Event
Render
Clips
20,812
1,400
20,526 ( × 5 traj.)
Source videos
–
846
–
Resolution
ERP 512×1024
ERP 512×1024
native ( ≥ 3840 wide)
Text
label / VLM caption
detailed / brief / empty
label / empty
Annotation
rule-based
events, verified
trajectories
Used for
E , Completion
E , Evolution
R
Table 1: PanoEvo-Data. Three subsets from real 360 ∘ videos, split by source video and sharing no time windows.
Visible
Re-observed
Content
Spatial–
Model
Params
Cam Align.
Integ.
Spatial
State
Spatial
State
Avg.
Avg.
State Avg.
Source-video condition
InSpatio-World 14B [ 19 ]
14B
.980
.700
.769
.624
.729
.643
.741
.693
.691
ReCamMaster [ 3 ]
1.3B
.937
.623
.653
.493
.662
.609
.663
.608
.604
HyDRA [ 9 ]
1.3B
.855
.611
.587
.460
.468
.410
.565
.507
.481
Geometry-cache condition
Table 2: Results on WRBench. Baselines are grouped by viewpoint condition as in [ 29 ] . Params: backbone size (total for MoE); for PanoEvo, both DiTs with added modules. Content Avg. averages Integ. and the four spatial/state metrics, and Spatial–State Avg. only the latter. The other nine officially evaluated models are low-sample and listed in the supplement. Best and second best: red and blue .
Figure 4: Qualitative comparison. Views at t=20 (turning away), 38 (farthest), and 76 (returned), with the subject boxed.
Model
State Progress
Physical Plaus.
Coherence
Obs. Control
Action Control
Total
Video models
HunyuanVideo 1.5 [ 48 ]
4.1
42.1
59.1
31.2
81.0
217.5
Wan2.2 [ 40 ]
7.7
52.0
58.4
46.2
76.5
240.8
CogVideoX [ 51 ]
1.4
68.5
67.1
22.2
75.5
234.7
Camera-controlled models
Gen3C [ 34 ]
0.0
30.6
82.4
90.6
57.6
261.2
Table 3: Results on STEVO-Bench (%). Baseline scores are the official leaderboard values reported in ReMind [ 50 ] .
Cam
Visible
Re-observed
E
Align.
Integ.
Spat.
State
Spat.
State
Avg.
Δ Avg.
Zero-shot [ 20 ]
.924
.645
.677
.531
.687
.625
.682
–
+ Completion
.922
.661
.743
.599
.740
.654
.720
+.038
+ Evolution
.964
.704
.788
.663
.771
.683
.762
+.042
Table 4: Training stages of E on WRBench. All rows use direct projection to isolate E . Δ Avg.: change over the previous row.
Metric
w/o Renderer
PanoEvo
Δ
WRBench
Integ.
.704
.720
+.016
Visible spatial
.788
.799
+.011
Visible state
.663
.651
− .012
Re-observed state
.683
.646
− .037
Avg.
.762
.753
− .009
Quality
VBench (8-dim mean)
.818
.854
+.036
Table 5: Effect of the Perspective Renderer. Both rows use the same Panoramic Evolver output, and “w/o Renderer” projects the decoded panorama directly. Sharpness: Laplacian variance.
Figure 5: Qualitative ablation. Each row adds one component (columns: t=20 , 38, 76). Zero-shot, the woman walks past instead of setting her handbag down, the Completion stage has her squat as her bag vanishes, the Evolution stage completes the action out of view, and the Renderer sharpens it without altering the state.
Cam
Vis.
Re-observed
Variant
Align.
State
Spat.
State
Avg.
Imaging
Full
.950
.651
.754
.646
.753
.729
w/o read-out loss
.935
.647
.748
.643
.751
.719
w/o frustum attention
.931
.645
.742
.637
.745
.714
w/o projected latent
.902
.642
.732
.625
.729
.703
Table 6: Ablation of the Perspective Renderer components. Each variant removes one component, is trained for 13,500 steps, and is tested on the same E output. Removing frustum attention also removes the read-out loss. Imaging: VBench imaging quality.
Params
New trainable
TFLOPs
Peak memory
Panoramic Evolver E
2.40B
0.106B
7,752 (38,759)
10.5 GiB
Perspective Renderer R
1.80B
0.385B
27,820
8.8 GiB
PanoEvo
4.20B
0.491B
35,572 (66,579)
10.5 GiB
Table 7: Inference cost for one 77-frame view. Parameters and FLOPs are those of the diffusion transformers, excluding encoders and VAEs. FLOPs of E are amortized over the five camera settings rendered from one panorama in WRBench (unamortized in parentheses). Peak memory is that of diffusion sampling.