World foundation models aim to simulate the evolution of the real world with physically plausible behavior. Unlike prior methods that handle spatial and temporal correlations separately, we propose RAYNOVA, a geometry-agonistic multiview world model for driving scenarios that employs a dual-causal autoregressive framework. It follows both scale-wise and temporal topological orders in the autoregressive process, and leverages global attention for unified 4D spatio-temporal reasoning. Different from existing works that impose strong 3D geometric priors, RAYNOVA constructs an isotropic spatio-temporal representation across views, frames, and scales based on relative Plücker-ray positional encoding, enabling robust generalization to diverse camera setups and ego motions. We further introduce a recurrent training paradigm to alleviate distribution drift in long-horizon video generation. RAYNOVA achieves state-of-the-art multi-view video generation results on nuScenes, while offering higher throughput and strong controllability under diverse input conditions, generalizing to novel views and camera configurations without explicit 3D scene representation. Our code will be released at https://raynova-ai.github.io/.
Figures & tables
Figure 1: Demonstrations of RayNova as Versatile World Foundation Model.
Figure 2: Overview of RayNova Framework. RayNova is composed of dual-casual (scale and time) blocks. The local scale attention and local cross attention works on each image indepedently, while the global causal attention works across multi-view and multiframe images enhanced with a unified ray-level relative position embedding for better spatio-temporal consistency.
Figure 3: Dual-Causality for Multi-View Video Generation. Green arrows represent the causal dependency, while the darkness indicates the topological order of autoregression (from light to dark ).
Method
Resolution
Metrics
FID ↓
FVD ↓
Throughput ↑
MagicDrive [ 13 ]
224 × 400
16.2
-
1.76
X-Drive [ 69 ]
224 × 400
16.0
-
0.83
DriveDreamer [ 64 ]
256 × 448
14.9
341
0.37
BEVWorld [ 78 ]
-
19.0
154
-
Panacea [ 68 ]
256 × 512
17.0
139
0.67
Table 1: Multi-view Video Generation Performance.
Table 2: Fidelity to Object Condition
Table 3: Fidelity to Map Condition
Method
Motion Planning ( L2↓ )
1s
2s
3s
Oracle
0.41
0.70
1.05
RayNova
0.42 (98%)
0.73 (96%)
1.09 (96%)
Table 4: Motion Cues in Generated Videos
Method
Shift 1m
Shift 2m
Shift 4m
FID ↓
FVD ↓
FID ↓
FVD ↓
FID ↓
FVD ↓
StreetGaussian [ 72 ]
32.12
153.45
43.24
256.91
67.44
429.98
OmniRe [ 9 ]
31.48
152.01
43.31
254.52
67.36
428.20
FreeVS [ 60 ]
51.26
431.99
62.04
497.37
77.14
556.14
RayNova
14.11
117.20
15.58
122.60
17.48
143.48
Table 5: Novel View Synthesis Performance with Camera Shifts.
S.-T. Module
Pos. Emb.
FID ↓
FVD ↓
✗
-
18.7
214
✓
Absolute
17.2
138
✓
Relative
17.2
124
Table 6: Ablation Study on Camera Ray.
Figure 4: Ablation Study on Scale Causality. Conditioning on all scales in history hurts the modeling of dynamics, while conditioning only on same scale is insufficient for temporal coherence.
Past-frame Condition
Size
FID ↓
FVD ↓
same scale
130M
20.7
151
all scales
130M
60.4
454
prefix scales
130M
17.2
124
prefix scales
2B
10.5
91
Table 7: Ablation Study on Scale Causality and Model Size.
Figure 5: Ablation Study on Model Size. Large scale model can bring significantly better visual quality.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
S.-T. Module
FID ↓
FVD ↓
decoupled
15.6
140
global
10.5
91
Appendix
Table 8: Ablation Study on Spatio-Temporal Module.
Random Errors
FID ↓
FVD ↓
✗
19.8
142
✓
17.2
124
Appendix
Table 9: Ablation Study on Random Errors in Training.
Figure 6: Zero-Shot to Unseen Waymo Open Dataset Camera Configuration.
Figure 7: Novel View Synthesis with Camera Rotation and Shifting.
Figure 8: Multi-View Video Generation with Eight Cameras.
Figure 9: Multi-View Video Generation with Six Cameras.