World models that forecast scene evolution by generating future video frames devote the bulk of their capacity to photometric details, yet the resulting predictions often remain geometrically inconsistent. We present VGGT-World, a geometry world model that side-steps video generation entirely and instead forecasts the temporal evolution of frozen geometry-foundation-model (GFM) features. Concretely, we repurpose the latent tokens of a frozen VGGT as the world state and train a lightweight temporal flow transformer to autoregressively predict their future trajectory. Two technical challenges arise in this high-dimensional (d=1024) feature space: (i) standard velocity-prediction flow matching collapses, and (ii) autoregressive rollout suffers from compounding exposure bias. We address the first with a clean-target (z-prediction) parameterization that yields a substantially higher signal-to-noise ratio, and the second with a two-stage latent flow-forcing curriculum that progressively conditions the model on its own partially denoised rollouts. Experiments on KITTI, Cityscapes, and TartanAir demonstrate that VGGT-World significantly outperforms the strongest baselines in depth forecasting while running 3.6-5 times faster with only 0.43B trainable parameters, establishing frozen GFM features as an effective and efficient predictive state for 3D world modeling.
Figures & tables
Figure 1: From video world models to geometry world models. Video world models predict future RGB in VAE latent space, coupling scene dynamics with appearance reconstruction. As a result, decoded predictions can remain geometrically invalid, with broken scene layout and mislocalized actors despite plausible video appearance. VGGT-World instead uses frozen geometry-foundation features as the latent state and models their temporal evolution directly, yielding a VAE-free, lightweight, and geometry-consistent alternative for future 3D forecasting.
Figure 2: Comparison of v -prediction and z -prediction in frozen VGGT latent space. Left: z -prediction consistently achieves substantially higher signal-to-noise ratio than v -prediction in the layer-4 latent space. Middle: A target VGGT feature map and its corresponding RGB frame. Right: PCA visualizations of predicted latents during training. v -prediction remains noisy even after prolonged training, whereas z -prediction progressively recovers structured latent patterns aligned with the target VGGT feature.
Method
2011_09_26_drive_0002_sync
2011_10_03_drive_0047_sync
Mean (Dataset Avg.)
AbsRel ( ↓ )
δ1 ( ↑ )
AbsRel ( ↓ )
δ1 ( ↑ )
AbsRel ( ↓ )
δ1 ( ↑ )
Short
Mid
Short
Mid
Short
Mid
Short
Mid
Short
Mid
Short
Mid
Cosmos-4B [ 1 ]
0.138
0.148
84.0
81.4
0.155
0.156
77.5
75.2
0.155
0.165
77.1
74.1
Cosmos-12B [ 1 ]
0.131
0.139
84.5
79.7
0.149
0.176
77.6
75.9
0.154
0.185
77.5
71.5
Copy Last (VGGT [ 50 ] )
0.112
0.125
89.6
82.6
0.140
0.153
84.4
79.6
0.137
0.144
83.7
78.5
Aether [ 44 ]
0.074
0.102
94.2
88.0
0.113
0.111
87.4
85.2
0.096
0.124
87.9
84.7
Table 1: Depth forecasting on KITTI. We evaluate representative world-model baselines on future scene depth prediction, reporting AbsRel ( ↓ ) and δ1 ( ↑ ) for short- and mid-term horizons.
Figure 3: The short- and mid-term depth forecasting results on KITTI. “Upper Bound” denotes the depth obtained by feeding the real future images into VGGT. “DINO-Foresight” predicts depth in the scale of Depth Anything [ 59 ] , leading to a different visualization appearance compared to others.
Method
AbsRel ( ↓ )
δ1 ( ↑ )
Short
Mid
Short
Mid
Cosmos-4B [ 1 ]
0.189
0.247
76.3
70.5
Cosmos-12B [ 1 ]
0.181
0.241
78.2
72.4
Copy-Last (VGGT [ 50 ] )
0.154
0.212
84.1
77.8
VISTA ft [ 15 ]
0.124
0.153
86.4
82.8
DINO-Foresight [ 23 ]
0.114
0.136
88.6
85.4
Table 2: Depth forecasting on Citys- capes with AbsRel and depth accuracy δ1 .
Method
Condition
1-view
2-views
Text-free
Pose-free
Acc. ↓
Comp. ↓
CD ↓
Acc. ↓
Comp. ↓
CD ↓
Aether [ 44 ]
✓
✗
3.1547
4.5366
3.8457
2.7745
3.4420
3.1082
WVD [ 61 ]
✓
✗
4.3944
3.0660
3.7302
4.3794
2.5268
3.4531
Gen3R [ 19 ]
✗
✗
3.0250
2.5367
2.7809
2.2825
1.6643
1.9734
Gen3R [ 19 ]
✓
✗
3.4579
4.7384
4.0982
2.6029
3.1154
2.8591
Gen3R [ 19 ]
✓
✓
3.6427
7.8528
5.7478
2.7441
6.1423
4.4432
Table 3: Geometry generation on TartanAir with different conditioning (text, pose) under 1-view and 2-view settings.
Figure 4: Point cloud forecasting on TartanAir . Gen3R produces structurally disorganized geometry on walls and rooftops, whereas our method preserves coherent structure and yields more accurate predictions.
Figure 8
Figure 6: Qualitative study of camera parameter prediction on KITTI and TartanAir.
Figure 7: Efficiency comparison with VideoGen world models. (a) Inference latency per generated frame. (b) Trainable vs. frozen parameter footprint.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: The short- and mid-term depth forecasting results on Cityscapes. “Upper Bound” denotes the depth obtained by feeding the real future images into VGGT. “DINO-Foresight” predicts depth in the scale of Depth Anything [ 59 ] , leading to a different visual appearance from the other methods.
Figure 9: Point cloud forecasting on TartanAir. Gen3R exhibits noticeable artifacts in structurally homogeneous regions such as walls and windows.
Figure 10: Single-frame point cloud forecasting results on TartanAir. The gray regions in each point cloud indicate the ground truth.
Metric
KITTI
TartanAir
ATE ↓ (m)
0.4157
0.1464
RTE ↓ (m)
1.1700
0.1821
RRE ↓ ( ∘ )
0.5711
1.7350
Appendix
Table 5: Quantitative study of camera parameter prediction on KITTI and TartanAir.
Modeling 3D scene geometry and its evolution over time is essential for autonomous driving and robotics. A common paradigm is to use world models to predict future images or latent representations of the environment and subsequently recover geometry from these predictions. However, this paradigm does not explicitly model geometric structure and typically relies on recursive rollouts to reach longer prediction horizons, leading to error accumulation and increasing computational cost. To address these limitations, we present GeoWM, a geometry world model that directly forecasts future scene geometry at specified future horizons without recursive rollout. The key idea is to leverage a geometry foundation model to transform observed RGB frames into a geometric history, which conditions a flow-matching transformer to predict the scene geometry at a specified future horizon. We further show that a lightweight camera-motion predictor can accurately estimate the future viewpoint, and that projecting the observed geometry into the predicted viewpoint provides an effective geometric prior for future geometry forecasting. Extensive experiments on four datasets spanning urban driving, aerial flight, and dynamic manipulation demonstrate that GeoWM outperforms the evaluated world models in forecasting depth, camera pose, and 3D scene geometry, while substantially reducing inference time at longer horizons.
Recent 3D world models increasingly use features from geometry foundation models (GFMs) as predictive world states, benefiting from their rich geometric priors. However, the view dependence of these features entangles scene dynamics with camera-induced variation, making the resulting state evolution harder to predict. We propose GeoState, a view-decoupled geometry representation for predictive 3D world modeling that separates scene-state evolution from camera-dependent observation. Specifically, a scene tokenizer aggregates multi-view GFM features into a view-decoupled scene state, which a geometry decoder queries using camera rays. An autoregressive world model then predicts future scene states, while camera motion is modeled separately. The predicted states can therefore be decoded under either predicted or externally specified cameras, while retaining a single context-derived scale throughout the rollout without per-frame scale alignment. Experiments show that GeoState outperforms prior 3D world models on Waymo, supports geometrically consistent cross-view querying, and generalizes zero-shot to KITTI and nuScenes. Project page: https://hongchang-liu.github.io/GeoState-project-page/
Hongchang Liu, Jian Liu, Yujuan Ding +5
Tongji University · Hong Kong Polytechnic University · Tencent +1
Previous works that leverage video models for image-to-3D scene generation often suffer from geometric distortions and blurry content. Using video generation models to implicitly maintain geometric consistency according to a single-frame input is ineffective. In this paper, we present a two-stage method, named GeoWorld, that renovates the image-to-3D scene generation pipeline by providing full-frame geometry features. The first-stage video generation model, followed by a multi-view geometry model, produces full-frame geometry features, which are then used as a mental draft of geometric conditions to aid the second-stage video-generation model. A geometric loss is proposed to impose real-world geometric constraints, and a geometry adaptation module is introduced to ensure the effective utilization of geometry features. Thanks to full-frame geometric modeling, the two smaller video models in our two-stage method can generate higher-fidelity 3D scenes than SOTA methods, while being even faster, e.g. 7.5× faster than Hunyuan-Voyager. Project page: https://peaes.github.io/GeoWorld.
Yuhao Wan, Lijuan Liu, Jingzhi Zhou +6
1VCIP & AAIS, Nankai University · 2ByteDance Inc. · 3Renmin University of China +2