Large-scale video diffusion models achieve impressive visual quality, yet often fail to preserve geometric consistency. Prior approaches improve consistency either by augmenting the generator with additional modules or applying geometry-aware alignment. However, architectural modifications can compromise the generalization of internet-scale pretrained models, while existing alignment methods are limited to static scenes and rely on RGB-space rewards that require repeated VAE decoding, incurring substantial compute overhead and failing to generalize to highly dynamic real-world scenes. To preserve the pretrained capacity while improving geometric consistency, we propose VGGRPO (Visual Geometry GRPO), a latent geometry-guided framework for geometry-aware video post-training. VGGRPO introduces a Latent Geometry Model (LGM) that stitches video diffusion latents to geometry foundation models, enabling direct decoding of scene geometry from the latent space. By constructing LGM from a geometry model with 4D reconstruction capability, VGGRPO naturally extends to dynamic scenes, overcoming the static-scene limitations of prior methods. Building on this, we perform latent-space Group Relative Policy Optimization with two complementary rewards: a camera motion smoothness reward that penalizes jittery trajectories, and a geometry reprojection consistency reward that enforces cross-view geometric coherence. Experiments on both static and dynamic benchmarks show that VGGRPO improves camera stability, geometry consistency, and overall quality while eliminating costly VAE decoding, making latent-space geometry-guided reinforcement an efficient and flexible approach to world-consistent video generation.
Figures & tables
Figure 1: World-consistent Video Generations with VGGRPO. We compare the baseline video diffusion model ( left , orange) with the VGGRPO-aligned model ( right , green). Each example depicts a challenging dynamic scene; we visualize representative keyframes from the generated video and reconstructed scene geometry from the inferred 4D scene representation. VGGRPO produces markedly more coherent scene structure and smoother camera motion over time, reducing geometric drift and structural artifacts in challenging dynamic settings.
Figure 2: Method Overview. (a) Latent Geometry Model. We connect latents from the diffusion VAE encoder to a geometry foundation model via a lightweight connector, yielding a Latent Geometry Model that predicts 4D scene geometry directly from video latents. (b) VGGRPO training. We perform latent-space GRPO using two complementary rewards, camera motion smoothness and geometry reprojection consistency, computed entirely in latent space with the latent geometry model. Together, these components align the video diffusion model toward 4D world-consistent generation on both static and dynamic scenes.
Method
Static
Dynamic
Sub. Cons. ↑
Bg. Cons. ↑
Aes. Qual. ↑
Img. Qual. ↑
Mot. Smooth. ↑
Dyn. Deg. ↑
VQ ↑
MQ ↑
Epi. ↓
VQ ↑
MQ ↑
Base Model: Wan2.1-1B
Base
-
-
0.133
-
-
0.7941
0.8930
0.5233
0.6178
0.9552
0.9231
SFT
45.26
46.84
0.137
40.00
39.00
0.8032
0.8896
0.5472
0.6256
0.9646
0.8795
Epipolar-DPO
54.21
55.79
0.098
45.50
43.00
0.8125
0.8916
0.5578
0.6461
0.9671
0.8816
VideoGPA
53.68
56.32
0.105
42.50
41.00
0.8068
0.8931
0.5562
0.6507
0.9650
0.8734
Table 1: Quantitative Comparison. The best and runner-up results are shown in bold and underlined , respectively. In both static- and dynamic-scene benchmarks, our method consistently improves geometric consistency and overall video quality over baselines across different base models, demonstrating the effectiveness of latent-space geometry-aware post-training.
Figure 3: Qualitative Comparison on Static and Dynamic Scenes. We show the first, middle, and last frames of video generations for a static-scene prompt ( left ) and a dynamic-scene prompt ( right ), with a representative segment of each prompt shown at the top. All baselines exhibit inconsistent artifacts, including geometric drift, temporal flicker, and unstable camera motion. In contrast, VGGRPO produces more coherent scene structure with smoother camera trajectories across frames.
Table 2: Additional Studies. (a) Impact of geometry foundation models: VGGRPO generalizes across latent geometry models constructed from different geometry foundation models. (b) Impact of reward components: rmotion and rgeo provide complementary gains, and their combination yields the best overall performance. (c) Test-time reward guidance: Differentiable latent-space guidance through the latent geometry model improves geometric consistency at test time with modest runtime overhead and no training. (d) Generalization: Beyond improving world consistency, VGGRPO transfers to standard VBench captions, indicating improved generation quality and demonstrating robustness in diverse scenarios. (e) Efficiency study: Latent-space reward computation reduces both runtime and peak GPU memory compared to RGB-based rewarding. VQ/MQ denote VideoReward win rates for Visual and Motion Quality, Epi. denotes the Sampson epipolar error on static scenes. Time is measured in seconds, and Mem denotes peak GPU memory (GB). Best results are in bold .
Figure 4: Reward Components Ablation. The reconstructed scene visualizes the estimated camera trajectory (red curve), with the first and last frames shown below each reconstruction. Compared to the Baseline, optimizing the motion reward rmotion stabilizes camera motion, but geometric artifacts persist (green circle). Adding the reprojection consistency reward ( rmotion+rgeo ) further improves scene geometry while preserving smooth camera motion, proving the complementarity of both components.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Analysis of the Latent Geometry Model. We compare the latent geometry model with the original RGB-based geometry model on 50 RealEstate10K test sequences under controlled perturbations applied in the video latent space. The top row reports camera pose estimation performance as the perturbation scale α increases, measured by relative rotation accuracy ( left , Racc@5), area under the accuracy curve ( middle , AUC@5), and relative translation accuracy ( right , Tacc@5). Our latent geometry model maintains stable performance across all noise levels, whereas the RGB-based geometry model degrades substantially as the perturbation grows. The bottom row shows a decoded RGB frame from the perturbed latents at different values of α . Even when perturbations produce only barely perceptible visual changes in RGB space, the RGB-based geometry model already degrades, reflecting the distribution gap when applied to generated content rather than real images. Our latent geometry model, trained directly on generated latents, avoids this gap and remains robust.
Model
Sub. Cons. ↑
Bg. Cons. ↑
Aes. Qual. ↑
Img. Qual. ↑
Mot. Smooth. ↑
Dyn. Deg. ↑
Baseline
0.9542
0.9528
0.5966
0.6733
0.9841
0.4237
SFT
0.9578
0.9556
0.5968
0.6304
0.9872
0.3988
Epipolar-DPO
0.9601
0.9564
0.5977
0.6353
0.9847
0.3325
VideoGPA
0.9605
0.9565
0.5973
0.6338
0.9835
0.3419
Ours
0.9644
0.9583
0.5991
0.6861
0.9895
0.3962
Appendix
Table 3: Generalization Performance on Standard VBench Captions. Compared with the base model, prior post-training methods provide only limited gains and even degrade certain metrics, such as Imaging Quality . In contrast, VGGRPO achieves the best overall performance on most metrics among the base model and prior post-training baselines. These results show that VGGRPO transfers effectively to the standard VBench caption set, improving generation quality while maintaining robust generalization across diverse scenarios. The best and second-best results are highlighted in bold and underlined , respectively.
Video diffusion models lack explicit geometric supervision during training, leading to inconsistency artifacts such as object deformation, spatial drift, and depth violations in generated videos. To address this limitation, we propose a geometry-based reward model that leverages pretrained geometric foundation models to evaluate multi-view consistency through cross-frame reprojection error. Unlike previous geometric metrics that measure inconsistency in pixel space, where pixel intensity may introduce additional noise, our approach conducts error computation in a pointwise fashion, yielding a more physically grounded and robust error metric. Furthermore, we introduce a geometry-aware sampling strategy that filters out low-texture and non-semantic regions, focusing evaluation on geometrically meaningful areas with reliable correspondences to improve robustness. We apply this reward model to align video diffusion models through two complementary pathways: post-training of a bidirectional model via SFT or Reinforcement Learning and inference-time optimization of a Causal Video Model (e.g., Streaming video generator) via test-time scaling with our reward as a path verifier. Experimental results validate the effectiveness of our design, demonstrating that our geometry-based reward provides superior robustness compared to other variants. By enabling efficient inference-time scaling, our method offers a practical solution for enhancing open-source video models without requiring extensive computational resources for retraining.
Tengjiao Yin, Jinglei Shi, Heng Guo +1
VCIP & TMCC & DISSec, College of Computer Science, Nankai University · Beijing University of Posts and Telecommunications · LIX, École Polytechnique, IP Paris
Generating geometrically consistent videos remains an open challenge: text-to-video diffusion models trained on web-scale data treat geometry only implicitly, leading to object deformation, texture drift, and non-rigid backgrounds under camera motion. Existing solutions either improve consistency as a byproduct, apply only to static scenes or realign the latent space of the model completely. We introduce a geometry-consistency reward that directly measures whether motion in a generated video is compatible with a coherent scene. Our key insight is that in physically consistent videos, background motion should be explainable by rigid camera-induced flow, while independently moving objects should preserve appearance identity along motion trajectories. We operationalize this using optical flow, depth--pose predictions, and feature-based correspondence to separate rigid and dynamic regions and evaluate their respective consistency. Integrating this reward with reinforcement fine-tuning transforms geometric consistency from an emergent property into an explicit optimization objective for video generators. The approach is model agnostic and applies to diverse dynamic scenes containing both camera and object motion. Experiments show substantial reductions in temporal geometric artifacts over strong baselines while preserving perceptual quality. Code and model weights are published.
Video diffusion models increasingly serve as priors for 4D generation, but the generated videos often lack geometric consistency. Recent approaches address this through post-training by designing geometric rewards, yet this can degrade texture and fine detail, requiring auxiliary appearance rewards and careful reward balancing. To address this challenge, we propose GeoFix, a unified framework that improves geometric consistency and visual quality with a single geometric reward, applicable to both training and inference-time reward alignment. Our reward measures cross-view depth consistency in static scene regions, targeting structural inconsistencies without directly constraining appearance or penalizing valid object motion. Crucially, depth agreement increases with camera field-of-view overlap even in real videos and 4D datasets, allowing a smaller camera movement to be mistaken for better geometry. We account for this effect by calibrating the reward against the overlap-dependent trend. GeoFix achieves a 39% improvement in WorldScore 3D consistency and a 4% increase in VBench Quality Score.