Consistency-Preserving Diverse Video Generation

Xinshuang Liu; Runfa Blark Li; Truong Nguyen

cs.CVFeb 17, 2026

Consistency-Preserving Diverse Video Generation

Authors: Xinshuang Liu, Runfa Blark Li, Truong Nguyen

Organizations: University of California, San Diego, CA, USA

Abstract

Text-to-video generation is expensive, so only a few samples are typically produced per prompt. In this low-sample regime, maximizing the value of each batch requires high cross-video diversity. Recent methods improve diversity for image generation, but for videos they often degrade within-video temporal consistency and require costly backpropagation through a video decoder. We propose a joint-sampling framework for flow-matching video generators that improves batch diversity while preserving temporal consistency. Our approach applies diversity-driven updates and then removes only the components that would decrease a temporal-consistency objective. To avoid image-space gradients, we compute both objectives with lightweight latent-space models, avoiding video decoding and decoder backpropagation. Experiments on a state-of-the-art text-to-video flow-matching model show diversity close to strong joint-sampling baselines while substantially improving temporal consistency and color naturalness. Our code is available at https://github.com/XinshuangL/Diverse-Video.

Figures & tables

Fig. 1 : Joint video generation with enhanced cross-video diversity and preserved intra-video temporal consistency based on latent-space embedding and interpolation.

Fig. 2 : Illustration of training procedure for latent-space embedding and interpolation models.

Method	Vendi-v $\uparrow$	Vendi-f $\uparrow$	MSE $\downarrow$	CNI $\uparrow$
IID	1.55(1)	1.69(1)	0.0010(1)	0.67(1)
DPP	1.61(1)	1.81(1)	0.0028(2)	0.65(2)
Particle Guidance	1.61(1)	1.82(1)	0.0028(2)	0.65(2)
DiverseFlow	1.61(1)	1.81(2)	0.0029(2)	0.65(2)
Ours	1.60(1)	1.76(1)	0.0019(1)	0.68(1)

TABLE I : Video generation results. Metrics are reported as mean(uncertainty) with 95% confidence intervals; uncertainty aligns with the last digit of the mean.

Panel of Figure 4 — Fig. 3 : Model behavior across flow-matching steps: (a) Extrapolation of $\hat{x}_{1}=x_{t}+(1-t)v_{\theta}(x_{t},t)$ from intermediate latent states and decoding to video frames. (b)–(e) Embedding alignment losses for our latent embedding models, compared to using a latent-mean baseline. (f) Frame interpolation loss for $M_{c}$ , compared to simple baselines (previous frame, next frame, and mean of both).

Diversity-v	ConsisReg	Vendi-v $\uparrow$	Vendi-f $\uparrow$	MSE $\downarrow$	CNI $\uparrow$
✗	✗	1.61(1)	1.77(1)	0.0017(1)	0.68(1)
✗	✓	1.60(1)	1.76(1)	0.0016(1)	0.68(1)
✓	✗	1.61(1)	1.77(1)	0.0021(1)	0.68(1)
✓	✓	1.60(1)	1.76(1)	0.0019(1)	0.68(1)

TABLE II : Ablation results. Metrics are reported as mean(uncertainty) with 95% confidence intervals.

Video Generation

Video generation studies models that create and edit moving images from text, images, audio, and structured controls.

See all 350 papers

GeoFlow: Enforcing Implicit Geometric Consistency in Video Generation
May 18, 2026Jan Ackermann, Shengqu Cai, Boyang Deng +3Physical Consistency in Video Generation Video Diffusion Models
Stanford University · Google DeepMind
SpecLoR: Spectral Lookahead Rectification for Motion-Coherent Text-to-Video Generation
Jun 10, 2026Xu Zhang, Yu Lu, Ruijie Quan +3Physical Consistency in Video Generation T2V Generation
ReLER, College of Artificial Intelligence, Zhejiang University · Huawei Central Research Institute
Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation
Aug 11, 2026Yueting Zhu, Yuehao Song, Kaicheng Zhang +5Video Diffusion Models Long-Horizon Video Generation
Huazhong University of Science & Technology · Anyverse Dynamics · Horizon Robotics

May 18, 2026J/K move · Enter open · S save

GeoFlow: Enforcing Implicit Geometric Consistency in Video Generation

Jan Ackermann, Shengqu Cai, Boyang Deng, Zhengfei Kuang+2

Stanford University · Google DeepMind

Physical Consistency in Video Generation Video Diffusion Models RL for Video Generation Video Generation Geometric Consistency in Video Generation

Open paper AskPDF arXiv

Generating geometrically consistent videos remains an open challenge: text-to-video diffusion models trained on web-scale data treat geometry only implicitly, leading to object deformation, texture drift, and non-rigid backgrounds under camera motion. Existing solutions either improve consistency as a byproduct, apply only to static scenes or realign the latent space of the model completely. We introduce a geometry-consistency reward that directly measures whether motion in a generated video is compatible with a coherent scene. Our key insight is that in physically consistent videos, background motion should be explainable by rigid camera-induced flow, while independently moving objects should preserve appearance identity along motion trajectories. We operationalize this using optical flow, depth--pose predictions, and feature-based correspondence to separate rigid and dynamic regions and evaluate their respective consistency. Integrating this reward with reinforcement fine-tuning transforms geometric consistency from an emergent property into an explicit optimization objective for video generators. The approach is model agnostic and applies to diverse dynamic scenes containing both camera and object motion. Experiments show substantial reductions in temporal geometric artifacts over strong baselines while preserving perceptual quality. Code and model weights are published.

Abstract

Video Generation

Explore similar work

GeoFlow: Enforcing Implicit Geometric Consistency in Video Generation

SpecLoR: Spectral Lookahead Rectification for Motion-Coherent Text-to-Video Generation

Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation