Generative inbetweening aims to synthesize temporally coherent intermediate frames between a start frame and a target frame. Although pretrained image-to-video diffusion models provide strong priors for large-motion inbetweening, they are not explicitly designed to reach a specified target frame. They may therefore generate realistic individual frames while drifting from the target or producing unstable motion under large displacements and non-rigid dynamics. We propose DSG-GI, a framework that adapts pretrained latent video diffusion models through Destination-Conditioned Adaptation and Semantic-Geometric Alignment. Destination-Conditioned Adaptation treats the target frame as the destination of the sequence and provides temporally progressive guidance along the generated trajectory. Semantic-Geometric Alignment aligns the token-wise spatio-temporal relational geometry of backbone features with that of a frozen video foundation encoder and applies a flow-matching-aware latent flow constraint to samples near the data endpoint of the probability path. Together, these components strengthen destination guidance and preserve coherent semantic-geometric motion structure. Extensive experiments on standard and large-motion inbetweening benchmarks demonstrate improvements in temporal stability, target-frame faithfulness, and perceptual quality. Our code will be made publicly available.
Figures & tables
Figure 1: Motivation for temporally progressive target-frame guidance. (a) Early frames should remain anchored to the start frame, while target-frame guidance should progressively increase at later temporal positions to steer the trajectory toward the destination. (b) EF-Net concentrates its temporal weights at the final latent position, whereas DSG-GI progressively increases guidance along the latent temporal axis.
Figure 2: (a) Standard I2V generation extrapolates from the start frame and may drift from the target frame. (b) Bidirectional sampling reduces target-frame drift by generating in both temporal directions, but may introduce a discontinuity between the trajectories. (c) Frame-wise methods infer intermediate conditions independently, potentially producing a trajectory that deviates from the ground-truth motion. (d) DSG-GI treats the target frame as the destination of the trajectory: DCA progressively strengthens target-frame guidance, while SGA regularizes the generated trajectory using the semantic-geometric structure of the ground-truth video.
Figure 3: Overview of DSG-GI. (a) DCA injects temporally progressive target-frame guidance into multiple DiT stages, while SGA regularizes intermediate generation through relation-level semantic alignment and flow-matching-aware latent flow consistency. (b) The DCA block encodes the target-frame condition, expands it along the temporal axis, and applies a zero-initialized projection.
Methods
FCVG
TGI-Bench
LPIPS ↓
FID ↓
FVD ↓
FVMD ↓
LPIPS ↓
FID ↓
FVD ↓
FVMD ↓
FILM (ECCV’22)
0.1540
25.02
472.8
8254.0
0.2293
26.39
568.3
15714.3
AMT (CVPR’23)
0.2050
43.18
529.9
8174.6
0.3006
41.28
669.2
14857.5
SPEED (ACM MM’26)
0.1501
25.00
488.0
5074.5
0.2371
26.99
587.0
9807.6
GI (ICLR’25)
0.2181
30.68
498.4
5542.7
0.2979
24.83
414.8
8637.6
FCVG (CVPR’25)
0.1831
24.09
392.0
5586.5
0.2799
22.22
365.1
8386.5
Table 1: Quantitative comparison on FCVG and TGI-Bench. The best and second-best results for each metric are highlighted in bold and underline , respectively. ↓ indicates lower is better.
Figure 4: Qualitative comparison with state-of-the-art video interpolation and generative inbetweening methods.
Method
Base Model Parameters
Video Size
Steps
Time (s)
GI
1.5B
49 × 1024 × 576
50
1707.38
FCVG
1.5B
49 × 1024 × 576
25
364.50
MPD
1.5B
49 × 1024 × 576
50
1037.24
EF-VI
5B
49 × 720 × 480
50
400.73
DSG-GI
1.3B
49 × 1024 × 576
50
180.10
Table 2: Comparison of base model parameters and inference time across video inbetweening methods.
DCA
SGA
LPIPS ↓
FID ↓
FVD ↓
FVMD ↓
Lrel
Lflow
0.1706
21.90
335.3
4572.7
✓
0.1726
22.19
327.6
4930.7
✓
✓
0.1635
21.15
324.4
4551.0
✓
✓
0.1623
21.17
326.2
4572.6
✓
✓
✓
0.1595
20.84
330.7
4437.8
Table 3: Quantitative ablation of DCA and the SGA losses on the FCVG test set.
Figure 5: Qualitative ablation of DCA and the SGA losses.
Method
LPIPS ↓
FID ↓
FVD ↓
FVMD ↓
LTRD
0.1724
22.73
370.8
6398.7
Lrel + V-JEPA2
0.1786
23.16
355.5
4688.8
Lrel + VideoMAEv2
0.1595
20.84
330.7
4437.8
Table 4: Quantitative comparison of semantic alignment variants.
Figure 6: Qualitative comparison of semantic alignment variants.
Figure 7: Examples of text prompts generated by Qwen3-VL-8B from start and end frames.
Figure 8: Failure cases of DSG-GI in generating intermediate frames between the start and end frames.
Figure 9: Additional qualitative comparisons with state-of-the-art video interpolation and generative inbetweening methods.
Figure 10: Examples of intermediate frames generated by DSG-GI between the start and end frames.
Generative inbetweening (GI) seeks to synthesize realistic intermediate frames between the first and last keyframes beyond mere interpolation. As sequences become sparser and motions larger, previous GI models struggle with inconsistent frames with unstable pacing and semantic misalignment. Since GI involves fixed endpoints and numerous plausible paths, this task requires additional guidance gained from the keyframes and text to specify the intended path. Thus, we give semantic and temporal guidance from the keyframes and text onto each intermediate frame through Keyframe-anchored Attention Bias. We also better enforce frame consistency with Rescaled Temporal RoPE, which allows self-attention to attend to keyframes more faithfully. TGI-Bench, the first benchmark specifically designed for text-conditioned GI evaluation, enables challenge-targeted evaluation to analyze GI models. Without additional training, our method achieves state-of-the-art frame consistency, semantic fidelity, and pace stability for both short and long sequences across diverse challenges.
Video diffusion models generate visually compelling short-term sequences but often struggle to preserve scene geometry and object structure over time. In this paper, we first demonstrate that current video attention cannot identify reliable correspondences across frames, with responses dispersed across multiple regions even after recognizable scene structure has emerged. We argue that reliable cross-frame associations are essential for information exchange as scene structure forms in video generation. Thus, we introduce Geometry Attention Distillation (GAD), which transfers correspondences from a frozen geometry model's attention to video attention. Extracted from clean videos, these correspondences link the same static surface or moving object across frames. Since structure formation begins in deeper layers at higher noise and shallower layers at lower noise, we apply attention distillation at a student layer selected according to the noise timestep. After training, video attention focuses more consistently on corresponding locations. Experiments across all three video backbones demonstrate improved long-range scene consistency and dynamic temporal consistency, with geometric gains across generation lengths. On Wan2.2 5B, GAD improves cross-frame matching accuracy by at least 30% over the Wan baseline on both long-range benchmarks. Notably, GAD achieves these gains without additional inference modules, offering a promising approach to efficient and consistent video-based world simulation. Project page: https://fantasy-amap.github.io/GAD/.
Junfeng Ni, Ziyu Zhu, Fan Jiang +7
Tsinghua University · AMAP, Alibaba Group · State Key Laboratory of General Artificial Intelligence, BIGAI
Generating geometrically consistent videos remains an open challenge: text-to-video diffusion models trained on web-scale data treat geometry only implicitly, leading to object deformation, texture drift, and non-rigid backgrounds under camera motion. Existing solutions either improve consistency as a byproduct, apply only to static scenes or realign the latent space of the model completely. We introduce a geometry-consistency reward that directly measures whether motion in a generated video is compatible with a coherent scene. Our key insight is that in physically consistent videos, background motion should be explainable by rigid camera-induced flow, while independently moving objects should preserve appearance identity along motion trajectories. We operationalize this using optical flow, depth--pose predictions, and feature-based correspondence to separate rigid and dynamic regions and evaluate their respective consistency. Integrating this reward with reinforcement fine-tuning transforms geometric consistency from an emergent property into an explicit optimization objective for video generators. The approach is model agnostic and applies to diverse dynamic scenes containing both camera and object motion. Experiments show substantial reductions in temporal geometric artifacts over strong baselines while preserving perceptual quality. Code and model weights are published.