Time-Correlated Video Bridge Matching

Viacheslav Vasilev; Arseny Ivanov; Nikita Gushchin; Maria Kovaleva; Alexander Korotin

cs.LGOct 14, 2025

Time-Correlated Video Bridge Matching

Authors: Viacheslav Vasilev, Arseny Ivanov, Nikita Gushchin, Maria Kovaleva, Alexander Korotin

Organizations: Kandinsky Lab, Moscow, Russia · Moscow Center for Advanced Studies, Moscow, Russia · AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia ·

Abstract

Bridge Matching (BM) learns a stochastic transport between two data distributions and is therefore a natural tool for data-to-data tasks such as video frame interpolation, image-to-video generation, and video super-resolution. Existing bridge models for video, however, use a reference process that treats the frames of a clip independently, so temporal structure enters only through the network or the choice of the bridge endpoints. We propose Time-Correlated Video Bridge Matching (TCVBM), which places temporal structure into the reference process itself: the frames of a clip are coupled by a linear SDE whose drift is a discrete Laplacian along the time axis, and known frames enter as boundary conditions. The transition and bridge distributions remain Gaussian and available in closed form, so training and sampling are as simple as in standard BM, which is recovered in the limit of zero coupling. A spectral analysis shows that the prior damps high temporal frequencies, injects temporally smooth noise, and relaxes towards a boundary-determined profile, such as the linear interpolation between key frames. In controlled comparisons with frame-independent bridges under identical architectures and training budgets, TCVBM improves most automatic metrics for frame interpolation (Moving MNIST, UCF-101, Vimeo-90K) and image-to-video generation (Moving MNIST, UCF-101), e.g., reducing FVD from 688 to 611 for frame interpolation on UCF-101, and human raters prefer its frame interpolations for temporal consistency in 49 $\%$ of comparisons versus 15 $\%$ for the baseline ( $p<10^{-17}$ ); for super-resolution, it performs on par with BM on automatic metrics.

Figures & tables

Figure 1: Bridge Matching (left) vs. TCVBM (right). Dots of the same color are frames of the same video. In BM, the reference bridge perturbs every frame independently, and frames interact only through the network. TCVBM treats a clip $\mathbf{X}_{0}$ as one sample: the drift $\mathbf{A}\mathbf{X}_{t}$ couples neighboring frames, the known frames enter through $\mathbf{b}$ , and generation runs from $\mathbf{X}_{T}$ to $\mathbf{X}_{0}$ by sampling the bridge of Proposition 2 .

Panel of Figure 2 — Figure 2: Generation results on the Moving MNIST dataset.

Method	Frame interpolation				Image-to-video generation				Video super-resolution
	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DDIM	33.61	0.105	15.80	0.760	77.72	0.294	10.88	0.603	334.70	0.514	17.31	0.613
DDPM	32.40	0.117	14.94	0.741	75.86	0.311	10.72	0.595	607.88	0.236	20.19	0.582
BM	34.32	0.079	17.10	0.794	49.32	0.271	10.63	0.579	53.52	0.020	22.64	0.954
TCVBM (ours)	30.54	0.077	17.28	0.813	44.96	0.258	10.75	0.591	59.49	0.020	22.67	0.970

Table 1: Results on the Moving MNIST dataset (mean over 5 seeds; standard deviations in Appendix E.1 ).

Method	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
BM	688.26	0.0583	34.51	0.9353
TCVBM (ours)	611.48	0.0588	35.72	0.9574

Table 2: Frame interpolation results on the UCF-101 test set for the model trained on the UCF-101 train set.

Method	UCF-101					Vimeo-90K
	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	FloLPIPS $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
RIFE ( Huang et al., 2022 )	35.06	0.9489	0.044	0.041	552	35.83	0.904	0.098
LDMVFI ( Danier et al., 2024 )	33.47	0.9509	0.036	0.035	369	35.26	0.909	0.094
BM	30.59	0.9134	0.052	0.059	678	31.51	0.829	0.113
TCVBM (ours)	35.63	0.9513	0.041	0.037	441	35.84	0.911	0.097

Table 3: Frame interpolation results on the UCF-101 test set and the Vimeo-90K septuplet test set for the models trained on the union of the UCF-101 and Vimeo-90K training sets. All baselines were retrained on the same data with their official implementations.

Method	FVD $\downarrow$	IS $\uparrow$	PIC $\uparrow$
I2VGen-XL † ( Zhang et al., 2023 )	571	–	0.5313
SparseCtrl † ( Guo et al., 2023a )	722	19	0.4818
FrameBridge (retrained) ( Wang et al., 2025b )	603	36	0.6857
TCVBM (ours)	467	59	0.7441

Table 4: Image-to-video generation on UCF-101 with the metrics of Wang et al. (2025b) . FrameBridge was retrained with the official code under the same Latte-S/2 backbone, data and budget as TCVBM (batch size 20,

256\times 256

); the values reported by Wang et al. (2025b) for Latte-S/2 under their original configuration (FVD 154 for FrameBridge, 122 with a neural prior, 171 for VDT-I2V) are not comparable to this budget. Rows marked with † are reported by Wang et al. (2025b) (I2VGen-XL and SparseCtrl zero-shot) for reference.

Method	Frame interpolation		Image-to-video generation
	Frame quality	Temporal consistency	Frame quality	Temporal consistency
Baseline	19%	15%	36%	30%
TCVBM (ours)	41%	49%	37.5%	33%
Both	40%	36%	26.5%	37%
$p$	$<10^{-7}$	$<10^{-17}$	0.77	0.49

Table 5: Human evaluation on 40 videos from the UCF-101 test set: share of votes (%) preferring each method or both (10 assessors, 400 votes per column). Baseline: BM (models of Table 2 ) for frame interpolation and FrameBridge for image-to-video generation.

p

: two-sided sign test of TCVBM vs. baseline votes, ties excluded.

Figure 3: Frame interpolation results on the UCF-101 test dataset, trained on the UCF-101 training set. TCVBM retains more structural information by linking to boundary frames, providing improved detail and dynamics.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Spectral view of the prior for $N=8$ , $\epsilon=1$ , $t=1$ . Left: contraction $e^{\alpha\lambda_{k}t}$ of temporal frequency $k$ of $\mathbf{X}_{0}$ in the prior mean. Middle: standard deviation $\sigma_{k}(t)$ of the injected noise per frequency; BM ( $\alpha\to 0$ ) corresponds to the dashed lines. Right: $-\mathbf{A}^{-1}$ , proportional to the stationary covariance between frames (the covariance of a discrete Brownian bridge).

Method	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DDIM	33.61 $\pm$ 5.80	0.105 $\pm$ 0.070	15.80 $\pm$ 0.12	0.760 $\pm$ 0.011
DDPM	32.40 $\pm$ 1.49	0.117 $\pm$ 0.009	14.94 $\pm$ 0.43	0.741 $\pm$ 0.024
BM	34.32 $\pm$ 0.39	0.079 $\pm$ 0.001	17.10 $\pm$ 0.39	0.794 $\pm$ 0.005
TCVBM (ours)	30.54 $\pm$ 4.03	0.077 $\pm$ 0.019	17.28 $\pm$ 0.46	0.813 $\pm$ 0.044

Appendix

Table 6: Frame interpolation quantitative results with standard deviation. The best values in each column are in bold, the second best are underlined.

Method	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DDIM	77.72 $\pm$ 24.11	0.294 $\pm$ 0.092	10.88 $\pm$ 0.51	0.603 $\pm$ 0.069
DDPM	75.86 $\pm$ 13.49	0.311 $\pm$ 0.054	10.72 $\pm$ 0.27	0.595 $\pm$ 0.046
BM	49.32 $\pm$ 0.56	0.271 $\pm$ 0.005	10.63 $\pm$ 0.05	0.579 $\pm$ 0.004
TCVBM (ours)	44.96 $\pm$ 0.91	0.258 $\pm$ 0.002	10.75 $\pm$ 0.03	0.591 $\pm$ 0.001

Appendix

Table 7: Image-to-video generation quantitative results with standard deviation. The best values in each column are in bold, the second best are underlined.

Method	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
DDIM	334.70 $\pm$ 5.18	0.514 $\pm$ 0.006	17.31 $\pm$ 0.10	0.613 $\pm$ 0.016
DDPM	607.88 $\pm$ 6.28	0.236 $\pm$ 0.001	20.19 $\pm$ 0.05	0.582 $\pm$ 0.004
BM	53.52 $\pm$ 20.68	0.020 $\pm$ 0.005	22.64 $\pm$ 1.04	0.954 $\pm$ 0.012
TCVBM (ours)	59.49 $\pm$ 23.15	0.020 $\pm$ 0.004	22.67 $\pm$ 1.42	0.970 $\pm$ 0.011

Appendix

Table 8: Video super-resolution quantitative results with standard deviation. The best values in each column are in bold, the second best are underlined.

Method	Frame interpolation		Image-to-video generation		Video super-resolution
	Frame quality	Temporal consistency	Frame quality	Temporal consistency	Frame quality	Temporal consistency
BM	17%	20.5%	28%	18.5%	11.5%	13.5%
TCVBM (ours)	36%	52.5%	51%	59.5%	19%	30.5%
Both	47%	27%	21%	22%	69.5%	56%

Appendix

Table 9: Human evaluation on the Moving MNIST dataset: share of votes (%) preferring each method or both.

Initialization method	Method	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
Linear interpolation	BM	34.80	0.109	15.44	0.756
	TCVBM (ours)	31.94	0.092	16.28	0.782
Gaussian noise from $\mathcal{N}(\textbf{0},\mathbf{I})$	BM	34.32	0.079	17.10	0.794
	TCVBM (ours)	30.54	0.077	17.28	0.813

Appendix

Table 10: Analysis of the impact of initialization of input video data for bridge-based methods in the task of frame interpolation. Using noise from a normal distribution shows a clear advantage. The best values in each column are in bold, the second best are underlined.

Initialization method	Method	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
Static video	BM	49.32	0.271	10.63	0.579
	TCVBM (ours)	44.96	0.258	10.75	0.591
Gaussian noise from $\mathcal{N}(\textbf{0},\mathbf{I})$	BM	52.57	0.287	10.61	0.568
	TCVBM (ours)	48.61	0.263	10.68	0.587

Appendix

Table 11: Comparison of two types of initialization of input data for image-to-video generation. The best values in each column are in bold, the second best are underlined.

Initialization method	Method	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
Low-resolution video	BM	60.69	0.014	24.888	0.972
	TCVBM (ours)	58.26	0.012	24.970	0.973
Low-resolution video	BM	60.56	0.022	24.892	0.973
concatenated with noise from $\mathcal{N}(\textbf{0},\mathbf{I})$	TCVBM (ours)	59.65	0.022	24.988	0.973

Appendix

Table 12: Comparison of two types of initialization of input data for video super-resolution. We perform this comparison for low-resolution

32\times 32

. The best values in each column are in bold, the second best are underlined.

$\epsilon$	$\alpha$	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
0.1	0.1	35.57	0.085	16.40	0.797
0.1	1	36.54	0.089	16.37	0.797
0.1	10	29.79	0.086	16.56	0.801
1	0.1	27.34	0.084	16.65	0.803
1	1	30.54	0.077	17.28	0.813
1	10	31.43	0.080	17.12	0.817

Appendix

Table 13: The results of TCVBM (ours) training with various hyperparameters

\epsilon

and

\alpha

for frame interpolation. The best values in each column are in bold, the second best are underlined.

$\epsilon$	$\alpha$	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
0.1	0.1	55.46	0.2670	10.80	0.587
0.1	1	51.62	0.2578	10.74	0.583
0.1	10	59.57	0.2571	10.67	0.585
1	0.1	51.80	0.2615	10.70	0.590
1	1	44.96	0.2575	10.75	0.591
1	10	51.27	0.2613	10.60	0.583

Appendix

Table 14: The results of TCVBM (ours) training with various hyperparameters

\epsilon

and

\alpha

for image-to-video generation. The best values in each column are in bold, the second best are underlined.

$\epsilon$	$\alpha$	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
0.1	0.1	59.49	0.020	22.67	0.970
0.1	1	60.41	0.023	22.35	0.968
1	0.1	63.23	0.029	21.00	0.957
1	1	65.02	0.032	20.12	0.949
10	0.1	68.46	0.033	20.08	0.921
10	1	70.75	0.039	19.28	0.906

Appendix

Table 15: The results of TCVBM (ours) training with various hyperparameters

\epsilon

and

\alpha

for video super-resolution. We perform this comparison for low-resolution

16\times 16

. The best values in each column are in bold, the second best are underlined.

$f(t)$	FVD $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$
$1-t$	49.569	0.115	13.938	0.752
$1-2t$	398.823	0.202	11.760	0.684
$1-0.5t$	427.942	0.226	12.063	0.620
$2\times(1-t)$	409.495	0.195	12.239	0.624
$0.5\times(1-t)$	269.899	0.182	12.297	0.676
$(1-t)^{2}$	43.651	0.112	13.916	0.751

Appendix

Table 16: The results of the selection of the function

f(t)

, which determines the inverse dependence of the values of the matrix

\mathbf{A}

on time

t

, for TCVBM (ours). The best values in each column are in bold, the second best are underlined.

Temporal Consistency in Video Generation

Temporal consistency in video generation studies how models preserve coherent motion, identities, scenes, and events across frames and long sequences.

See all 375 papers

DuoMatching: Joint-Marginal Distribution Matching for Few-Step Video Generation
Oct 2, 2026Jiahao Zhan, Yan Wang, Yongrui Ma +5Single-Step Video Generation Streaming Video Generation
MMLab, CUHK · ByteDance Inc. · HKUST +1
FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation
Mar 5, 2026Ganggui Ding, Hao Chen, Xiaogang XuVideo Frame Interpolation Video Diffusion Models
⋆Zhejiang University · † Chinese University of Hong Kong
Bridging Event Streams and DiT: Event-Guided Video Frame Interpolation
Aug 11, 2026Guixu Lin, Yuyang Yu, Xiang Ji +6Video Frame Interpolation Temporal Consistency in Video Generation
The University of Tokyo · South China University of Technology · Wuhan University +1

Oct 2, 2026J/K move · Enter open · S save

DuoMatching: Joint-Marginal Distribution Matching for Few-Step Video Generation

Jiahao Zhan, Yan Wang, Yongrui Ma, Qunliang Xing+4

MMLab, CUHK · ByteDance Inc. · HKUST+1

Single-Step Video Generation Streaming Video Generation Distribution Matching Distillation Video Diffusion Models

Open paper AskPDF arXiv

Streaming video generation has benefited from distribution matching distillation (DMD), which matches the joint distribution of video frames to a video teacher's approximation of the real video distribution. Although this joint matching mitigates drift during autoregressive rollouts, limitations remain in visual quality and semantic alignment. To address these limitations, we propose DuoMatching, a distribution matching framework that approximates the real video distribution through a unified joint-marginal formulation. On top of existing joint matching formulations, the additional marginal matching objective provides dedicated frame-level supervision from an image generator, transferring complementary visual and semantic priors from it. To apply this frame-level supervision in video generation, we introduce LatentBridge to resolve the latent representation mismatch between the video student and the image teacher. Latent Variation Sampling further distributes such frame-level supervision across distinct temporal segments, reducing redundancy. Experiments demonstrate that DuoMatching improves visual quality, composition, and semantic alignment while largely preserving motion dynamics. Human evaluations show overall preference rates above 80% against all evaluated baselines. The project page is available at https://johnzhan2023.github.io/DuoMatching/.

Abstract

Temporal Consistency in Video Generation

Explore similar work

DuoMatching: Joint-Marginal Distribution Matching for Few-Step Video Generation

FC-VFI: Faithful and Consistent Video Frame Interpolation for High-FPS Slow Motion Video Generation

Bridging Event Streams and DiT: Event-Guided Video Frame Interpolation