GVCC: Zero-Shot Video Compression via Codebook-Driven Stochastic Rectified Flow

Ziyue Zeng; Xun Su; Haoyuan Liu; Bingyu Lu; Yui Tatsumi; Hiroshi Watanabe

cs.CVMar 27, 2026

GVCC: Zero-Shot Video Compression via Codebook-Driven Stochastic Rectified Flow

Authors: Ziyue Zeng, Xun Su, Haoyuan Liu, Bingyu Lu, Yui Tatsumi, Hiroshi Watanabe

Organizations: Graduate School of Fundamental Science and Engineering, Waseda University, Tokyo, Japan

Abstract

At ultra-low bitrates, high-fidelity reconstruction requires sampling plausible videos from the posterior rather than regressing to oversmoothed conditional means. We propose Generative Video Codebook Codec (GVCC), a zero-shot framework in which a pretrained video generative model serves directly as the decoder, and the transmitted bitstream specifies its generation trajectory. Modern rectified-flow video models are typically sampled with deterministic ODE solvers, which leave no per-step stochastic channel for transmitting compressed information. GVCC addresses this by converting the deterministic flow sampler into an equivalent marginal-preserving stochastic process, so that information can be transmitted by encoding the per-step stochastic innovations. Unlike images, videos introduce longer temporal dependencies and more diverse conditioning modes. We instantiate GVCC in three practical modes: Text-to-Video (T2V) without a reference frame, autoregressive Image-to-Video (I2V) with tail latent correction, and First-Last-Frame-to-Video (FLF2V) with boundary-sharing Group of Pictures (GOP) chaining. On the seven-sequence UVG dataset, local atom-count sweeps characterize the rate--quality behavior of all three variants. We report full-dataset perceptual and fidelity metrics together with temporal diagnostics, without inferring matched-rate or global RD improvements from these limited local sweeps.

Figures & tables

Figure 1: Qualitative, perceptual, and user-preference comparison. Left: the Jockey example at the displayed sequence-specific operating points. Middle: full-UVG LPIPS-Alex; DCVC-RT (0.2913 at 0.00533 bpp) and GVCC-T2V (0.1329 at 0.00268 bpp) use our unified evaluation, while GNVC-VD ∗ (0.1433 at ${\sim}$ 0.00540 bpp) is read from its published curve. The displayed DCVC-RT and GVCC-T2V points differ by 54%. Right: the user study preferred GVCC-T2V to DCVC-RT and GNVC-VD in 97% and 88% of comparisons, respectively.

Figure 2: Overview of the GVCC framework. Top: shared pipeline—a frozen 3D VAE encodes the GOP into latent space, where GVCC compresses it into codebook noise indices; the decoder replays the same trajectory to reconstruct the video. Bottom: three conditioning strategies. (a) T2V: codebook only, no reference frame. (b) I2V: autoregressive GOP chaining with tail residual correction. (c) FLF2V: dual-anchor boundary sharing across consecutive GOPs.

Method	Setting	BPP	$E_{\mathrm{warp}}\downarrow$	tOF $\downarrow$	CLIP-F $\uparrow$
GVCC-T2V	—	0.0048	0.0102	0.565	0.9823
GVCC-FLF2V	—	0.0096	0.0103	0.624	0.9838
GVCC-I2V	capped	0.050	0.0117	1.258	0.9830
GLC-Video	q0	0.0095	0.0179	2.949	0.9743
DCVC-RT	q0	0.0045	0.0060	1.353	0.9812

Table 1: Compact temporal diagnostics on UVG Jockey at 720p. I2V’s tail-residual stream is capped to a 0.050-bpp total budget.

	GVCC-T2V			GVCC-I2V			GVCC-FLF2V			DCVC-RT
Seq.	PSNR $\uparrow$	LPIPS $\downarrow$	BPP	PSNR $\uparrow$	LPIPS $\downarrow$	BPP	PSNR $\uparrow$	LPIPS $\downarrow$	BPP	LPIPS $\downarrow$
Beau.	31.79	0.154	0.0048	32.90	0.109	0.0567	32.26	0.158	0.0075	0.442
Bosp.	30.32	0.098	0.0048	33.63	0.055	0.0546	32.72	0.086	0.0108	0.233
Honey.	30.78	0.052	0.0048	36.21	0.020	0.0543	34.73	0.033	0.0141	0.180
Jock.	31.28	0.090	0.0048	33.27	0.070	0.0534	33.09	0.080	0.0105	0.251
RSG	26.74	0.086	0.0048	29.48	0.053	0.0516	29.44	0.063	0.0168	0.282

Table 2: Comparison of GVCC variants on UVG 720p (3 GOPs per sequence, 33 frames/GOP). DCVC-RT at avg.

{\sim}

0.005 bpp—matched to GVCC-T2V’s bitrate—is included as a rate-matched perceptual reference. Best and second-best results among GVCC variants are highlighted.

Figure 3: Full-UVG 1080p ultra-low-rate curves. From left to right: PSNR, LPIPS-Alex, and DISTS versus measured BPP. Only measured points below

0.013

bpp are shown, and lines connect operating points only within the same method or GVCC variant. I2V is outside this shared rate region and is reported in Appendix 8 . Free-GVC is our reproduction, and GNVC-VD ∗ is digitized from its published curves.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Consecutive-frame MAE on HoneyBee , Beauty , and Jockey . Dashed vertical lines indicate GOP boundaries.

Variant	BPP	Frames / seams	Intra	Boundary	$\Delta$ / std.
T2V	0.00483	198 / 5	0.0880	0.0898	$+0.0018$ / 0.0048
FLF2V	0.009591	193 / 5	0.0780	0.0856	$+0.0076$ / 0.0056
I2V	0.050 (capped)	198 / 5	0.0712	0.1147	$+0.0435$ / 0.0217

Appendix

Table 3: Boundary LPIPS statistics on Jockey at 720p. I2V uses the same capped stream as Table 1 .

Figure 5: Per-frame LPIPS profiles used for Table 3 . Dashed lines mark GOP seam indices. The complete arrays, exact boundary indices, and evaluation code are included in the supplementary material.

Component	Version and configuration
HEVC / VVC	HM-16.25 / VTM-17.0, low-delay configuration with intra period 32.
DCVC-RT / DCVC-FM	Official CVPR 2025 / 2024 weights; RT uses serialized streams and FM uses entropy-model rate estimates.
GLC-Video	Official released code and weights.
Free-GVC	Our implementation of the published method using CogVideoX-2B.
GNVC-VD	Approximate readings from the published RD curves; not locally recomputed.
LPIPS / DISTS	Official LPIPS-Alex v0.1 and the original DISTS implementation of Ding et al.

Appendix

Table 4: Codec and metric implementations used in evaluation.

Method	Setting	BPP	VMAF $\uparrow$	Method	Setting	BPP	VMAF $\uparrow$
DCVC-RT	q42	0.01835	84.59	GLC-Video	q42	0.01256	44.90
VVC	QP32	0.01208	80.67	GVCC-T2V	—	0.00268	44.25
GVCC-FLF2V	—	0.00680	67.80	GLC-Video	q21	0.00899	42.87
GVCC-I2V	—	0.05230	67.59	VVC	QP45	0.00146	42.02
HEVC	QP37	0.00776	66.68	Free-GVC	$t^{*}=250$	0.00578	41.83
DCVC-RT	q21	0.00533	65.88	HEVC	QP45	0.00251	40.51

Appendix

Table 5: VMAF on UVG 1080p. BPP denotes the measured operating point; no rate interpolation is applied.

Method	Setting	BPP	FVD $\downarrow$	LOO range
GVCC-FLF2V	$M=80$	0.006895	40.41	40.26–49.63
GLC-Video	$q_{I}=3,q_{P}=63$	0.016982	41.64	41.25–51.43
DCVC-RT	$q_{I}=q_{P}=63$	0.061345	9.69	4.87–11.80

Appendix

Table 6: Descriptive FVD results on the fixed 233-clip UVG support. The leave-one-sequence-out (LOO) range is reported as a sensitivity check. Since the rates differ and the 400-dimensional covariance is rank deficient at this sample count, no cross-method efficiency or distribution-level superiority is inferred.

Method	Setting	BPP	FID $\downarrow$	KID $\downarrow$
GVCC-FLF2V-14B	Local	0.0070	14.92	0.00216
Free-GVC	$t^{*}=215$	0.0072	18.36	$0.00304\pm 0.00031$

Appendix

Table 7: Patch-level FID/KID on UVG 1080p under a unified local evaluation protocol.

Method	Setting	BPP	PSNR $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$
I2V	$M=60$	0.05163	32.72	0.0859	0.0501
I2V	$M=80$	0.05230	32.83	0.0840	0.0497
I2V	$M=100$	0.05297	32.86	0.0835	0.0496
DCVC-RT	q42	0.01835	36.94	0.2149	0.1420
DCVC-RT	q63	0.06289	39.64	0.1661	0.1071

Appendix

Table 8: Raw full-UVG 1080p I2V and DCVC-RT operating points. I2V BPP includes the initial reference frame.

Residual budget	BPP	PSNR $\uparrow$	LPIPS $\downarrow$	Status
Full	0.050	32.83	0.084	Stable
80%	0.041	30.12	0.108	Stable
50%	0.027	27.53	0.172	Stable
0%	${\sim}0.005$	—	—	Unstable from GOP 3

Appendix

Table 9: I2V tail-residual budget sweep on Jockey at 1080p.

Parameter	720p	1080p	Role
$M$	64	80	Atoms per step (bitrate knob)
$M_{\mathrm{tail}}$ (I2V)	128	128	Atoms for AR tail frames
$K$	16384	16384	Codebook size
$T$	20	20	Total sampling steps
$N$	3	3	Bit-free ODE tail steps
$g_{\mathrm{scale}}$	3.0	3.0	SDE diffusion coefficient

Appendix

Table 10: Default hyperparameters for GVCC (720p / 1080p).

Figure 6: Hyperparameter sweeps on UVG Beauty (T2V-1.3B, 720p). Blue: PSNR ( $\uparrow$ ). Red: LPIPS ( $\downarrow$ ). Purple: encoding time. Stars: selected defaults. (a) Atom count $M$ : quality saturates around $M{=}64$ while BPP grows linearly. (b) Codebook size $K$ : diminishing returns beyond 16384 at rapidly increasing cost. (c) Steps $T$ : marked degradation at $T{=}5$ and smaller changes beyond $T{=}20$ . (d) Diffusion scale $g_{\mathrm{scale}}$ : narrow sweet spot at 2.0–3.0; collapse at higher values. (e) GOP length: 17 frames insufficient; 33 and 49 comparable.

Test	PSNR $\uparrow$	LPIPS $\downarrow$	MS-SSIM $\uparrow$	$\Delta E$ $\downarrow$	Face ID $\uparrow$
Color	48.60	0.004	0.998	0.26	—
Text	38.87	0.002	0.998	0.26	—
Face	31.58	0.022	0.986	2.34	0.857
OOD	25.15	0.119	0.910	2.77	—

Appendix

Table 11: Faithfulness and robustness evaluation (GVCC-FLF2V, 720p, 3 GOPs).

\Delta E

: CIEDE2000 color difference (

<

1: imperceptible, 1–3: barely noticeable,

>

3: clearly visible). Face ID: ArcFace Deng et al. (2022) cosine similarity between original and reconstructed faces.

Test	VAE PSNR	FLF2V PSNR	$\Delta$ (GVCC loss)
Color	52.63	48.60	$-$ 4.03
Text	40.46	38.87	$-$ 1.59
Face	35.01	31.58	$-$ 3.43
OOD	30.86	25.15	$-$ 5.71

Appendix

Table 12: VAE upper bound vs. FLF2V compression (720p).

$M$	$K$	PSNR (dB)	BPP	kbps
8	256	22.50	0.00081	5.3
16	512	24.32	0.00183	11.9
16	1024	24.76	0.00201	13.1
32	2048	26.32	0.00438	28.5
32	4096	26.62	0.00474	30.9
64	4096	27.61	0.00948	61.7

Appendix

Table 13: T2V-1.3B R-D sweep on UVG 480p (7 seq.

\times

3 GOPs).

Figure 7: Rate–distortion curve of GVCC-T2V (1.3B, 480p, UVG average). Each point corresponds to a $(M,K)$ configuration from Table 13 . The curve spans from 5.3 kbps to 322.8 kbps with monotonically increasing quality.

Video Compression

Video compression reduces the bits needed to store or transmit moving images while balancing visual quality, computational cost, and downstream usefulness.

See all 73 papers

ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors
Jun 21, 2026Yixin Gao, Xiaohan Pan, Lin Liu +3Video Diffusion Models Neural Video Compression
University of Science and Technology of China · 2Huawei Inc.
Unified Generative Video Compression with Autoregressive and Bidirectional Video Diffusion
Oct 9, 2026Tianyu Zhang, Shuai Huo, Dengchao Jin +7Video Compression Generative Video Compression
University of Science and Technology of China · Kuaishou Technology
ResGVC: Residual Coding for High-fidelity Generative Video Compression
Oct 22, 2026Ding Ding, Daowen Li, Ying Chen +1Generative Video Compression Neural Video Compression
Alibaba Group

Jun 21, 2026J/K move · Enter open · S save

ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors

Yixin Gao, Xiaohan Pan, Lin Liu, Xin Li+2

University of Science and Technology of China · 2Huawei Inc.

Video Diffusion Models Neural Video Compression Video Compression Diffusion Models Generative Video Compression

Open paper AskPDF arXiv

Recent generative video compression methods leverage powerful generative priors to achieve perceptually pleasing reconstructions. However, most existing approaches require additional training to adapt generative models to produce realistic reconstructions from compact representations. In this paper, we propose ZeroGVC, a zero-shot generative video compression framework that leverages pretrained autoregressive diffusion priors for low-delay video reconstruction. ZeroGVC encodes the first frame of each group of pictures (GOP) with an image codec and represents subsequent P-frames through Codebook-Guided Autoregressive Latent Compression. This design is motivated by our observation that the compression scheme of denoising diffusion codebook models is effective in few-step consistency sampling. By selecting compact combinations of reproducible codebook noise vectors, ZeroGVC steers the latent denoising trajectory toward the target P-frame while allowing the decoder to reproduce the same trajectory in only a few denoising steps. In addition, we design an optional bidirectional reference mode that mitigates error propagation by leveraging the next I-frame context without introducing any additional bitrate overhead. Extensive experiments on standard video compression benchmarks demonstrate that ZeroGVC achieves superior perceptual reconstruction quality at ultra-low bitrates without any additional training.

Abstract

Video Compression

Explore similar work

ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors

Unified Generative Video Compression with Autoregressive and Bidirectional Video Diffusion

ResGVC: Residual Coding for High-fidelity Generative Video Compression