ProGVC: Progressive-based Generative Video Compression via Auto-Regressive Context Modeling

Daowen Li; Ruixiao Dong; Kai Li; Ying Chen; Ding Ding; Li Li

cs.CVMar 18, 2026

ProGVC: Progressive-based Generative Video Compression via Auto-Regressive Context Modeling

Authors: Daowen Li, Ruixiao Dong, Kai Li, Ying Chen, Ding Ding, Li Li

Organizations: Alibaba Group, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center

Abstract

Perceptual video compression leverages generative priors to reconstruct realistic textures and motions at low bitrates. However, existing perceptual codecs often lack native support for variable bitrate and progressive delivery, and their generative modules are weakly coupled with entropy coding, limiting bitrate reduction. Inspired by the next-scale prediction in the Visual Auto-Regressive (VAR) models, we propose ProGVC, a Progressive-based Generative Video Compression framework that unifies progressive transmission, efficient entropy coding, and detail synthesis within a single codec. ProGVC encodes videos into hierarchical multi-scale residual token maps, enabling flexible rate adaptation by transmitting a coarse-to-fine subset of scales in a progressive manner. A Transformer-based multi-scale autoregressive context model estimates token probabilities, utilized both for efficient entropy coding of the transmitted tokens and for predicting truncated fine-scale tokens at the decoder to restore perceptual details. Extensive experiments demonstrate that as a new coding paradigm, ProGVC delivers promising perceptual compression performance at low bitrates while offering practical scalability at the same time.

Figures & tables

Figure 1: Overview of the proposed ProGVC video compression framework. We design a progressive generative video compression framework based on the visual autoregressive model. The left panel illustrates the output pyramid structure of the multi-scale residual quantization. Besides, the sparse attention mask used in the context model is shown on the right.

Figure 2: Overview of the multi-scale autoregressive context model . The transformer network predicts both intra scale and inter scale tokens in an autoregressive manner. The resulting probability distributions are subsequently utilized for entropy coding and for sampling discarded tokens during reconstruction.

Figure 3: Illustration of three attention mask designs: VAR-like attention (left), InfinityStar attention (middle), and our sparse attention (right). For visualization simplicity, we illustrate the masks using three spatial scales (1, 2, 3) and a temporal length of $T=2$ for inter scale tokens.

Panel of Figure 4 — Figure 4: Rate and perception/fidelity curves on Xiph, HEVC B and MCL-JCV datasets.

Dataset	Method	DISTS	LPIPS	NIQE	PSNR(dB)
Xiph	VTM-17.0 [ 4 ]	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000
	DCVC-FM [ 15 ]	92.0 / -0.0257	40.4 / -0.0228	71.3 / -0.2525	111.1 / -1.8623
	SEVC [ 3 ]	87.7 / -0.0179	19.4 / -0.0062	41.8 / -0.1331	51.5 / -0.9035
	DCVC-B [ 32 ]	63.2 / -0.0148	17.3 / -0.0065	94.7 / -0.2306	56.8 / -1.0048
	PLVC [ 46 ]	-40.3 / 0.0109	-83.7 / 0.0565	169.9 / -0.0902	825.9 / -5.5261
	ProGVC	-61.5 / 0.0300	3.5 / -0.0025	-62.2 / 0.2670	891.2 / -5.8437

Table 1: BD-rate

\downarrow

(%) / BD-metric

\uparrow

on the Xiph, HEVC B, and MCL-JCV datasets. Red and Blue indicate the best and the second-best performance in terms of BD-metric, respectively. “N/A” indicates that BD-rate cannot be calculated due to the lack of quality overlap.

Table 6

Panel of Figure 5 — Figure 5: Visual comparisons with baselines on the Kimono sequence in the Xiph dataset and the videoSRC07 sequence in the MCL-JCV dataset.

Model Variants	BD-rate $\downarrow$ (%) / BD-metric $\uparrow$
Model Variants	DISTS	LPIPS	PSNR
Base	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000
w/o context model (uniform prob.)	122.4 / -0.0259	119.6 / -0.0360	119.4 / -2.3437
w/o token generation	N/A / -0.1183	280.9 / -0.1273	189.9 / -3.8102

Table 4: Effect of multi-scale autoregressive context modeling . “uniform prob.” denotes utilizing uniform probability distribution.

Attn. Variants	BD-rate $\downarrow$ (%) / BD-metric $\uparrow$			Enc. T (s) $\downarrow$	Dec. T (s) $\downarrow$
Attn. Variants	DISTS	LPIPS	PSNR	Enc. T (s) $\downarrow$	Dec. T (s) $\downarrow$
Ours	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000	0.56	1.53
Self-only	6.8 / -0.0021	12.8 / -0.0062	10.9 / -0.3482	0.50	1.23
Full causal	-6.5 / 0.0021	-3.8 / 0.0019	0.5 / -0.0184	0.64	1.96

Table 5: Effect of three different temporal-aligned attention masks . The per-frame encoding and decoding times (Enc. T and Dec. T) are averaged over the lowest and highest bitrates.

Attn. Variants	BD-rate $\downarrow$ (%) / BD-metric $\uparrow$			Enc. T (s) $\downarrow$	Dec. T (s) $\downarrow$
Attn. Variants	DISTS	LPIPS	PSNR	Enc. T (s) $\downarrow$	Dec. T (s) $\downarrow$
Ours (largest scale)	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000	0.56	1.53
No intra reference	37.3 / -0.0171	41.7 / -0.0254	46.6 / -1.6698	0.53	1.46
Smallest scale	42.7 / -0.0191	40.4 / -0.0252	46.8 / -1.6803	0.53	1.46
Same resolution scale	11.7 / -0.0042	12.2 / -0.0064	14.4 / -0.5131	0.54	1.49

Table 6: Effect of different intra scales to the context modeling of inter scales . The per-frame encoding and decoding times (denoted as Enc. T and Dec. T) are averaged across the lowest and highest bitrate settings.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Rate and perception/fidelity curves on UVG datasets.

Dataset	Anchor (VTM-17.0 LD)	Method	DISTS	LPIPS	NIQE	PSNR (dB)
MCL-JCV	1080p, 96f	GNVC-VD [ 23 ]	-88.3 / 0.0882	N/A / 0.1470	- / -	N/A / -2.2600
	1080p, 96f	DiffVC [ 21 ]	N/A / 0.0555	N/A / 0.0946	N/A / 0.9003	N/A / -5.7767
	720p, 81f	ProGVC	-64.4 / 0.0416	-30.3 / 0.0269	N/A / 0.8172	533.5 / -5.5354
UVG	1080p, 96f	GNVC-VD [ 23 ]	-86.5 / 0.0670	N/A / 0.1263	- / -	120.5 / -1.8390
	1080p, 96f	DiffVC [ 21 ]	N/A / 0.0423	N/A / 0.0657	N/A / 0.9939	N/A / -5.4513
	720p, 81f	ProGVC	-63.9 / 0.0322	-35.5 / 0.0236	N/A / 0.7499	744.6 / -5.2541

Appendix

Table 7: Relative BD-rate

\downarrow

(%) / BD-metric

\uparrow

on MCL-JCV, UVG and HEVC B datasets using VTM-17.0 low-delay (LD) configuration as anchor. Diffusion-based results are taken from the published papers and are reported under 1080p/96-frame setting, whereas ProGVC is evaluated under 720p/81-frame setting. “N/A” indicates that BD-rate cannot be calculated due to the lack of quality overlap. “-” means data is not available. Red marks the best diffusion-based result.

Panel of Figure 7 — Figure 7: Rate and perception/fidelity curves on Xiph, HEVC B, MCL-JCV and UVG datasets, comparing with previous pioneering neural video codecs.

Dataset	Method	DISTS	LPIPS	NIQE	PSNR(dB)
Xiph	ProGVC	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000
	DVC [ 20 ]	1171.4 / N/A	437.40 / N/A	1306.9 / N/A	N/A / N/A
	DCVC [ 13 ]	1352.8 / N/A	321.4 / N/A	302.3 / N/A	N/A / N/A
HEVC-B	ProGVC	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000
	DVC [ 20 ]	937.9 / N/A	336.0 / N/A	731.3 / N/A	N/A / N/A
	DCVC [ 13 ]	999.5 / N/A	229.1 / N/A	N/A / N/A	N/A / N/A

Appendix

Table 8: BD-rate

\downarrow

(%) / BD-metric

\uparrow

on the Xiph, HEVC B, MCL-JCV and UVG datasets, comparing with previous pioneering neural video codecs. “N/A” indicates that BD-rate cannot be calculated due to the lack of quality overlap. Red indicates the best performance. The best model in terms of PSNR metric is estimated from Fig. 7 .

	Xiph	HEVC-B	MCL-JCV	UVG
Min ratio	11.1%	13.8%	12.2%	13.4%
Max ratio	15.8%	15.4%	15.6%	15.5%

Appendix

Table 9: Minimum and maximum ratios of intra scale bit cost to the total sequence bit cost across datasets.

Conditioning Strategy	Dropping Strategy	DISTS	LPIPS	PSNR
largest	no drop	0.0 / 0.0000	0.0 / 0.0000	0.0 / 0.0000
	drop1	-13.3 / 0.0020	-17.4 / 0.0015	-22.5 / 0.1398
	drop2	16.8 / -0.0060	9.3 / -0.0092	16.4 / -0.9551
no intra	drop1	7.2 / -0.0021	9.7 / 0.0050	13.4 / -0.4636
no intra	drop2	1.4 / -0.0010	0.6 / -0.0033	6.5 / -0.4308
smallest	drop1	14.3 / -0.0045	9.8 / -0.0050	13.9 / -0.4784

Appendix

Table 10: BD-rate

\downarrow

(%) / BD-metric

\uparrow

on the Xiph, analyzing the impact of dropping intra scales under different intra scale conditioning strategies. The anchor is ProGVC, i.e . inter scales attend to the largest intra scale and no intra scale is dropped. “drop1” and “drop2” denote dropping the largest intra scale and the two largest intra scales, respectively. Blue indicates method that shows compression efficiency improvement.

Conditioning Strategy	Dropping Strategy	DISTS	LPIPS	PSNR
largest	drop1	50.00%	62.50%	50.00%
same resolution	drop1	56.25%	56.25%	56.25%

Appendix

Table 11: Ratio of sequences exhibiting BD-rate performance degradation. “drop1” denotes dropping the largest intra scale.

Attn. Variants	BD-rate $\downarrow$ (%)				Enc. T (s) $\downarrow$	Dec. T (s) $\downarrow$
Attn. Variants	AVG	DISTS	LPIPS	PSNR	Enc. T (s) $\downarrow$	Dec. T (s) $\downarrow$
ProGVC	-51.6	-51.5	-49.4	-53.8	0.56	1.53
Self-only	-46.4	-46.3	-43.8	-49.2	0.50	1.23
Full causal	-52.9	-53.4	-51.2	-54.0	0.64	1.96

Appendix

Table 12: Progressive image compression vs. progressive video compression with three temporal-aligned attention masks. The anchor is frame-wise progressive image compression scheme, where each frame adopts the proposed intra scale coding of ProGVC. “AVG” denotes the average BD-rate performance over DISTS, LPIPS and PSNR. The per-frame encoding and decoding times (Enc. T and Dec. T) are averaged over the lowest and highest bitrates.

Attn. Variants	BD-rate $\downarrow$ (%)				Enc. T (s) $\downarrow$	Dec. T (s) $\downarrow$
Attn. Variants	AVG	DISTS	LPIPS	PSNR	Enc. T (s) $\downarrow$	Dec. T (s) $\downarrow$
ProGVC (largest scale)	-51.6	-51.5	-49.4	-53.8	0.56	1.53
No intra reference	-24.5	-25.8	-22.3	-25.4	0.53	1.46
Smallest scale	-24.4	-25.9	-22.1	-25.1	0.53	1.46
Same resolution scale	-45.5	-45.6	-43.9	-46.9	0.54	1.49

Appendix

Table 13: Effect of different intra scale references for inter scale coding. The anchor is frame-wise progressive image compression scheme, with each frame utilizes the proposed intra scale coding of ProGVC. “AVG” denotes the average BD-rate performance over DISTS, LPIPS and PSNR. The per-frame encoding and decoding times (Enc. T and Dec. T) are averaged over the lowest and highest bitrates.

Panel of Figure 8 — Figure 8: Visual comparisons with baselines across several test sequences. Zoom-in patches highlight area with perceptual differences.

Video Compression

Video compression reduces the bits needed to store or transmit moving images while balancing visual quality, computational cost, and downstream usefulness.

See all 73 papers

ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors
Jun 21, 2026Yixin Gao, Xiaohan Pan, Lin Liu +3Video Diffusion Models Neural Video Compression
University of Science and Technology of China · 2Huawei Inc.
ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization
Mar 3, 2026Hao Cao, Chengbin Liang, Wenqi Guo +2Generative Image Compression Learned Image Compression
Tsinghua University · State Key Laboratory of Space Network and Communications
ResGVC: Residual Coding for High-fidelity Generative Video Compression
Oct 22, 2026Ding Ding, Daowen Li, Ying Chen +1Generative Video Compression Neural Video Compression
Alibaba Group

Jun 21, 2026J/K move · Enter open · S save

ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors

Yixin Gao, Xiaohan Pan, Lin Liu, Xin Li+2

University of Science and Technology of China · 2Huawei Inc.

Video Diffusion Models Neural Video Compression Video Compression Diffusion Models Generative Video Compression

Open paper AskPDF arXiv

Recent generative video compression methods leverage powerful generative priors to achieve perceptually pleasing reconstructions. However, most existing approaches require additional training to adapt generative models to produce realistic reconstructions from compact representations. In this paper, we propose ZeroGVC, a zero-shot generative video compression framework that leverages pretrained autoregressive diffusion priors for low-delay video reconstruction. ZeroGVC encodes the first frame of each group of pictures (GOP) with an image codec and represents subsequent P-frames through Codebook-Guided Autoregressive Latent Compression. This design is motivated by our observation that the compression scheme of denoising diffusion codebook models is effective in few-step consistency sampling. By selecting compact combinations of reproducible codebook noise vectors, ZeroGVC steers the latent denoising trajectory toward the target P-frame while allowing the decoder to reproduce the same trajectory in only a few denoising steps. In addition, we design an optional bidirectional reference mode that mitigates error propagation by leveraging the next I-frame context without introducing any additional bitrate overhead. Extensive experiments on standard video compression benchmarks demonstrate that ZeroGVC achieves superior perceptual reconstruction quality at ultra-low bitrates without any additional training.

Abstract

Video Compression

Explore similar work

ZeroGVC: Zero-Shot Generative Video Compression with Autoregressive Diffusion Priors

ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization

ResGVC: Residual Coding for High-fidelity Generative Video Compression