Perceptual video compression leverages generative priors to reconstruct realistic textures and motions at low bitrates. However, existing perceptual codecs often lack native support for variable bitrate and progressive delivery, and their generative modules are weakly coupled with entropy coding, limiting bitrate reduction. Inspired by the next-scale prediction in the Visual Auto-Regressive (VAR) models, we propose ProGVC, a Progressive-based Generative Video Compression framework that unifies progressive transmission, efficient entropy coding, and detail synthesis within a single codec. ProGVC encodes videos into hierarchical multi-scale residual token maps, enabling flexible rate adaptation by transmitting a coarse-to-fine subset of scales in a progressive manner. A Transformer-based multi-scale autoregressive context model estimates token probabilities, utilized both for efficient entropy coding of the transmitted tokens and for predicting truncated fine-scale tokens at the decoder to restore perceptual details. Extensive experiments demonstrate that as a new coding paradigm, ProGVC delivers promising perceptual compression performance at low bitrates while offering practical scalability at the same time.
Figures & tables
Figure 1: Overview of the proposed ProGVC video compression framework. We design a progressive generative video compression framework based on the visual autoregressive model. The left panel illustrates the output pyramid structure of the multi-scale residual quantization. Besides, the sparse attention mask used in the context model is shown on the right.
Figure 2: Overview of the multi-scale autoregressive context model . The transformer network predicts both intra scale and inter scale tokens in an autoregressive manner. The resulting probability distributions are subsequently utilized for entropy coding and for sampling discarded tokens during reconstruction.
Figure 3: Illustration of three attention mask designs: VAR-like attention (left), InfinityStar attention (middle), and our sparse attention (right). For visualization simplicity, we illustrate the masks using three spatial scales (1, 2, 3) and a temporal length of T=2 for inter scale tokens.
Figure 4: Rate and perception/fidelity curves on Xiph, HEVC B and MCL-JCV datasets.
Dataset
Method
DISTS
LPIPS
NIQE
PSNR(dB)
Xiph
VTM-17.0 [ 4 ]
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
DCVC-FM [ 15 ]
92.0 / -0.0257
40.4 / -0.0228
71.3 / -0.2525
111.1 / -1.8623
SEVC [ 3 ]
87.7 / -0.0179
19.4 / -0.0062
41.8 / -0.1331
51.5 / -0.9035
DCVC-B [ 32 ]
63.2 / -0.0148
17.3 / -0.0065
94.7 / -0.2306
56.8 / -1.0048
PLVC [ 46 ]
-40.3 / 0.0109
-83.7 / 0.0565
169.9 / -0.0902
825.9 / -5.5261
ProGVC
-61.5 / 0.0300
3.5 / -0.0025
-62.2 / 0.2670
891.2 / -5.8437
Table 1: BD-rate ↓ (%) / BD-metric ↑ on the Xiph, HEVC B, and MCL-JCV datasets. Red and Blue indicate the best and the second-best performance in terms of BD-metric, respectively. “N/A” indicates that BD-rate cannot be calculated due to the lack of quality overlap.
Table 6
Figure 5: Visual comparisons with baselines on the Kimono sequence in the Xiph dataset and the videoSRC07 sequence in the MCL-JCV dataset.
Model Variants
BD-rate ↓ (%) / BD-metric ↑
DISTS
LPIPS
PSNR
Base
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
w/o context model (uniform prob.)
122.4 / -0.0259
119.6 / -0.0360
119.4 / -2.3437
w/o token generation
N/A / -0.1183
280.9 / -0.1273
189.9 / -3.8102
Table 4: Effect of multi-scale autoregressive context modeling . “uniform prob.” denotes utilizing uniform probability distribution.
Attn. Variants
BD-rate ↓ (%) / BD-metric ↑
Enc. T (s) ↓
Dec. T (s) ↓
DISTS
LPIPS
PSNR
Ours
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
0.56
1.53
Self-only
6.8 / -0.0021
12.8 / -0.0062
10.9 / -0.3482
0.50
1.23
Full causal
-6.5 / 0.0021
-3.8 / 0.0019
0.5 / -0.0184
0.64
1.96
Table 5: Effect of three different temporal-aligned attention masks . The per-frame encoding and decoding times (Enc. T and Dec. T) are averaged over the lowest and highest bitrates.
Attn. Variants
BD-rate ↓ (%) / BD-metric ↑
Enc. T (s) ↓
Dec. T (s) ↓
DISTS
LPIPS
PSNR
Ours (largest scale)
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
0.56
1.53
No intra reference
37.3 / -0.0171
41.7 / -0.0254
46.6 / -1.6698
0.53
1.46
Smallest scale
42.7 / -0.0191
40.4 / -0.0252
46.8 / -1.6803
0.53
1.46
Same resolution scale
11.7 / -0.0042
12.2 / -0.0064
14.4 / -0.5131
0.54
1.49
Table 6: Effect of different intra scales to the context modeling of inter scales . The per-frame encoding and decoding times (denoted as Enc. T and Dec. T) are averaged across the lowest and highest bitrate settings.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Rate and perception/fidelity curves on UVG datasets.
Dataset
Anchor (VTM-17.0 LD)
Method
DISTS
LPIPS
NIQE
PSNR (dB)
MCL-JCV
1080p, 96f
GNVC-VD [ 23 ]
-88.3 / 0.0882
N/A / 0.1470
- / -
N/A / -2.2600
DiffVC [ 21 ]
N/A / 0.0555
N/A / 0.0946
N/A / 0.9003
N/A / -5.7767
720p, 81f
ProGVC
-64.4 / 0.0416
-30.3 / 0.0269
N/A / 0.8172
533.5 / -5.5354
UVG
1080p, 96f
GNVC-VD [ 23 ]
-86.5 / 0.0670
N/A / 0.1263
- / -
120.5 / -1.8390
DiffVC [ 21 ]
N/A / 0.0423
N/A / 0.0657
N/A / 0.9939
N/A / -5.4513
720p, 81f
ProGVC
-63.9 / 0.0322
-35.5 / 0.0236
N/A / 0.7499
744.6 / -5.2541
Appendix
Table 7: Relative BD-rate ↓ (%) / BD-metric ↑ on MCL-JCV, UVG and HEVC B datasets using VTM-17.0 low-delay (LD) configuration as anchor. Diffusion-based results are taken from the published papers and are reported under 1080p/96-frame setting, whereas ProGVC is evaluated under 720p/81-frame setting. “N/A” indicates that BD-rate cannot be calculated due to the lack of quality overlap. “-” means data is not available. Red marks the best diffusion-based result.
Figure 7: Rate and perception/fidelity curves on Xiph, HEVC B, MCL-JCV and UVG datasets, comparing with previous pioneering neural video codecs.
Dataset
Method
DISTS
LPIPS
NIQE
PSNR(dB)
Xiph
ProGVC
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
DVC [ 20 ]
1171.4 / N/A
437.40 / N/A
1306.9 / N/A
N/A / N/A
DCVC [ 13 ]
1352.8 / N/A
321.4 / N/A
302.3 / N/A
N/A / N/A
HEVC-B
ProGVC
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
DVC [ 20 ]
937.9 / N/A
336.0 / N/A
731.3 / N/A
N/A / N/A
DCVC [ 13 ]
999.5 / N/A
229.1 / N/A
N/A / N/A
N/A / N/A
Appendix
Table 8: BD-rate ↓ (%) / BD-metric ↑ on the Xiph, HEVC B, MCL-JCV and UVG datasets, comparing with previous pioneering neural video codecs. “N/A” indicates that BD-rate cannot be calculated due to the lack of quality overlap. Red indicates the best performance. The best model in terms of PSNR metric is estimated from Fig. 7 .
Xiph
HEVC-B
MCL-JCV
UVG
Min ratio
11.1%
13.8%
12.2%
13.4%
Max ratio
15.8%
15.4%
15.6%
15.5%
Appendix
Table 9: Minimum and maximum ratios of intra scale bit cost to the total sequence bit cost across datasets.
Conditioning Strategy
Dropping Strategy
DISTS
LPIPS
PSNR
largest
no drop
0.0 / 0.0000
0.0 / 0.0000
0.0 / 0.0000
drop1
-13.3 / 0.0020
-17.4 / 0.0015
-22.5 / 0.1398
drop2
16.8 / -0.0060
9.3 / -0.0092
16.4 / -0.9551
no intra
drop1
7.2 / -0.0021
9.7 / 0.0050
13.4 / -0.4636
drop2
1.4 / -0.0010
0.6 / -0.0033
6.5 / -0.4308
smallest
drop1
14.3 / -0.0045
9.8 / -0.0050
13.9 / -0.4784
Appendix
Table 10: BD-rate ↓ (%) / BD-metric ↑ on the Xiph, analyzing the impact of dropping intra scales under different intra scale conditioning strategies. The anchor is ProGVC, i.e . inter scales attend to the largest intra scale and no intra scale is dropped. “drop1” and “drop2” denote dropping the largest intra scale and the two largest intra scales, respectively. Blue indicates method that shows compression efficiency improvement.
Conditioning Strategy
Dropping Strategy
DISTS
LPIPS
PSNR
largest
drop1
50.00%
62.50%
50.00%
same resolution
drop1
56.25%
56.25%
56.25%
Appendix
Table 11: Ratio of sequences exhibiting BD-rate performance degradation. “drop1” denotes dropping the largest intra scale.
Attn. Variants
BD-rate ↓ (%)
Enc. T (s) ↓
Dec. T (s) ↓
AVG
DISTS
LPIPS
PSNR
ProGVC
-51.6
-51.5
-49.4
-53.8
0.56
1.53
Self-only
-46.4
-46.3
-43.8
-49.2
0.50
1.23
Full causal
-52.9
-53.4
-51.2
-54.0
0.64
1.96
Appendix
Table 12: Progressive image compression vs. progressive video compression with three temporal-aligned attention masks. The anchor is frame-wise progressive image compression scheme, where each frame adopts the proposed intra scale coding of ProGVC. “AVG” denotes the average BD-rate performance over DISTS, LPIPS and PSNR. The per-frame encoding and decoding times (Enc. T and Dec. T) are averaged over the lowest and highest bitrates.
Attn. Variants
BD-rate ↓ (%)
Enc. T (s) ↓
Dec. T (s) ↓
AVG
DISTS
LPIPS
PSNR
ProGVC (largest scale)
-51.6
-51.5
-49.4
-53.8
0.56
1.53
No intra reference
-24.5
-25.8
-22.3
-25.4
0.53
1.46
Smallest scale
-24.4
-25.9
-22.1
-25.1
0.53
1.46
Same resolution scale
-45.5
-45.6
-43.9
-46.9
0.54
1.49
Appendix
Table 13: Effect of different intra scale references for inter scale coding. The anchor is frame-wise progressive image compression scheme, with each frame utilizes the proposed intra scale coding of ProGVC. “AVG” denotes the average BD-rate performance over DISTS, LPIPS and PSNR. The per-frame encoding and decoding times (Enc. T and Dec. T) are averaged over the lowest and highest bitrates.
Figure 8: Visual comparisons with baselines across several test sequences. Zoom-in patches highlight area with perceptual differences.
Video compression reduces the bits needed to store or transmit moving images while balancing visual quality, computational cost, and downstream usefulness.
Recent generative video compression methods leverage powerful generative priors to achieve perceptually pleasing reconstructions. However, most existing approaches require additional training to adapt generative models to produce realistic reconstructions from compact representations. In this paper, we propose ZeroGVC, a zero-shot generative video compression framework that leverages pretrained autoregressive diffusion priors for low-delay video reconstruction. ZeroGVC encodes the first frame of each group of pictures (GOP) with an image codec and represents subsequent P-frames through Codebook-Guided Autoregressive Latent Compression. This design is motivated by our observation that the compression scheme of denoising diffusion codebook models is effective in few-step consistency sampling. By selecting compact combinations of reproducible codebook noise vectors, ZeroGVC steers the latent denoising trajectory toward the target P-frame while allowing the decoder to reproduce the same trajectory in only a few denoising steps. In addition, we design an optional bidirectional reference mode that mitigates error propagation by leveraging the next I-frame context without introducing any additional bitrate overhead. Extensive experiments on standard video compression benchmarks demonstrate that ZeroGVC achieves superior perceptual reconstruction quality at ultra-low bitrates without any additional training.
Yixin Gao, Xiaohan Pan, Lin Liu +3
University of Science and Technology of China · 2Huawei Inc.
Recent advances in generative image compression (GIC) have delivered remarkable improvements in perceptual quality. However, many GICs rely on large-scale and rigid models, which severely constrain their utility for flexible transmission and practical deployment in low-bitrate scenarios. To address these issues, we propose Progressive Generative Image Compression (ProGIC), a compact codec built on residual vector quantization (RVQ). In RVQ, a sequence of vector quantizers encodes the residuals stage by stage, each with its own codebook. The resulting codewords sum to a coarse-to-fine reconstruction and a progressive bitstream, enabling previews from partial data. We pair this with a lightweight backbone based on depthwise-separable convolutions and small attention blocks, enabling practical deployment on both GPUs and CPU-only devices. Experimental results show that ProGIC attains comparable compression performance compared with previous methods. It achieves bitrate savings of up to 57.57% on DISTS and 58.83% on LPIPS compared to MS-ILLM on the Kodak dataset. Beyond perceptual quality, ProGIC enables progressive transmission for flexibility, and also delivers over 10 times faster encoding and decoding compared with MS-ILLM on GPUs for efficiency.
Hao Cao, Chengbin Liang, Wenqi Guo +2
Tsinghua University · State Key Laboratory of Space Network and Communications
Generative video compression (GVC) methods usually employ decoder-side diffusion models to denoise reconstructed signals based on generative priors. Besides, most of these methods perform denoising in latent space rather than directly in pixel domain. In this paper, we propose ResGVC, a residual coding framework for GVC that guides denoising toward original pictures by transmitting residual signals. ResGVC first encodes the original pictures within a learned video codec, and then decomposes the distortion between the codec reconstruction and the original pictures into a sequence of residual segments, where each residual segment is acquired at each denoising step. Residual segments can further be compressed by a novel residual codec with a diffusion transformer as an entropy model. Unlike prior methods, ResGVC introduces explicit source constraints throughout denoising rather than relying on decoder-side generation alone. The diffusion-based entropy model also enables more effective distribution modeling capability based on the powerful generative priors. Moreover, denoising directly in pixel space avoids the irreversible information loss caused by latent representation bottlenecks. Experimental results show that, compared with the previous state-of-the-art method, ResGVC achieves 63.7% and 83.8% BD-rate reductions in terms of LPIPS and PSNR, respectively, while also delivering superior visual quality.