Dense-Prior-Guided Generative Video Compression
Organizations: Alibaba Group · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Abstract
Diffusion-based generative video compression offers a promising paradigm for low-bitrate reconstruction, but existing keyframe-based controllable approaches rely on sparse priors for non-keyframes, which often lack sufficient low-level details for faithful reconstruction and require non-trivial training to adapt generative models to specialized controls. We propose Dense-Prior-Guided Generative Video Compression (DGVC), a training-free framework that uses codec-compressed luminance components of non-keyframes as dense priors to guide reconstruction. Unlike sparse controls, the dense luminance priors are compatible with modern fidelity-oriented video codecs and remain sufficiently faithful after compression, eliminating the need for prior-specific codec training and making additional generative-model adaptation unnecessary in practice. Moreover, luminance priors preserve richer structural and textural information, thereby providing more reliable guidance for controllable video generation. DGVC further introduces a color-distance-guided keyframe selection strategy to capture color variations for chrominance recovery. Extensive experiments show that DGVC consistently surpasses prior controllable compression methods in both signal fidelity and perceptual quality, while remaining competitive with representative post-enhancement approaches.
Figures & tables
| MCL-JCV | HEVC B | UVG | ||||||||||
| Method | NIQE | DISTS | t-LPIPS | PSNR (dB) | NIQE | DISTS | t-LPIPS | PSNR (dB) | NIQE | DISTS | t-LPIPS | PSNR (dB) |
| DCVC-FM | -28.4 / 0.28 | -11.3 / 0.0057 | 30.0 / -0.0022 | -6.21 / 0.19 | -31.7 / 0.28 | -1.70 / 0.0006 | -7.40 / 0.0004 | -13.9 / 0.38 | -33.2 / 0.25 | -13.1 / 0.0060 | 15.7 / -0.0014 | -18.7 / 0.52 |
| PLVC | N/A / 0.65 | N/A / 0.0454 | 477 / -0.0108 | 353 / -4.39 | N/A / 0.43 | N/A / 0.0436 | 256 / -0.0063 | 356 / -4.02 | N/A / 1.01 | N/A / 0.0486 | 276 / -0.0074 | 473 / -4.27 |
| DiffVC | N/A / 0.75 | N/A / 0.0554 | - / - | 1596 / -7.74 | - / - | - / - | - / - | - / - | N/A / 0.85 | N/A / 0.0545 | - / - | 2336 / -7.61 |
| DGVC | -83.4 / 1.19 | -50.1 / 0.0261 | -18.6 / 0.0011 | 20.1 / -1.15 | -78.4 / 1.07 | -41.1 / 0.0186 | -12.8 / 0.0007 | 35.0 / -1.25 | N/A / 1.79 | -48.9 / 0.0284 | -13.1 / 0.0010 | 46.2 / -1.14 |
| The following results are reported on source sequences resized to 320 512 , in accordance with the configuration proposed in the T-GVC paper. | ||||||||||||
| Encoding runtime (ms) | Decoding runtime (ms) | ||
| T-GVC | DGVC | T-GVC | DGVC |
| 1614 | 8503 | 878.6 | 584.3 |
| Method | NIQE | DISTS | t-LPIPS | PSNR(dB) |
| w/o prior | 133 / -1.14 | 88.9 / -0.0251 | N/A / -0.0371 | N/A / -9.32 |
| w/ poses | 41.4 / -0.52 | 35.1 / -0.0062 | N/A / -0.0195 | N/A / -8.87 |
| w/ edge maps | 65.2 / -0.86 | 16.8 / -0.0043 | 94.1 / -0.0038 | N/A / -4.65 |
| w/ sparse trajectories | 28.9 / -0.35 | 28.2 / -0.0058 | N/A / -0.0122 | N/A / -5.91 |
| keyframe interval=16 | 19.3 / -0.23 | 5.94 / -0.0018 | 27.1 / -0.0009 | 3.23 / -0.18 |
| keyframe interval=32 | 10.8 / -0.14 | 7.80 / -0.0019 | 5.83 / -0.0002 | 7.91 / -0.23 |
Generative Video Compression
Generative video compression uses learned generative models to reconstruct perceptually convincing video from compact transmitted signals.
See all 23 papers