Traditional neural video compression (NVC) methods predominantly follow strictly causal predictive-coding paradigm, treating zero-frame delay as a fixed constraint rather than delay as an explicit design parameter. From an information-theoretic perspective, allowing limited frame delay enables a codec to exploit correlations with future frames within the delay window, thereby reducing uncertainty that is inaccessible to strictly causal prediction and potentially improving the rate-distortion tradeoff. In this work, we formalize this intuition through a rigorous rate-distortion analysis builds on the clique factorization of Markov networks, and propose Graph-Factored Sequential Coding (GFSC), a principled framework for sequential video coding under limited delay. To our knowledge, we provide the first theoretical result showing that, for general K-order Markov sources with positive distributions, a K-frame delay is sufficient to attain joint-coding rate-distortion optimality over a nontrivial distortion region, a guarantee unavailable to zero-delay causal coding in general. In practice, we instantiate GFSC through lightweight neural modules, allowing existing causal predictive codecs to be augmented without modifying their core architectures. Experiments across multiple NVC models and datasets show that GFSC consistently improves rate-distortion performance over the corresponding causal counterparts. Under matched delay budgets, GFSC also achieves competitive performance compared with representative bidirectional codecs.
Figures & tables
Figure 1: Left : GFSC- 1 coding architecture for three frames. Middle : Rate-PSNR comparison between zero-delay DCVC and GFSC with delays K=1,2,3 on UVG. Right : Latency (top; lower the better) and BD-rate (bottom; lower the better) of GFSC and DCVC-B at matched delays, with zero-delay DCVC as the baseline. BD-rate is measured relative to zero-delay DCVC.
Figure 2: Temporal-dependence analysis for UVG dataset. Left : Estimated information matrix for 32 -frame UVG video clips. Right : Mean APC vs. Separation curve. APC denotes Absolute partial correlation. Separation denotes the distance between two frame positions, i.e. ∣j−k∣ for (Xj,Xk) .
Figure 3: Second-order Markov structure illustration. Left : Maximal cliques and clique factorization. Middle : Second-order Markov network. Right : The information matrix J for Gauss-Markov satisfying Jij=0 for ∣i−j∣>2 . “ ∗ ” denotes potentially nonzero entries.
Figure 4: Left : Second-order Markov network (top), its augmentation by clique-wise auxiliary variables Zj (middle), and further augmentation by node-wise auxiliary variables Wℓ (bottom). Right : GFSC architecture with 2 -delay and optimal auxiliary variables constructed in Left .
Figure 5: Left : A second-order GGM and its augmented representation within DϵG ; dashed arrows indicate sequential coding. Right : Edge-wise auxiliary construction by linear interpolation and additive noise, where ωˉj,k=1−ωj,k .
Figure 6: Rate-PSNR comparison between causal and GFSC-enhanced models on UVG and MCL-JCV dataset. Columns show DCVC, VCT, and DHVC, respectively.
UVG ( 1080 p)
MCL-JCV ( 1080 p)
Delay
DCVC
VCT
DHVC
DCVC
VCT
DHVC
K=1
-20.57
-28.38
-7.39
-10.90
-32.35
-13.35
K=2
-25.29
-48.00
-10.17
-17.32
-50.14
-14.65
K=3
-28.22
-55.78
-11.91
-20.30
-56.70
-16.07
Table 1: BD-rate (%) of GFSC-enhanced models relative to causal baselines.
Figure 7: Rate-PSNR comparison under delay K≤3 on (a) UVG and (b) MCL-JCV. Colors identify model families. Line styles and markers distinguish delay settings.
Method
Delay
BD-rate
Enc. Latency
Dec. Latency
Memory
FLOPs
Params
K
(%)
(s/frame)
(s/frame)
(GB)
(G/frame)
(M)
DCVC
0
0.0
0.52
0.44
5.54
4430.53
34.53
DCVC-L
0
-8.57
0.53
0.44
5.54
4447.51
39.21
DCVC-B
1
3.70
0.84
0.64
10.95
7153.37
55.38
2
-18.64
1.14
0.89
10.95
8805.79
55.38
3
-27.24
1.16
0.90
10.95
9632.01
55.38
Table 2: BD-rate (%) and computational complexity under matched delay budgets. Latency, memory, FLOPs, and model parameters are reported for DCVC, DCVC-GFSC, and DCVC-B.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Illustration of three coding architectures for a four-frame source under 2 -frame delay. Left : C-NC. Middle : NC-NC. Right : NC-C.
Figure 9: Left : graphical illustration for causal predictive coding. Middle : graphical illustration for bidirectional coding. Right : graphical illustration for proposed GFSC- 1 .
Figure 10: Clique-wise augmentation of a second-order Markov network, where Uj is associated with the maximal clique (Xj,Xj+1,Xj+2) .
Figure 11: Augmented graphical model with node-wise and edge-wise auxiliary variables.
Figure 12: Gaussian source (X1,X2) with σ12=σ22=1 and ρ=0.7 . Left : rate-distortion R(D) comparisons for symmetric distortion constraint D . Interpolation coefficient ω1,2=0.5 for ISC. Middle : Rate gap between joint coding (ISC) and causal coding for symmetric distortion case. Right : rate-distortion R(D2) comparisons for asymmetric distortion constraint with fixed d1=0.1,0.2 , and 0.4 upside down.
Figure 13: Illustration of the delay induced by B-frame coding over an N -length video clip.
Figure 14: Simplified DCVC and GFSC architectures. GFSC introduces temporal joint modeling (TJM) modules to both motion and context branch, highlighted in orange and purple , respectively. One delayed frame case is shown. Additional delayed frames provide further inputs to these modules. The mappings fm,fc and gm,gc denote motion/context encoders and decoders, respectively.
Figure 15: Temporal joint modeling modules for motion and context branches in DCVC architecture. Left : TJMm concatenates motion features and their absolute difference as input. Right : TJMc jointly updates the two context features using a spatial-temporal self attention module.
Figure 16: Left : Rate-PSNR curves on LIVE-HDR 4K. Right : BD-rate relative to causal DCVC. Negative values indicate rate savings.
While neural video codecs (NVCs) have demonstrated superior compression ratio, their prohibitive computational complexity remains a critical barrier to real-world deployment. This paper introduces a chunk-based coding framework designed to significantly improve the rate-distortion-complexity trade-off. Instead of processing frames sequentially, our approach encodes a chunk of multiple frames into a single compact latent representation and decodes them simultaneously. This is enabled by cross-frame interaction modules for joint spatial-temporal modeling and frame-specific decoders for parallel reconstruction. This paradigm not only dramatically enhances coding throughput but also facilitates more effective modeling of long-term temporal correlations. To further boost speed, we propose a streamlined entropy coding mechanism that consolidates bit-stream interactions into a single step, substantially reducing decoding overhead. Building on these innovations, we present DCVC-UF (Ultra-Fast), a new NVC that sets a new SOTA in performance. Our experiments show that DCVC-UF can achieve ultra-fast encoding and decoding speeds, significantly outperforming previous leading codecs. DCVC-UF serves as a notable landmark in the journey of NVC evolution. The code is at https://github.com/microsoft/DCVC.
Jiahao Li, Wenxuan Xie, Zhaoyang Jia +4
Microsoft Research Asia · University of Science and Technology of China · *Done during Zhaoyang Jia’s internship at Microsoft Research Asia.
Neural video compression (NVC) technologies have advanced rapidly in recent years, yielding state-of-the-art schemes such as DCVC-RT that offer superior compression efficiency to H.266/VVC and real-time encoding/decoding capabilities. Nonetheless, existing NVC schemes have several limitations, including inefficiency in dealing with disocclusion and new content, interframe error propagation and accumulation, among others. To eliminate these limitations, we borrow the idea from classic video coding schemes, which allow intra coding within inter-coded frames. With the intra coding tool enabled, disocclusion and new content are properly handled, and interframe error propagation is naturally intercepted without the need for manual refresh mechanisms. We present an NVC framework with unified intra and inter coding, where every frame is processed by a single model that is trained to perform intra/inter coding adaptively. Moreover, we propose a simultaneous two-frame compression design to exploit interframe redundancy not only forwardly but also backwardly. Experimental results show that our scheme outperforms DCVC-RT by an average of 12.1% BD-rate reduction, delivers more stable bitrate and quality per frame, and retains real-time encoding/decoding performances. Code and models will be released.
Hui Xiang, Yifan Bian, Li Li +3
University of Science and Technology of China · Tencent Shannon Lab
In this paper we propose DCVC-Mamba (DCVC-MB), a neural video codec framework for B-frame coding. Our approach incorporates an IBP frame strategy for low-delay B-frame coding, a spatio-temporal fusion model based on state-space models for bidirectional temporal prediction, and an entropy-aware skipping mechanism that selectively omits coding certain latents to reduce entropy coding times. In addition to our model contributions we also implement two inference-time strategies that enhance compression performance. Experimental evaluation shows that DCVC-MB compares favorably to existing NVCs and traditional codecs. The method demonstrates BD-rate reductions of up to 8.98% on average compared to prior neural video codecs, and improvements of up to 30.45% and 1.81% over the VTM-19.0-LDP and VTM-19.0-RA(Inter-GoP=16) benchmarks, respectively, contributing to advances in neural video compression.
Arjun Arora, Calvin-Khang Ta, Carlos Restrepo-Galeano +7
Dolby Laboratories · Voia · University of Delaware +1