Recent advances in learned video compression (LVC) have led to significant performance gains, with codecs such as DCVC-RT surpassing the H.266/VVC low-delay mode in compression efficiency. However, existing LVCs still exhibit key limitations: they often require separate models for intra and inter coding modes, and their performance degrades when temporal references are unreliable. To address this, we introduce Uni-LVC, a unified LVC method that supports both intra and inter coding with low-delay and random-access in a single model. Building on a strong intra-codec, Uni-LVC formulates inter-coding as intra-coding conditioned on temporal information extracted from reference frames. We design an efficient cross-attention adaptation module that integrates temporal cues, enabling seamless support for both unidirectional (low-delay) and bidirectional (random-access) prediction modes. A reliability-aware classifier is proposed to selectively scale the temporal cues, making Uni-LVC behave closer to intra coding when references are unreliable. We further propose a multistage training strategy to facilitate adaptive learning across various coding modes. Extensive experiments demonstrate that Uni-LVC achieves superior rate-distortion performance in intra and inter configurations while maintaining comparable computational efficiency.
Figures & tables
Fig. 1: PSNR and BPP vs frames. The results are obtained on videoSRC21 sequence from MCL-JCV, which contains a scene change at frame 48. At this frame, DCVC-RT continues to rely on previous time references, causing a sharp drop in PSNR. In contrast, Uni-LVC automatically suppresses unreliable temporal features and switches to intra‐dominant coding ( αt≈0.1 ), keeping stable PSNR quality.
Fig. 2: Overview of the proposed Uni-LVC. “Enc.” and “Dec.” denote the encoder and decoder, respectively, while “Tem.” represents the temporal modeling module. Inter coding (LD and RA) is formulated as intra coding conditioned on auxiliary temporal features extracted from temporal references stored in the buffer.
Fig. 3: Architecture of the proposed intra codec. “Cross Attn” and “DC Block” denote the cross-attention module and enhanced depthwise convolution block. “Q” indicates quantization. A hybrid buffer ft is formed from ftd and ftr and stores temporal references. For inter coding, the temporal feature ft−1 (or its bidirectional counterpart) is fed through Cross Attn; for pure intra coding, ft−1 is a quality-specific learnable vector fp to preserve compatibility and learn global priors. HPCM denotes the hierarchical progressive context model.
Fig. 9: Rate-distortion (R-D) curves for various methods for intra coding. Please zoom in for more details.
Fig. 10: Rate-distortion (R-D) curves for various methods for inter coding, intra period of 32. Please zoom in for more details.
Method
Type
Total Params. ↓
Latency ↓
BD-Rate (%) w.r.t. VTM 18.0 ↓
Encoding
Decoding
HEVC Class B
HEVC Class C
HEVC Class D
HEVC Class E
UVG
MCL-JCV
Average
VTM-18.0
AI
-
-
-
0%
0%
0%
0%
0%
0%
0%
LALIC [ 16 ]
AI
396.8M
0.491s
0.165s
-16.57%
-15.06%
-14.54%
-20.60%
-16.92%
-17.46%
-16.86%
DCAE [ 15 ]
AI
716.4M
0.346s
0.353s
-20.76%
-16.17%
-14.26%
-23.71%
-19.29%
-20.62%
-19.14%
HPCM [ 14 ]
AI
538.3M
0.232s
0.212s
-19.88%
-20.24%
-19.56%
-26.10%
-19.89%
-20.76%
-21.07%
DCVC-RT AI [ 2 ]
AI
45.7M
0.023s
0.018s
-13.85%
-15.91%
-15.67%
-19.45%
-13.84%
-14.77%
-15.58%
TABLE I: Computational Complexity and BD-Rate Compared to Existing Methods. Best result is underlined . Bold denotes our method.
Variant
BD-Rate (AI, Class B)
B0: DCVC-RT (re-impl)
-12.35%
B1: B0 + enhanced blocks
-14.03%
B2: B1 + HPCM
-16.05%
B3: B2 + LVQ (ours final)
-18.41%
TABLE II: Ablation on intra codec components on HEVC Class B in the AI configuration. Anchor is VTM 18.0 AI (lower is better).
Variant
BD-Rate (LD, Class B)
I0: direct concatenate temporal features
26.71%
I1: temporal path (Swin-CA), no classifier
25.97%
I2: replace Swin-CA with DN-CA
22.93%
I3: add PAL-CA to I2
18.63%
I4: add the reliability-aware classifier (ours final)
16.71%
TABLE III: Ablation on temporal adaptation and the reliability-aware classifier on HEVC Class B. Anchor is VTM 18.0 RA (lower is better).
Scheme
AI
LD
RA
T0: single-stage joint training
-10.19%
27.9%
20.1%
T1: sequential w/o replay
40.51%
60.56%
9.01%
T2: staged + replay (ours final)
-18.41%
16.71%
10.31%
TABLE IV: Ablation on the training strategy on HEVC Class B. Numbers are BD-Rate (%) versus the VTM reference for AI, LD, and RA configurations (lower is better).
Variant
BD-Rate (LD IP32, Class B)
No gating
18.63%
Decoder-side spatial gating (no original frame input)
Video compression reduces the bits needed to store or transmit moving images while balancing visual quality, computational cost, and downstream usefulness.