MDFI: A Multi-Domain Features Integration for Compressed Video Quality Enhancement
Authors: Sang NguyenQuang, Hieu Bui Minh, Dang BuiDinh, Xiem HoangVan
Organizations: Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan · Faculty of Electronics and Telecommunications, VNU-University of Engineering and Technology, Vietnam National University, Hanoi 100000, Vietnam
The latest video coding standard, H.266/VVC, has demonstrated significant improvements in compression efficiency compared to H.265/HEVC. Despite its advanced coding techniques, H.266/VVC still faces challenges in meeting the increasing demand for higher perceptual quality and enhanced compression performance. To address these limitations, we propose MDFI (Multi-Domain Features Integration), a compressed video quality enhancement approach that features a novel Frame-Prediction Feature Transform (FPFT) module to process prediction information. Moreover, MDFI integrates a multi-domain feature fusion strategy that effectively combines spatiotemporal characteristics, cross-frequency representations, and compressed-domain prediction information to enhance decoded video quality. Additionally, we introduce a comprehensive dataset that encompasses uncompressed video sequences, corresponding reconstructed versions at multiple QP levels, and predicted frames generated from H.266/VVC compressed bitstreams, providing essential resources for developing and benchmarking video enhancement approaches. Extensive experiments demonstrate that our MDFI approach achieves superior performance to state-of-the-art methods in both objective metrics and visual quality, effectively mitigating video compression artifacts. The code is available at: https://github.com/dangdinh17/MDFI.git.
Figures & tables
Fig. 1: PSNR comparison and exploited information in TGAF [ 38 ] , TVQE [ 35 ] , OVQE [ 22 ] and our MDFI.
Fig. 2: Overview of the MDFI architecture with the proposed Frame-Prediction Feature Transform. The full framework is illustrated in the gray background, while the SFT submodule is highlighted in the bottom-right corner.
Fig. 3: Visualization of decoded information used in MDFI.
TABLE II: Overall comparison for Δ PSNR (dB) and Δ SSIM ( ×10−2 ) over standard test sequences at four QPs
Fig. 4: PSNR comparison across frames on BasketballDrill (top) and BQMall (bottom) sequences at QP = 37.
Sequence
TGAF [ 38 ]
TVQE [ 35 ]
OVQE [ 22 ]
Wang et al [ 25 ]
MDFI
A
Traffic
-18.31
-18.59
-25.15
-25.50
-29.23
PeopleOnStreet
-13.39
-12.48
-15.00
-15.82
-18.91
B
Kimono
-17.31
-15.06
-20.53
-19.86
-24.83
ParkScene
-20.60
-17.34
-26.77
-27.16
-30.80
Cactus
-15.12
-14.08
-19.09
-19.11
-22.45
BQTerrace
-12.71
-12.39
-18.64
-18.81
-23.46
TABLE III: BD-rate (%) with H.266/VVC as the anchor.
Fig. 5: Visual comparison of results between experimented methods. The sources of the video frames in the visualization results are: BasketballPass (416 × 240), BQMall (832 × 480).
Fig. 6: Rate-Distortion performance comparison on Traffic, BQSquare, RaceHorses, FourPeople.
Fig. 7: Trade-off analysis between quality improvement and model complexity.
Model
Parameters (M)
FLOPs (T)
Δ PSNR (dB)
BD-rate (%)
TGAF [ 38 ]
1.39
1.32
0.62
-16.82
TVQE [ 35 ]
2.08
0.12
0.60
-14.95
OVQE [ 22 ]
3.06
2.13
0.74
-21.03
Wang et al [ 25 ]
2.51
4.42
0.75
-21.35
MDFI (Ours)
3.84
2.85
0.85
-24.66
TABLE IV: Parameters, FLOPs, Δ PSNR, and BD-rate comparisons of different methods at QP = 37.
Model
Configuration
Parameters (M)
FLOPs (T)
Time (ms)
Δ PSNR (dB)
Ratio
MDFI
out_nc = 64, N=16
3.84
2.85
724
0.85
3.35
MDFI_S1
out_nc = 64, N=1
3.44
2.48
550
0.83
2.99
MDFI_S2
out_nc = 32, N=16
2.22
1.61
501
0.74
2.17
MDFI_S3
out_nc = 32, N=1
1.82
1.24
326
0.62
2.00
OVQE
Baseline
3.06
2.13
475
0.74
2.88
TABLE V: Ablation study under different out_nc and N settings of MDFI, compared with baseline OVQE at QP=37.
FPFT
STFF
GMFF
FQE
Δ PSNR
Δ SSIM
✓
0.35
0.74
✓
✓
0.46
1.00
✓
✓
✓
0.74
1.49
✓
✓
✓
✓
0.85
1.68
TABLE VI: Impact of different MDFI components on performance in terms of Δ PSNR and Δ SSIM at QP=37.
Model
Δ PSNR ↑
Δ SSIM ↑
Δ VMAF ↑
Δ LPIPS ↓
TGAF [11]
0.62
1.23
2.33
-0.63
TVQE [12]
0.60
1.19
2.86
-0.49
OVQE [13]
0.74
1.49
2.96
-1.01
Wang et al [14]
0.75
1.48
3.13
-0.79
MDFI (Our)
0.85
1.68
3.79
-1.17
TABLE VII: Overall evaluation for Δ PSNR (dB), Δ SSIM ( ×10−2 ) , Δ VMAF and Δ LPIPS ( ×10−2 ) over standard test sequences at QP=37.
Video compression reduces the bits needed to store or transmit moving images while balancing visual quality, computational cost, and downstream usefulness.
Perceptual quality enhancement of severely compressed videos remains challenging due to complex artifact patterns and substantial information loss. Recent diffusion models have demonstrated strong generative capability for visual restoration, but directly applying them to compressed video often ignores compression degradation characteristics and may introduce structure-inconsistent hallucinations. To address this issue, this paper presents a diffusion-based compressed video enhancement method, named DiffCVE. Coding Prior-enhanced Dual Conditioning (CPDC) branches are designed to jointly model compressed video and coding prior conditions, where coding priors including residuals and motion vectors provide complementary structural and motion guidance during the diffusion denoising process. To make the diffusion process aware of compression severity, a Compression Degradation Semantic Prompting (CDSP) mechanism is introduced to leverage QP-conditioned textual prompts together with LoRA fine-tuning. In addition, a Coding Prior-guided Weighted Fusion (CPWF) module is incorporated into the VAE decoder to fuse VAE encoder and coding prior encoder features with QP-predicted weights. Extensive experiments demonstrate the effectiveness of the proposed method in improving perceptual quality, especially under severe compression settings. The project page with enhanced video demonstrations is available at https://wqmaker.github.io/projects/DiffCVE/.
Wenqiang Xiao, Wenzhuo Ma, Junxi Zhang +1
School of Remote Sensing and Information Engineering, Wuhan University
Video compression aims to maximize reconstruction quality with minimal bitrates. Beyond standard distortion metrics, perceptual quality and temporal consistency are also critical. However, at ultra-low bitrates, traditional end-to-end compression models tend to produce blurry images of poor perceptual quality. Besides, existing generative compression methods often treat video frames independently and show limitations in time coherence and efficiency. To address these challenges, we propose the Efficient Video Diffusion with Sparse Information Transmission (Diff-SIT), which comprises the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE). The STEM sparsely encodes the original frame sequence into an information-rich intermediate sequence, achieving significant bitrate savings. Subsequently, the ODFTE processes this intermediate sequence as a whole, which exploits the temporal correlation. During this process, our proposed Frame Type Embedder (FTE) guides the diffusion model to perform adaptive reconstruction according to different frame types to optimize the overall quality. Extensive experiments on multiple datasets demonstrate that Diff-SIT establishes a new state-of-the-art in perceptual quality and temporal consistency, particularly in the challenging ultra-low-bitrate regime. Code is released at https://github.com/MingdeZhou/Diff-SIT.
Video technology is advancing toward Ultra High Definition (UHD) and High Dynamic Range (HDR), which intensifies the need for higher compression efficiency for these high-specification videos. Beyond advances in traditional codecs, neural video codecs (NVCs) have attracted significant research attention and have evolved rapidly over the past few years. The coding artifacts of NVCs often exhibit content-varying and generative characteristics, which differ from those of conventional codecs and are challenging for traditional video quality assessment (VQA) methods to capture. Therefore, VQA metrics are required to generalize across different codecs, content types, and dynamic ranges to better support video codec research and evaluation. In this paper, we propose FDIM, a feature-distance-based generic video quality metric for both traditional and neural video codecs across SDR and HDR formats. FDIM employs a hybrid architecture that integrates deep and hand-crafted features. The deep feature component learns multi-scale representations to capture distortions ranging from structural and textural fidelity degradation to high-level semantic deviations, while the hand-crafted feature component provides stable complementary cues to improve overall generalization. We trained FDIM on a large-scale subjective quality assessment dataset (DCVQA) consisting of over 16k video sequences encoded by traditional block-based hybrid video codecs and end-to-end perceptually optimized neural video codecs. Extensive experiments on ten SDR/HDR VQA datasets containing diverse, previously unseen codecs demonstrate that FDIM achieves strong generalization and high correlation with subjective assessment. The source code for FDIM and the DCVQA validation set will be released at https://github.com/MCL-ZJU/FDIM.
Jiayi Wang, Lichun Zhang, Xiaoqi Zhuang +3
College of Information Science and Electronic Engineering, Zhejiang University, and also with the Zhejiang Key Laboratory of Multimodal Communication Networks and Intelligent Information Processing, Hangzhou 310058, China · Central Media Technology Institute, Huawei, Hangzhou 310058, China