Bitstream-corrupted video recovery aims to restore realistic content degraded during video storage or transmission. Existing methods typically assume that predefined masks of corrupted regions are available, but manually annotating these masks is labor-intensive and impractical in real-world scenarios. To address this limitation, we introduce a new blind video recovery setting that removes the reliance on predefined masks. This setting presents two major challenges: accurately identifying corrupted regions and recovering content from extensive and irregular degradations. We propose a Metadata-Guided Diffusion Model (M-GDM) to tackle these challenges. Specifically, intrinsic video metadata are leveraged as corruption indicators through a dual-stream metadata encoder that separately embeds motion vectors and frame types before fusing them into a unified representation. This representation interacts with corrupted latent features via cross-attention at each diffusion step. To preserve intact regions, we design a prior-driven mask predictor that generates pseudo masks using both metadata and diffusion priors, enabling the separation and recombination of intact and recovered regions through hard masking. To mitigate boundary artifacts caused by imperfect masks, a post-refinement module enhances consistency between intact and recovered regions. Extensive experiments demonstrate the effectiveness of our method and its superiority in blind video recovery. Code is available at: https://github.com/Shuyun-Wang/M-GDM.
Figures & tables
Figure 2: Comparison of the Previous and Proposed Approaches. (a) Previous approaches rely on predefined input masks, which are labor-intensive and difficult to obtain. (b) Our approach employs easily accessible metadata extracted from the bitstream as guidance without using any predefined masks.
Figure 3: Overview of Our Proposed Metadata-Guided Diffusion Model (M-GDM). M-GDM consists of three key components for corrupted bitstream video recovery: (1) a dual-stream metadata encoder that extracts metadata representations from the corrupted bitstream, (2) a prior-driven mask predictor that predicts masks for corrupted regions, and (3) a post-refinement module that eliminates boundary artifacts after hard-masking, where RSTB represents a residual swin transformer block. The metadata guides the diffusion model to focus on corrupted areas, while the prior-driven mask predictor and post-refinement module facilitate a better fusion of intact and recovered regions, resulting in high quality video recovery. ‘0’ and ‘1’ represent the raw bitstream.
Figure 4: Qualitative Comparison on the YouTube-VOS Dataset. From left to right, we present the corrupted frames (with the ground-truth masks shown in the top-left corner), the results of E2 FGVI, ProPainter, BSCVR, our proposed M-GDM, and the ground-truth. Our approach consistently produces results that are visually more appealing.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
VFID ↓
E2 FGVI [ 12 ]
26.26
0.8774
0.0547
0.0783
ProPainter [ 37 ]
27.01
0.9024
0.0439
0.0681
BSCVR [ 16 ]
27.68
0.9173
0.0359
0.0662
M-GDM (Ours)
28.21
0.9202
0.0333
0.0661
Table 1: Quantitative Comparison on the YouTube-VOS Dataset. We compare with the results of state-of-the-art video recovery methods on the YouTube-VOS dataset. Our proposed M-GDM achieves the best performance across all the metrics.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
VFID ↓
E2 FGVI [ 12 ]
23.83
0.8272
0.0569
0.2401
ProPainter [ 37 ]
24.13
0.8436
0.0510
0.2159
BSCVR [ 16 ]
24.50
0.8613
0.0433
0.1864
M-GDM (Ours)
26.05
0.8822
0.0340
0.1621
Table 2: Quantitative Comparison on the DAVIS Dataset. We report the performance of different video recovery methods on the DAVIS dataset. Our M-GDM achieves the highest values across all the metrics.
Figure 5: Qualitative Comparison on the DAVIS Dataset. We showcase the qualitative comparison among different video recovery methods on DAVIS. The artifacts caused by bitstream corruption have been significantly reduced by our method while the other methods still suffer obvious artifacts.
Exp.
DME
PMP
PRM
PSNR ↑
SSIM ↑
LPIPS ↓
(a)
21.56
0.8054
0.0565
(b)
✓
22.27
0.8122
0.0538
(c)
✓
✓
25.84
0.8784
0.0328
(d)
✓
✓
✓
26.05
0.8822
0.0340
Table 3: Ablation Studies of the Proposed Components. DME represents the Dual-stream Metadata Encoder, PMP is the Prior-driven Mask Predictor, and PRM indicates the Post-Refinement Module. Starting from the fine-tuned temporal U-Net (a), we progressively add each component for evaluation.
This paper reports on the NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration (BSCVR). The challenge aims to advance research on recovering visually coherent videos from corrupted bitstreams, whose decoding often produces severe spatial-temporal artifacts and content distortion. Built upon recent progress in bitstream-corrupted video recovery, the challenge provides a common benchmark for evaluating restoration methods under realistic corruption settings. We describe the dataset, evaluation protocol, and participating methods, and summarize the final results and main technical trends. The challenge highlights the difficulty of this emerging task and provides useful insights for future research on robust video restoration under practical bitstream corruption.
Bitstream-corrupted Harsh Visual Understanding (BcHVU) aims to understand harshly degraded videos originally decoded from a severely corrupted bitstream in real-world multimedia communication. The ill-posed nature of BcHVU poses a major challenge for existing vision models, as even subtle bitstream corruption can lead to irreversible pixel distortion and significant semantic loss. To address these challenges in BcHVU, we propose Bitstream Language Modeling as Robust Semantic Priors (BLMSP), a framework for learning and injecting bitstream-native semantic cues. Our proposed BLMSP framework learns to extract bitstream-native semantic cues by bitstream language modeling, and leverages them as priors by injecting into off-the-shelf vision models of BcHVU tasks. Specifically, we present a Video Bitstream Byte Model (VBBM) that integrates byte-level modeling and cross-codec semantic distillation, enabling it to interpret robust semantics from byte sequences in multiple corrupted bitstream formats. The learned bitstream semantics are leveraged as robust priors and fused into BcHVU model backbones for improving the quality of video restoration, captioning, and human pose estimation. To train BLMSP, we construct a large-scale multi-source Corrupted-bitstream Harsh-video Paired (CHP) dataset containing 607k corrupted bitstream segments and 287k paired harsh video clips. Extensive experimental results show that the learned bitstream priors improve video restoration, captioning, and human pose estimation by 2.51 dB in PSNR, 0.20 in CIDEr, and 0.18 in [email protected] on average, respectively. These results demonstrate that corrupted bitstream can serve as robust semantic priors in solving pixel distortion and semantic loss in BcHVU.
Recovering visually faithful content from damaged video bitstreams is important for reliable multimedia communication and storage. Existing methods exploit local residuals and non-local RGB references of the input video sequence, but both sources can become unreliable after decoding, while processing additional references increases receiver computation and memory. This paper studies multimodal bitstream-corrupted video recovery (MBSCVR) using RGB and depth information, where pre-transmission depth-based geometry aids reconstruction when the received appearance is corrupted. We propose a novel MBSCVR framework that selects and effectively utilizes a single selected non-local RGB reference per recovery window to achieve higher performance. It integrates a reliability-aware multimodal selector that combines valid appearance with depth compatibility before reference encoding. Depth-Conditioned Reference Adaptation (DCRA) introduces selected RGB evidence into feature completion and content generation, while frame-aligned depth conditions completion queries. On the complete BSCV benchmark with oracle masks, MBSCVR uses one selected reference to achieve 32.30 dB on YouTube-VOS and 28.93 dB on DAVIS, surpassing full-reference B2SCVR by 0.27 and 0.33 dB and improving SSIM on both subsets. Matched receiver-forward measurements show 36.8% lower latency and 40.8% lower peak memory. These results demonstrate how compact geometric guidance and adapted appearance processing improve the quality--resource trade-off of multimedia recovery.
Tianyi Liu, Jiaojiao Wang, Ye Lu +4
Nanyang Technological University, Singapore · School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China +2