Improved Adversarial Diffusion Compression for Real-World Video Super-Resolution

Bin Chen; Weiqi Li; Shijie Zhao; Xuanyu Zhang; Junlin Li; Li Zhang; Jian Zhang

cs.CVFeb 28, 2026

Improved Adversarial Diffusion Compression for Real-World Video Super-Resolution

Authors: Bin Chen, Weiqi Li, Shijie Zhao, Xuanyu Zhang, Junlin Li, Li Zhang, Jian Zhang

Organizations: School of Electronic and Computer Engineering, Peking University · ByteDance Inc.

Abstract

While many diffusion models have achieved impressive results in real-world video super-resolution (Real-VSR) by generating rich and realistic details, their reliance on multi-step sampling leads to slow inference. One-step networks like SeedVR2, DOVE, and DLoRAL alleviate this through condensing generation into one single step, yet they remain heavy, with billions of parameters and multi-second latency. Recent adversarial diffusion compression (ADC) offers a promising path via pruning and distilling these models into a compact AdcSR network, but directly applying it to Real-VSR fails to balance spatial details and temporal consistency due to its lack of temporal awareness and the limitations of standard adversarial learning. To address these challenges, we propose an improved ADC method for Real-VSR. Our approach distills a large diffusion Transformer (DiT) teacher DOVE equipped with 3D spatio-temporal attentions, into a pruned 2D Stable Diffusion (SD)-based AdcSR backbone, augmented with lightweight 1D temporal convolutions, achieving significantly higher efficiency. In addition, we introduce a dual-head adversarial distillation scheme, in which discriminators in both pixel and feature domains explicitly disentangle the discrimination of details and consistency into two heads, enabling both objectives to be effectively optimized without sacrificing one for the other. Experiments demonstrate that the resulting compressed AdcVSR model reduces complexity by 95% in parameters and achieves an 8 $\times$ acceleration over its DiT teacher DOVE, while maintaining competitive video quality and efficiency.

Figures & tables

Figure 1: Comparison of methods in compressing diffusion networks for Real-VSR. (a) Traditional ADC ( Chen et al., 2025a ) distills an SD network with 2D spatial attentions into a pruned student using a single adversarial signal without temporal modeling, suffering from frame flickering. (b) Our improved ADC distills a larger DiT-based teacher with heavier 3D spatio-temporal attention into the same 2D student, augmented by 1D temporal convolutions, using dual-head discriminators

{\mathcal{D}}

in pixel and feature domains. Through disentangling the discriminations of detail richness and temporal consistency into different heads, it balances the optimization of both.

Figure 2: Illustration of the proposed improved ADC method, with application to compressing DOVE (teacher) into AdcVSR model (student). (a) We augment the 2D AdcSR Real-ISR network ( Chen et al., 2025a ) , consisting of pruned SD2.1 UNet and VAE decoder, through inserting 1D temporal residual blocks (RBs) after each 2D spatial RB and Transformer block (TB), enabling temporal modeling, while maintaining efficiency for Real-VSR. The resulting AdcVSR network is then fully trained end-to-end via adversarial distillation from DOVE. (b) For adversarial learning, we design dual-head discriminators with pretrained backbones for feature extractions. Each discriminator uses 2D and 1D convolutions, followed by two linear projection heads at the tail, to separately evaluate detail richness and temporal consistency. Training is guided by five curated types of video and image data (1-5) with head-specific labels, achieving a balanced optimization of details and consistency.

Method	RealBasicVSR	Upscale-A-Video	MGLD-VSR	STAR	SeedVR2	DOVE	DLoRAL	PiSA-SR	AdcSR	HYPIR	AdcVSR (Ours)
Test Dataset: UDM10 (Synthetic)
PSNR↑	24.39	23.03	24.28	22.61	25.92	26.00	22.49	23.21	23.39	22.55	25.36
SSIM↑	0.7376	0.6189	0.7491	0.6534	0.7674	0.7805	0.7130	0.6799	0.6772	0.6995	0.7697
LPIPS↓	0.3283	0.4218	0.3103	0.5055	0.2653	0.2645	0.3201	0.3658	0.3781	0.3736	0.3065
DISTS↓	0.2078	0.2360	0.1909	0.2665	0.1532	0.1732	0.2066	0.2213	0.2287	0.2125	0.2112
MANIQA↑	0.5725	0.5331	0.5558	0.3468	0.5232	0.5133	0.5679	0.6257	0.5696	0.5856	0.5793

Table 1: Quantitative comparison of Real-VSR performance. Inference time is measured on an NVIDIA H20 GPU for generating a 25-frame video at spatial resolution

512\times 512

. The best, second- best, and third-best results are labeled in bold red , underlined blue , and italic green , respectively.

Panel of Figure 3 — Figure 3: Qualitative comparison of Real-VSR performance on 13th frames of two videos: “028” from RealVSR (top) and “016” from VideoLQ (bottom). Temporal profiles are provided below each frame, obtained by taking slices along the width-temporal plane at the vertical centers of the frames.

Figure 4: Performance comparison among diffusion-based Real-VSR methods in temporal consistency and complexity (parameter number and inference time) (see Tab. 1 ). AdcVSR attains the lowest warping error

E_{\text{warp}}^{*}

, the second-lowest parameter number, and the second-highest inference speed. Bubble colors represent method types: green for multi-step, blue for one-step, and red for AdcVSR.

Method	CLIPIQA↑	$E_{\text{warp}}^{*}$ ↓
Single-Head, Dual-Domain	0.6745	6.32
Dual-Head, Single-Domain	0.6421	3.59
Dual-Head, Dual-Domain (Ours)	0.6861	2.22

Table 3: Comparison of discriminators on YouHQ40.

Method	PSNR↑	LPIPS↓	MUSIQ↑
No Adversarial Loss	23.97	0.3596	54.33
No Teacher (HR GT Only)	24.85	0.3641	50.32
SeedVR2 as Teacher	23.24	0.3489	60.74
DLoRAL as Teacher	23.08	0.3554	54.61
DOVE as Teacher (Ours)	23.81	0.3337	61.48

Table 4: Comparison of distillation setups on MVSR4x.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Training Configuration	LPIPS $\downarrow$	CLIPIQA $\uparrow$	$E_{\text{warp}}^{*}$ $\downarrow$
No ${\mathbf{x}}_{\text{video}}^{}$ and No ${\mathbf{x}}_{\text{image}}^{}$ (Types 3 & 5)	0.3430	0.6521	5.92
Real Videos as Detail-Real (Label “1”)	0.3152	0.6703	1.98
Teacher Videos Replacing Real Videos	0.3224	0.6652	2.47
Full Scheme (Ours)	0.3065	0.6818	1.67

Appendix

Table 7: Ablation of curated data for

{\mathcal{D}}

on UDM10.

Method	RealBasicVSR	Upscale-A-Video	MGLD-VSR	STAR	SeedVR2	DOVE	DLoRAL	PiSA-SR	AdcSR	HYPIR	AdcVSR (Ours)
Test Dataset: SPMCS (Synthetic)
PSNR↑	20.69	19.83	20.83	20.32	20.89	20.94	20.35	20.12	20.14	18.13	20.83
SSIM↑	0.5056	0.4301	0.5201	0.4897	0.5234	0.5221	0.4955	0.4674	0.4702	0.4284	0.5055
LPIPS↓	0.3886	0.4336	0.3583	0.5657	0.3122	0.3165	0.3531	0.3662	0.3695	0.3969	0.3282
DISTS↓	0.2334	0.2466	0.2198	0.2957	0.1823	0.1896	0.2164	0.2286	0.2304	0.2251	0.2036
MANIQA↑	0.5977	0.5862	0.5941	0.3654	0.6289	0.5774	0.6066	0.6561	0.6332	0.6961	0.6500

Appendix

Table 8: Quantitative comparison of Real-VSR performance on 4 additional benchmark datasets.

Panel of Figure 8 — Figure 8: Qualitative comparison of Real-VSR performance on three real videos: the 4th frame of “029” (top), the 2nd frame of “090” (middle), and the 15th frame of “247” (bottom) from RealVSR.

Diffusion Model Compression

See all 27 papers

Streaming-Aware Diffusion for Real-Time Video Super-Resolution via Cross-Step Attention
Oct 8, 2026Harris Partaourides, Sotirios ChatzisVideo Super-Resolution Diffusion Model Inference Acceleration
Ethical AI Novelties Limassol, Cyprus · Cyprus University of Technology Limassol, Cyprus
TinySR: Pruning Diffusion for Real-World Image Super-Resolution
Aug 24, 2025Linwei Dong, Qingnan Fan, Yuhang Yu +4Diffusion Models for Image Restoration Diffusion Model Pruning
Zhejiang University · Vivo Mobile Communication Co. Ltd · Zhejiang Lab
InstaVSR: Taming Diffusion for Efficient and Temporally Consistent Video Super-Resolution
Mar 27, 2026Jintong Hu, Bin Chen, Zhenyu Hu +3Video Super-Resolution Diffusion Models
Insta360 Research · Peking University · Wuhan University

Oct 8, 2026J/K move · Enter open · S save

Streaming-Aware Diffusion for Real-Time Video Super-Resolution via Cross-Step Attention

Harris Partaourides, Sotirios Chatzis

Ethical AI Novelties Limassol, Cyprus · Cyprus University of Technology Limassol, Cyprus

Video Super-Resolution Diffusion Model Inference Acceleration Temporal Consistency in Video Generation Diffusion Timestep Scheduling

Open paper AskPDF arXiv

Real-time video super-resolution requires high spatio-temporal fidelity under strict latency constraints, challenging diffusion models due to their iterative sampling cost and limited temporal coordination. We propose a streaming-aware framework that adapts pretrained single-image latent diffusion models for efficient video super-resolution (VSR) by exploiting the sequential structure of video streams. Our Cross-Step Attention mechanism reuses intermediate denoising features across adjacent frames and diffusion steps, enabling temporal information exchange without explicit temporal modeling. We further introduce Trajectory-Coupled Diffusion Scheduling, which aligns adjacent diffusion states and provides cleaner intermediate representations for cross-step conditioning, improving temporal coherence. These components are integrated into a streaming inference pipeline that incrementally propagates latent states across frames, reducing the effective computational complexity from

O(N \cdot S)

O(N + S)

for

N

frames and

S

diffusion steps. Experiments on REDS4 and YouHQ40-Test demonstrate improved perceptual quality and temporal realism while maintaining frame-wise stability. Our method achieves over 40 FPS at

512 \times 512

resolution after cold start, enabling real-time VSR without explicit temporal modeling.

Abstract

Diffusion Model Compression

Explore similar work

Streaming-Aware Diffusion for Real-Time Video Super-Resolution via Cross-Step Attention

TinySR: Pruning Diffusion for Real-World Image Super-Resolution

InstaVSR: Taming Diffusion for Efficient and Temporally Consistent Video Super-Resolution