Video super-resolution (VSR) seeks to reconstruct high-resolution frames from low-resolution inputs. While diffusion-based methods have substantially improved perceptual quality, extending them to video remains challenging for two reasons: strong generative priors can introduce temporal instability, and multi-frame diffusion pipelines are often too expensive for practical deployment. To address both challenges simultaneously, we propose InstaVSR, a lightweight diffusion framework for efficient video super-resolution. InstaVSR combines three ingredients: (1) a pruned one-step diffusion backbone that removes several costly components from conventional diffusion-based VSR pipelines, (2) recurrent training with flow-guided temporal regularization to improve frame-to-frame stability, and (3) dual-space adversarial learning in latent and pixel spaces to preserve perceptual quality after backbone simplification. On an NVIDIA RTX 4090, InstaVSR processes a 30-frame video at 2K×2K resolution in under one minute with only 7 GB of memory usage, substantially reducing the computational cost compared to existing diffusion-based methods while maintaining favorable perceptual quality with significantly smoother temporal transitions.
Figures & tables
Figure 1: Visual and quantitative comparison of video super-resolution methods on real-world content. Left: super-resolution results showing the low-quality input and the outputs of different methods, with zoomed-in patch comparisons. Right: comparison of computational efficiency (latency, parameters, and VRAM) and visual quality metrics. Our method achieves a strong trade-off between efficiency and perceptual quality compared with RealViformer [ 62 ] and SeedVR2 [ 47 ] .
Figure 2: The main pipeline of GaussianSR. GaussianSR begins with an encoder that extracts feature representations from the input image, followed by Selective Gaussian Splatting which assigns a learnable Gaussian kernel to each pixel, converting dicrete feature points into Gaussian fields. Features at any arbitrary query point xq in the plane are computed using the overlapping Gaussian functions that modulate their influence based on the spatial location. Finally, these continuous-domain features are rendered into a high-resolution space and refined through the decoder to reconstruct the desired RGB output at the specified query coordinates.
Figure 3: Recurrent VSR Training with Progressive Super Resolution and Temporal Loss. Center frames are progressively super-resolved via 5-frame sliding window optical flow alignment, with SR frames replacing LR frames for continuous buffer update. Cyclic inference generates second-order SR results, and temporal loss between adjacent SR frames enforces recurrent training consistency.
Dataset
Method
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
FID ↓
NIQE ↓
MANIQA ↑
MUSIQ ↑
Ewarp↓
Etc↓
DOVER ↑
RealESRGAN [ 50 ]
26.0692
0.7715
0.4339
0.2147
95.7204
3.472
0.6088
69.2294
3.1010
5.9872
71.9639
RealViformer [ 62 ]
23.4239
0.6150
0.2144
0.1500
49.3322
3.793
0.6111
66.0845
2.3705
4.5062
65.7693
RealBasicVSR [ 6 ]
22.8704
0.6084
0.2076
0.1278
48.6680
3.292
0.6249
69.4399
2.8075
5.7612
69.9027
AdcSR [ 7 ]
22.4068
0.5952
0.2553
0.1533
71.9473
3.500
0.6382
72.0246
3.9597
6.4241
71.5490
UpscaleAVideo [ 65 ]
23.7654
0.6312
0.3420
0.1995
95.4628
4.821
0.4886
53.8774
2.7397
3.9518
52.0868
DOVE [ 9 ]
20.2957
0.5064
0.2192
0.1287
56.5175
4.463
0.6320
70.1333
2.2717
5.7750
48.3011
Table 1: Quantitative comparison on the SPMCS and YouHQ datasets. The best, second best, and third best results in the columns marked in red , blue , and green respectively.
AdcSR
UAV
DOVE
STAR
DLoRAL
Ours
Params (B)
0.46
14.44
10.55
2.49
1.30
0.45
Time (min)
1.3/1.0*
25
1.4
24.5
3.5
1.0/0.77*
Memory (GB)
17.23
38.85
31.79
75.25
24.93
7.10
Table 2: Comparison of model complexity, inference time, and GPU memory usage for Generating a 30-frame 2K2 Video.
Figure 4: Qualitative comparisons on real-world videos. Our method is capable of generating more realistic and fine-grained details. Compared to existing methods, it notably excels in restoration quality: for the stone-structured building (first row), our method produces more delicate and authentic stone textures; for the red bricks and leaves (second row), our method generates well-aligned and structurally regular patterns that better conform to human visual preferences. (Zoom-in for best view).
Figure 5: Temporal Profiles of Video Reconstruction: Comparison of Our Method with Baselines. Our method yields smooth profiles with well-preserved structural textures, especially straight architectural lines. In contrast, competing methods either remove these features or introduce geometric distortions, producing jagged patterns. This highlights our method’s advantage in maintaining both temporal consistency and geometric fidelity. (Zoom-in for best view).
Configuration
NIQE ↓
MANIQA ↑
CLIPIQA ↑
MUSIQ ↑
DOVER ↑
Full (Ours)
3.441
0.678
0.555
72.31
72.55
w/o Dlatent
+0.202
+0.002
− 0.033
− 0.16
+0.83
w/o Dpixel
+0.001
− 0.023
− 0.023
− 2.20
− 2.63
Dpixel→UNet
− 0.033
− 0.011
+0.003
− 0.57
+0.69
Table 3: Ablation study on discriminator design. “Full” reports the absolute metric values of our model equipped with both Dlatent and Dpixel . Other rows show the difference ( Δ ) from Full; red and teal denote performance degradation and improvement, respectively.
Figure 6: Qualitative results of the ablation study on the pixel discriminator and latent discriminator.
Figure 7: Qualitative results of the ablation study on the recurrent training and temporal loss.
Diffusion-based models have shown strong performance in video super-resolution (VSR) and video frame interpolation (VFI). However, their role in the coupled space-time video super-resolution (STVSR) setting remains limited. Existing diffusion-based STVSR approaches suffer from two issues: (1) low inference efficiency and (2) insufficient utilization of spatiotemporal information. These limitations impede deployment. To address these issues, we introduce DiffST, an efficient spatiotemporal-aware video diffusion framework for real-world STVSR. To improve efficiency, we adapt a pre-trained diffusion model for one-step sampling and process the entire video directly rather than operating on individual frames. Furthermore, to enhance spatiotemporal information utilization, we introduce cross-frame context aggregation (CFCA) and video representation guidance (VRG). The CFCA module aggregates information across multiple keyframes to produce intermediate frames. The VRG module extracts video-level global features to guide the diffusion process. Extensive experiments show that DiffST obtains leading results on real-world STVSR tasks. It also maintains high inference efficiency, running about 17× faster than previous diffusion-based STVSR methods. Code is available at: https://github.com/zhengchen1999/DiffST.
Zheng Chen, Ruofan Yang, Jin Han +5
Shanghai Jiao Tong University · Huawei Noah’s Ark Lab · Duke University +1
Real-time video super-resolution requires high spatio-temporal fidelity under strict latency constraints, challenging diffusion models due to their iterative sampling cost and limited temporal coordination. We propose a streaming-aware framework that adapts pretrained single-image latent diffusion models for efficient video super-resolution (VSR) by exploiting the sequential structure of video streams. Our Cross-Step Attention mechanism reuses intermediate denoising features across adjacent frames and diffusion steps, enabling temporal information exchange without explicit temporal modeling. We further introduce Trajectory-Coupled Diffusion Scheduling, which aligns adjacent diffusion states and provides cleaner intermediate representations for cross-step conditioning, improving temporal coherence. These components are integrated into a streaming inference pipeline that incrementally propagates latent states across frames, reducing the effective computational complexity from O(N⋅S) to O(N+S) for N frames and S diffusion steps. Experiments on REDS4 and YouHQ40-Test demonstrate improved perceptual quality and temporal realism while maintaining frame-wise stability. Our method achieves over 40 FPS at 512×512 resolution after cold start, enabling real-time VSR without explicit temporal modeling.
Harris Partaourides, Sotirios Chatzis
Ethical AI Novelties Limassol, Cyprus · Cyprus University of Technology Limassol, Cyprus
Diffusion-based video super-resolution (VSR) methods deliver strong perceptual quality but are often unsuitable for latency-sensitive scenarios due to reliance on future frames and expensive multi-step denoising. We propose Stream-DiffVSR, a causally conditioned diffusion framework for efficient online VSR. Operating strictly on past frames, Stream-DiffVSR integrates a four-step distilled denoiser for fast inference, an Auto-regressive Temporal Guidance (ARTG) module that injects motion-aligned cues during latent denoising, and a lightweight temporal-aware decoder with a Temporal Processor Module (TPM) to enhance detail and temporal coherence. Unlike chunk-wise streaming inference, our strictly frame-by-frame causal design avoids sequence-level waiting, substantially reducing time-to-first-frame and end-to-end latency. Stream-DiffVSR processes 720p frames in 0.328 seconds on an RTX 4090 and consistently outperforms prior diffusion-based baselines. Compared with the online state-of-the-art TMP, it improves perceptual quality (LPIPS +0.095). Compared with prior diffusion-based VSR methods such as MGLD-VSR, it reduces per-frame runtime by over 130x. Moreover, Stream-DiffVSR substantially lowers time-to-first-frame for diffusion-based VSR, reducing initial delay from over 4600 seconds to 0.328 seconds, making diffusion-based VSR markedly more practical for low-latency online and streaming deployment. Project page: https://jamichss.github.io/stream-diffvsr-project-page/
Hau-Shiang Shiu, Chin-Yang Lin, Zhixiang Wang +4
National Yang Ming Chiao Tung University · Shanda AI Research Tokyo · MediaTek Inc.