High-Throughput Generative Video Super-Resolution on a Single Consumer GPU

Georgios Panaretou; Sotirios Chatzis

cs.CVOct 9, 2026

High-Throughput Generative Video Super-Resolution on a Single Consumer GPU

Authors: Georgios Panaretou, Sotirios Chatzis

Organizations: Ethical AI Novelties Limassol, Cyprus · Cyprus University of Technology Limassol, Cyprus

Abstract

Generative video super-resolution (VSR) can recover visually compelling high-frequency detail by leveraging diffusion priors, but iterative sampling and heavy spatio-temporal coupling make practical deployments difficult. We study a different operating point: practical HD generative VSR on a single high-end consumer GPU without end-to-end retraining. To this end, we present a zero-shot framework built entirely from pretrained components. The proposed system decouples spatial enhancement from temporal stabilization. A time-distilled one-step diffusion backbone handles spatial upscaling, while temporal coherence is promoted by a lightweight pretrained flow-based interpolator that synthesizes intermediate frames between super-resolved keyframes. To further improve scalability, we introduce a resolution-aware hybrid attention scheme that, in a zero-shot manner, replaces selected high-resolution global-attention layers with neighborhood attention while preserving global attention in semantically critical low-resolution layers, while also improving temporal consistency. Experiments show that the proposed framework reaches up to ~32 FPS at 720p resolution under interleaved inference on a single NVIDIA RTX 4090. At 4K, the proposed attention substitution can yield up to a 44% speedup over the original attention implementation. Across standard VSR benchmarks, the method consistently shifts the speed-quality trade-off decisively toward runtime efficiency, trading reconstruction fidelity for strong no-reference perceptual quality, while remaining reasonably competitive against generative approaches in temporal consistency. It also exposes clear limitations in handling more challenging motion, which a simple motion gate partially mitigates. Code will be released upon acceptance.

Figures & tables

Figure 1: Our Video Super-Resolution compared to bicubic interpolation on consecutive frames. $K$ denotes the super-resolution frequency; in-between frames are interpolated. $L$ denotes the attention blocks replaced with NATTEN.

Panel of Figure 2 — Figure 2: Proposed Approach. The system reuses two pretrained assets, namely AdcSR for sparse keyframe super-resolution and RIFE for temporal interpolation. At the same time, the original contribution of this work lies in how these components are combined: a motion-aware interleaved keyframe policy controls when the generative backbone is invoked, zero-shot hybrid attention accelerates the high-resolution layers of the spatial generator, and temporal assembly merges super-resolved keyframes with interpolated intermediate frames.

$K$	$B$	Memory (GB)	FPS
1	1	2.03	3.04
1	8	8.94	16.76
2	1	2.23	5.69
2	8	8.98	25.64
3	1	2.33	8.28
3	8	8.99	32.66

Table 1: Throughput and memoryof the proposed pipeline at 1280x720 resolution for different keyframe intervals

K

and batch sizes

B

Method	Memory (GB)	FPS
Non-Diffusion
RealBasicVSR	4.6	45.53
RVRT	7.1	1.97
Diffusion-based
DLoRAL	8.5	0.15
DiffIR2VR-Zero	9.5	0.08

Table 2: Single-batch HD (1280x720) inference on the same GPU. Peak memory is reported by PyTorch.

L{=}3

for all runs.

Dataset	$L$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	BRISQUE $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	$E_{\text{warp}}\downarrow$	tOF $\downarrow$
UDM10	0	21.94	0.7879	0.3731	0.1301	11.62	71.65	0.5187	0.0279	1.1653
UDM10	2	23.21	0.8075	0.3733	0.1203	10.57	68.12	0.4487	0.0228	0.7843
SPMCS	0	19.84	0.5468	0.4671	0.2006	14.92	67.75	0.4363	0.0364	1.1553
SPMCS	2	20.33	0.5443	0.4793	0.2288	13.62	63.96	0.3831	0.0315	0.9226
Vid4	0	18.17	0.6116	0.4290	0.1535	6.50	70.74	0.4397	0.0432	0.2589
Vid4	2	18.92	0.6161	0.4288	0.1541	7.42	68.73	0.3936	0.0387	0.2754

Table 3: Effect of

L{=}2

K{=}1

against baseline (

L{=}0

). Best value per dataset and metric is in bold .

Resolution	B	FA FPS	NATTEN FPS	Change
1280 $\times$ 720	1	3.01	2.98	-1.0%
	8	15.30	16.76	+9.1%
1920 $\times$ 1080	1	3.06	3.00	-1.8%
	4	6.20	7.31	+18.0%
2560 $\times$ 1440	1	2.73	3.49	+27.9%
3840 $\times$ 2160	1	0.76	1.10	+44.7%

Table 4: Inference speed with NATTEN relative to PyTorch FlashAttention for

K{=}1

and

L{=}3

							Ours
	Metric	RealBasicVSR	RVRT	DLoRAL	DiffIR2VR-Zero	SeedVR2	$K=1$ , $L=2$	$K=2$ , $L=2$	$K=3$ , $L=2$
UDM10 $\times$ 4	PSNR $\uparrow$	29.389	33.279	20.957	24.929	29.470	23.206	23.208	23.111
	SSIM $\uparrow$	0.937	0.985	0.694	0.854	0.939	0.807	0.805	0.801
	LPIPS $\downarrow$	0.253	0.154	0.325	0.389	0.214	0.373	0.368	0.368
	BRISQUE $\downarrow$	21.295	49.080	29.212	22.888	25.090	10.569	13.694	13.767
	MUSIQ $\uparrow$	65.784	59.387	62.180	67.046	65.900	68.123	67.676	67.555

Table 5: Quantitative comparison on UDM10, SPMCS, REDS4, and Vid4 for

\times 4

super-resolution. Best results are marked in bold , second-best in italic green .

K

is the interpolation interval,

L

are the attention layers replaced with NATTEN.

$K$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	BRISQUE $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	$E_{\text{warp}}\downarrow$	tOF $\downarrow$	LPIPS ${}_{\text{inter}}\downarrow$
2 (Ungated)	20.440	0.713	0.390	16.080	68.190	0.427	0.051	4.088	0.364
2 (Gated)	21.500	0.796	0.368	13.410	68.990	0.445	0.034	1.079	0.370
3 (Ungated)	19.570	0.649	0.410	15.920	67.920	0.418	0.056	5.226	0.364
3 (Gated)	21.570	0.804	0.367	12.950	69.120	0.449	0.034	0.848	0.372

Table 6: Super-resolution quality and temporal-consistency metrics on REDS4 under the adaptive motion-gated interpolation schedule with

L{=}2

and key-frame intervals

K\in\{2,3\}

. Fallback rates for

K{=}2

are 74% and 90% for

K{=}3

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

$L$	stage	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	BRISQUE $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	$E_{\text{warp}}\downarrow$	tOF $\downarrow$
0	baseline	21.9362	0.7879	0.3731	0.1301	11.6197	71.6498	0.5187	0.0279	1.1653
1	up	22.4926	0.7995	0.3705	0.1237	11.4900	70.3318	0.4915	0.0263	1.0258
1	down	22.0091	0.7777	0.3883	0.1342	11.3235	70.5214	0.4881	0.0266	0.7196
2	up	23.2063	0.8075	0.3733	0.1203	10.5693	68.1227	0.4487	0.0228	0.7843
2	down	22.3193	0.7728	0.4028	0.1457	12.8239	68.5334	0.4450	0.0245	0.6884

Appendix

Table 7: NATTEN attention replacement in the UNet’s DownSampling (encoder) vs. UpSampling (decoder) stage, head-to-head at matched layer count

L

, on the full UDM10 dataset (10 clips) with

K{=}1

Panel of Figure 3 — Figure 3: UDM10 clip 000 : comparison on replacing NATTEN on encoder vs decoder of UNet.

Dataset	$K$	$L$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	BRISQUE $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	LPIPS ${}_{\text{inter}}\!\downarrow$	$E_{\text{warp}}\downarrow$	tOF $\downarrow$
UDM10	1	0	21.94	0.7879	0.3731	0.1301	11.62	71.65	0.5187	0.2155	0.0279	1.1653
		1	22.49	0.7995	0.3705	0.1237	11.49	70.33	0.4915	0.2190	0.0263	1.0258
		2	23.21	0.8075	0.3733	0.1203	10.57	68.12	0.4487	0.2200	0.0228	0.7843
		3	23.29	0.8085	0.3806	0.1263	12.80	68.17	0.4518	0.2209	0.0245	0.9951
	2	0	21.97	0.7862	0.3689	0.1297	14.33	71.49	0.5095	0.2003	0.0205	0.6811
		1	22.52	0.7976	0.3660	0.1231	14.36	70.08	0.4813	0.2033	0.0190	0.6257

Appendix

Table 8: Ablation on the number of NATTEN-replaced attention layers (

L

) on UDM10, across

K

. Best value per

K

group and metric is in bold .

\uparrow

(

\downarrow

) indicates that higher (lower) is better.

Figure 5: UDM10 clip 000 : qualitative effect of NATTEN layer count $L$ .

Figure 6: SPMCS clip AMVTG_004 : qualitative effect of NATTEN layer count $L$ .

Figure 7: REDS4 clip 000 : qualitative effect of NATTEN layer count $L$ .

Figure 8: Vid4 clip calendar : qualitative effect of NATTEN layer count $L$ .

$K$	stat	$\tau_{gate}$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	BRISQUE $\downarrow$	MUSIQ $\uparrow$	MANIQA $\uparrow$	$E_{\text{warp}}\downarrow$	tOF $\downarrow$	LPIPS ${}_{\text{inter}}\downarrow$	Fallback% $\downarrow$	FPS $\uparrow$
1	–	–	21.59	0.8058	0.3664	0.1074	12.76	69.23	0.4520	0.0349	0.8184	0.3756	n/a	5.83
2	–	ungated	20.44	0.7133	0.3904	0.1126	16.08	68.19	0.4273	0.0510	4.0881	0.3640	0.0	10.83
	mean	$\tau_{gate}{=}4$ (strict)	21.52	0.7988	0.3681	0.1083	13.50	68.90	0.4429	0.0332	1.3015	0.3695	68.0	6.84
	max	$\tau_{gate}{=}15$ (strict)	21.50	0.7960	0.3683	0.1082	13.41	68.99	0.4449	0.0341	1.0788	0.3711	74.0	6.62
	mean	$\tau_{gate}{=}6$ (mid)	21.21	0.7730	0.3744	0.1097	14.25	68.65	0.4373	0.0380	2.1588	0.3648	45.0	7.81
	max	$\tau_{gate}{=}20$ (mid)	21.23	0.7746	0.3736	0.1094	14.06	68.81	0.4406	0.0382	1.7905	0.3680	56.0	7.31

Appendix

Table 9: Max-stat vs. mean-stat motion score compared head-to-head on REDS4, at fixed

L{=}2

, across key-frame intervals

K\in\{2,3\}

. Fallback % is the share of frames that exceeded

\tau_{gate}

and fell back to per-frame SR instead of interpolating. Batch size = 8.

Figure 9: REDS4 clip 000 : frame-level comparison against baselines.

Figure 10: REDS4 clip 020 : frame-level comparison against baselines.

Figure 11: SPMCS clip AMVTG_004 : frame-level comparison against baselines.

Figure 12: SPMCS clip hdclub_003_001 : frame-level comparison against baselines.

Figure 13: UDM10 clip 000 : frame-level comparison against baselines.

Figure 14: UDM10 clip 004 : frame-level comparison against baselines.

Figure 15: Vid4 clip city : frame-level comparison against baselines.

Figure 16: Vid4 clip calendar : frame-level comparison against baselines.

Video Super-Resolution

Video super-resolution reconstructs sharper, higher-resolution video from low-resolution frames while preserving detail and temporal consistency.

See all 36 papers

Mar 27, 2026

InstaVSR: Taming Diffusion for Efficient and Temporally Consistent Video Super-Resolution

Video super-resolution (VSR) seeks to reconstruct high-resolution frames from low-resolution inputs. While diffusion-based methods have substantially improved perceptual quality, extending them to video remains challenging for two reasons: strong generative priors can introduce temporal instability, and multi-frame diffusion pipelines are often too expensive for practical deployment. To address both challenges simultaneously, we propose InstaVSR, a lightweight diffusion framework for efficient video super-resolution. InstaVSR combines three ingredients: (1) a pruned one-step diffusion backbone that removes several costly components from conventional diffusion-based VSR pipelines, (2) recurrent training with flow-guided temporal regularization to improve frame-to-frame stability, and (3) dual-space adversarial learning in latent and pixel spaces to preserve perceptual quality after backbone simplification. On an NVIDIA RTX 4090, InstaVSR processes a 30-frame video at 2K

\times

2K resolution in under one minute with only 7 GB of memory usage, substantially reducing the computational cost compared to existing diffusion-based methods while maintaining favorable perceptual quality with significantly smoother temporal transitions.

Jintong Hu, Bin Chen, Zhenyu Hu +3

Insta360 Research · Peking University · Wuhan University

Video Super-Resolution Diffusion Models Diffusion Model Inference Acceleration

Ask PDF

Sep 29, 2026

RelayVSR: Large-Small Model Collaboration for Efficient Real-World Video Super-Resolution

Large generative models can recover realistic detail in real-world video super-resolution (VSR), but processing an entire video with them is computationally expensive. In this work, we present RelayVSR, a streaming VSR framework built on the Sparse Generative Relay mechanism. A large generative model generates reference latents for sparse keyframes, while a lightweight VSR network uses these references and low-resolution video to super-resolve every frame. The lightweight VSR network, implemented as a Dual-Memory Video Transformer, reuses keyframe information across frames and updates recent video context, supporting first-keyframe conditioning and dual-endpoint conditioning with bounded lookahead. However, errors in shared keyframes can propagate and accumulate across output frames, making keyframe quality alone an insufficient optimization target. We address this collaboration gap with Video-Aware Reference Optimization (VARO), which uses reinforcement learning to update the large generative model with two reward levels: a system-level reward evaluates videos produced by the fixed lightweight VSR network, while a reference-level reward evaluates decoded keyframe quality. VARO improves final video quality over direct joint training, and its dual-level rewards outperform a system-level reward alone. At 1080p on a single NVIDIA A100 80GB, dual-endpoint RelayVSR with a 15-frame keyframe interval reaches 29.29 FPS, 13.82 GB peak GPU memory, and 0.327 s first-frame model latency, compared with 7.80 FPS, 24.447 GB, and 2.83 s for FlashVSR-Tiny. The code is available at https://github.com/kopperx/RelayVSR.

Xijun Wang, Xin Li, Zirui Lang +3

University of Science and Technology of China

Video Super-Resolution

Ask PDF

Oct 13, 2026

FlashVSR+: Efficient Video Super-Resolution via Hybrid Attention and Reward-Yield Distribution Matching

Recent video super-resolution (VSR) models achieve efficient one-step generation through diffusion distillation. However, high-resolution inference remains costly, and distilled models are difficult to align with human preferences. These challenges call for improvements in both model architecture and training strategy. We present FlashVSR+ to address both limitations. On the architecture side, we introduce hybrid attention based on the strong locality observed in pretrained VSR models, while retaining sparse global interaction for long-range consistency. It achieves up to

2\times

DiT speedup and 45% lower peak memory, enabling spatially untiled 4K VSR on a single 24GB RTX 4090. On the training side, we propose Reward-Yield Distribution Matching Distillation (Ry-DMD) for preference-aligned one-step generation. Ry-DMD uses reward-weighted teacher rollouts to incorporate black-box rewards into distribution matching, without backpropagating through the reward model. We further show that Ry-DMD generalizes beyond VSR to T2I generation. Together, FlashVSR+ delivers stronger restoration quality and substantially improves high-resolution efficiency, with consistent gains in user preference. We release our code, models, and training data at \href{https://www.flashvsr.tech}{\textcolor{FlashBlue}{**www.flashvsr.tech**}}.

Ruchang Yao, Shi Guo, Junhao Zhuang +7

Shanghai AI Lab · MMLAB CUHK · Tsinghua University +1

Video Super-Resolution Video Diffusion Models Diffusion Model Distillation Diffusion Model Alignment Diffusion Transformer

Ask PDF

Abstract

Video Super-Resolution

Explore similar work

InstaVSR: Taming Diffusion for Efficient and Temporally Consistent Video Super-Resolution

RelayVSR: Large-Small Model Collaboration for Efficient Real-World Video Super-Resolution

FlashVSR+: Efficient Video Super-Resolution via Hybrid Attention and Reward-Yield Distribution Matching