Real-time streaming video generation has gained rapid momentum in live content creation and interactive media. Deploying such systems in practice requires flexible scaling factors to accommodate diverse source resolutions, target displays, and computational budgets. However, extending fixed-scale generative video super-resolution (VSR) to varying target scales poses two challenges: downsampling from a larger fixed scale incurs redundant cost, while re-optimizing the diffusion transformer (DiT) is costly and alters the shared generative prior. To address both, we propose FlexVSR, a decoupled framework for continuous-scale streaming generative VSR with supervision at only a few scales. Our key insight is that naively resampling fixed-scale latents onto a new target grid induces scale-dependent input mismatch, which can be effectively addressed through parameter-efficient, scale-aware adaptation of the latent interface without updating the DiT. To this end, FlexVSR introduces ReLSA (Refined Latent Scale Adaptation), a lightweight latent adapter that constructs and refines target-grid latents before they enter the DiT, enabling a shared frozen VSR prior to generalize across continuous scales with only 0.274M trainable parameters and an adaptation cost of 7.3 H100 GPU-hours, while introducing no additional temporal dependencies and preserving the native causal streaming protocol. Extensive experiments demonstrate competitive or superior reconstruction and temporal quality, lower latency than most native-scale streaming baselines and offline methods, and effective generalization to unseen continuous scales, filling the continuous-scale gap in real-time streaming generative VSR.
Figures & tables
Figure 2: Overview of FlexVSR. Left: Stage 1 prepares a 2× streaming SR prior via latent velocity regression (dashed arrow before Dω denotes inference-only decoding). Stage 2 freezes the restorer and feature extractor Fφ , replacing the 2× head with ReLSA to adapt latents across scales under discrete 1.5× / 2× supervision (latent warm-up followed by end-to-end latent+RGB training). Right: Architecture of the target-grid Residual Refiner ( Rθ ) , zero-initialized at the output so Δz(s)=0 initially.
Dataset
Type
Method (venue)
1.5×
2×
Avg. Rank ↓
PSNR ↑
LPIPS ↓
tLPIPS ↓
PSNR ↑
LPIPS ↓
tLPIPS ↓
LiteVSR N [ 5 ] (ICML 2026)
26.79
0.1943
0.01038
25.97
0.2200
0.01187
2.83
Offline VSR methods
LiteVSR D (ICML 2026)
23.06
0.3576
0.01470
23.35
0.3392
0.01434
9.67
LUVE N [ 47 ] (ICML 2026)
16.64
0.4398
0.02030
16.32
0.4540
0.02222
12.00
VEnhancer N [ 16 ] (arXiv 2024)
19.90
0.3274
0.01736
20.15
0.3254
0.01798
10.50
FlashVSR-T N [ 49 ] (CVPR 2026)
26.51
0.1723
0.01440
25.72
0.1851
0.01421
4.33
Table 1: Quantitative comparison with generative VSR systems. Best and second-best scores are bold and underlined .
Method
1.5×
2×
T13↓
t16↓
FPS ↑
T13↓
t16↓
FPS ↑
FlashVSR-T N
10.335
6.408
2.50
5.678
3.588
4.46
FlashVSR-T D
2.353
1.870
8.55
1.890
1.411
11.34
FlashVSR-F N
18.994
14.105
1.13
9.564
6.188
2.59
FlashVSR-F D
3.402
2.563
6.24
2.948
2.138
7.48
InfVSR N
97.299
89.151
0.18
57.139
52.347
0.31
Table 2: Streaming SR efficiency from LR latents (w/ required decoding/resizing). Latency (in s) uses t16=(T77−T13)/4 .
Metric
Method
1.2×
1.25×
1.3×
1.4×
1.5×
1.6×
1.7×
1.75×
1.8×
2×
Actual (H/W)
1.20/1.20
1.25/1.25
1.29/1.30
1.40/1.40
1.50/1.50
1.61/1.60
1.69/1.69
1.74/1.74
1.78/1.79
2.00/2.00
FlashVSR-T N
27.19
27.08
26.95
26.77
26.51
26.33
26.14
26.08
25.99
25.72
FlashVSR-T D
23.64
23.64
23.64
23.65
23.65
23.66
23.66
23.66
23.66
23.67
PSNR ↑
InfVSR N
24.21
24.24
24.29
24.42
24.47
24.53
24.57
24.59
24.62
24.72
InfVSR D
24.53
24.56
24.60
24.66
24.72
24.78
24.82
24.83
24.85
24.92
Ultra Flash N
25.80
25.65
25.54
25.29
25.10
24.85
24.71
24.60
24.52
24.14
Table 3: Cross-scale comparison on YouHQ40 (supervised at 1.5×/2× , matching Table 1 ; Actual (H/W) lists ratios for the 37 landscape clips). Best/second-best scores are bold / underlined .
Method
2.1×
2.2×
PSNR ↑
LPIPS ↓
tLPIPS ↓
PSNR ↑
LPIPS ↓
tLPIPS ↓
FlashVSR-T N
25.52
0.1897
0.01404
25.44
0.1911
0.01386
FlashVSR-T D
23.68
0.2207
0.01416
23.68
0.2206
0.01416
InfVSR N
24.79
0.2471
0.01572
24.84
0.2462
0.01570
InfVSR D
24.93
0.2238
0.01446
24.94
0.2242
0.01453
Ultra Flash N
18.72
0.2227
0.01420
18.63
0.2284
0.01440
Table 4: Above-range extrapolation ( 2.1× – 2.2× ) on YouHQ40 without retraining (conventions follow Table 3 ).
Variant
Settings
1.2× (unseen)
1.5×
1.8× (unseen)
2×
Q
R
E2E
SR
PSNR ↑
LPIPS ↓
DISTS ↓
PSNR ↑
LPIPS ↓
DISTS ↓
PSNR ↑
LPIPS ↓
DISTS ↓
PSNR ↑
LPIPS ↓
DISTS ↓
(a) Stage-2 adaptation data ablation (w/o VSR-17K for ReLSA; shared Stage-1 SR prior)
FlexVSR (Stage-2 REDS)
✓
✓
✓
✓
25.99
0.1469
0.1016
26.98
0.1174
0.0821
25.85
0.1539
0.1061
26.35
0.1448
0.1102
(b) Architecture ablations
w/o Query
×
✓
✓
✓
25.37
0.1713
0.1108
27.13
0.1132
0.0773
25.32
0.1518
0.1004
26.58
0.1320
0.0967
w/o Res
✓
×
✓
✓
26.40
0.1413
0.0851
26.63
0.1266
0.0838
25.77
0.1497
0.1046
25.93
0.1630
0.1164
Table 5: Ablation studies on YouHQ40 sharing the frozen Stage-1 Wan SR prior. Best and second-best scores are bold and underlined .
Figure 3: Residual-strength intervention on YouHQ40 (40 videos) with fixed FlexVSR weights, varying zα(s)=zbase(s)+αΔz(s) . At unseen 1.8× , pre-SR latent RMSE bottoms out at α=0.75 (a) whereas post-SR PSNR peaks at α=1 (b), confirming that Δz(s) calibrates inputs for frozen-DiT compatibility beyond mere latent proximity.
Table S2 : Fixed-LR results on YouHQ40. Best and second-best scores are bold and underlined .
Dataset
Scale
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
tLPIPS ↓
tOF ↓
YouHQ40
1.5×
26.38
0.7729
0.1421
0.0819
0.01406
0.4734
YouHQ40
2×
25.98
0.7551
0.1525
0.0864
0.01476
0.4945
Inter4K-40
1.5×
25.55
0.7747
0.1220
0.0790
0.00884
0.2093
Inter4K-40
2×
25.56
0.7726
0.1347
0.0823
0.00837
0.2190
Appendix
Table S3 : Complete quality results of FlexVSR + TAEHV (Ours). Full-frame post-SR RGB, 40 videos per dataset and scale, using the same LR inputs and original HR references as main-paper Table 1. Metrics are averaged over real frames per video, then equally over videos.
Model
YouHQ40, 1.5×
YouHQ40, 2×
Inter4K-40, 1.5×
Inter4K-40, 2×
PSNR ↑
LPIPS ↓
tLPIPS ↓
PSNR ↑
LPIPS ↓
tLPIPS ↓
PSNR ↑
LPIPS ↓
tLPIPS ↓
PSNR ↑
LPIPS ↓
tLPIPS ↓
FlexVSR (main)
27.25
0.1122
0.01244
26.79
0.1277
0.01242
26.17
0.1040
0.00723
26.38
0.1184
0.00801
FlexVSR-KV
27.30
0.1131
0.01178
26.84
0.1311
0.01187
26.59
0.0964
0.00764
26.46
0.1136
0.00838
FlexVSR + TAEHV
26.38
0.1421
0.01406
25.98
0.1525
0.01476
25.55
0.1220
0.00884
25.56
0.1347
0.00837
FlexVSR-KV + TAEHV
26.39
0.1397
0.01448
26.04
0.1519
0.01517
25.91
0.1153
0.00933
25.85
0.1303
0.00864
Appendix
Table S4 : Historical KV-cache variant: quality at the main-paper settings. Full-frame post-SR RGB, 40 videos per dataset and scale, with the LR inputs and HR references of main-paper Table 1. Within each decoder, the better value is bold.
Model
1.5×
2×
T13↓
t16↓
FPS ↑
T13↓
t16↓
FPS ↑
FlexVSR (main)
1.957
2.387
6.70
1.885
2.388
6.70
FlexVSR-KV
1.911
2.287
7.00
1.893
2.263
7.07
FlexVSR + TAEHV
0.389
0.674
23.73
0.385
0.668
23.95
FlexVSR-KV + TAEHV
0.407
0.562
28.44
0.403
0.558
28.69
Appendix
Table S5 : Historical KV-cache variant: runtime with the main-paper Table 2 protocol (one H100, 704×1280 output, final decoding included). T13 and t16 are in seconds. Within each decoder, the better value is bold.
Method
Mean metric rank ↓
Overall Avg. Rank ↓
Position
PSNR
LPIPS
tLPIPS
FlashVSR-T D
2.59
2.47
2.47
2.51
3
Ultra Flash N
2.35
2.35
2.29
2.33
2
FlexVSR (Ours)
1.06
1.18
1.24
1.16
1
Appendix
Table S6 : Wider-scale comparison with the two fastest streaming baseline configurations on YouHQ40: mean metric ranks over 17 scales ( 1.2× – 4× ), each with 40 videos and full-frame post-SR RGB. The overall rank weights all 51 scale–metric columns equally. Lower ranks are better; best and second-best values are bold and underlined . † marks scales unseen during wider-scale training in the following panels.
Figure S1 : Continuous-scale SR comparisons. All methods share the same upstream Wan2.1-1.3B-generated LR latent.
Figure S2 : Temporal and identity consistency at 1.5× on a produce-stall video, using the same input protocol as Fig. S1 .
DiT backbone
1.2× (unseen)
1.5×
1.8× (unseen)
2×
PSNR ↑
LPIPS ↓
tLPIPS ↓
PSNR ↑
LPIPS ↓
tLPIPS ↓
PSNR ↑
LPIPS ↓
tLPIPS ↓
PSNR ↑
LPIPS ↓
tLPIPS ↓
Wan2.1-1.3B (main)
26.42
0.1366
0.01151
27.25
0.1122
0.01244
26.20
0.1428
0.01048
26.79
0.1277
0.01242
Wan2.1-14B
26.46
0.1326
0.01216
27.51
0.1072
0.01181
26.27
0.1387
0.01006
27.10
0.1219
0.01195
MiniMax-H3
23.47
0.2001
0.01429
23.86
0.1883
0.01409
22.61
0.2231
0.01470
23.74
0.2086
0.01487
Appendix
Table S7 : Cross-backbone applicability on YouHQ40: full-frame post-SR RGB quality against original HR RGB, averaged over 40 videos per scale. PSNR is in dB; tLPIPS uses the main-paper temporal protocol.
Method
RGB resize before the model
Subsequent processing
FlashVSR-T D [ 49 ]
PIL bicubic to (Ht,Wt) ; normalize to [−1,1] in BF16.
Reflect-pad right/bottom to multiples of 128; LQ projection, DiT, and TCDecoder; remove padding.
FlashVSR-F D [ 49 ]
Same RGB resize and padding as FlashVSR-T D .
Same target-grid LQ conditioning and DiT; Wan VAE HR decoder replaces TCDecoder.
InfVSR D [ 46 ]
Reshape each RGB chunk from BCTHW to (BT)CHW ; FP32 bicubic with antialiasing to (Ht,Wt) .
Restore the temporal layout, then Wan VAE encoding, causal DiT restoration, and Wan VAE decoding.
LiteVSR D [ 5 ]
FP32 bilinear resize of the TCHW RGB tensor to (Ht,Wt) ; align_corners=False ; normalize to [−1,1] .
Wan2.2 VAE encoding, five-step UniPC restoration with the released adapter, and VAE decoding.
Appendix
Table S8 : Exact input processing for D routes. Sizes are height × width. All start from the common LR RGB frames, quantized to uint8 as expected by the released video-input pipelines.
Method
1.5×
2×
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
tLPIPS ↓
tOF ↓
PSNR ↑
SSIM ↑
LPIPS ↓
DISTS ↓
tLPIPS ↓
tOF ↓
Offline VSR methods
LiteVSR N
26.79
0.8240
0.1943
0.1172
0.01038
0.3742
25.97
0.7805
0.2200
0.1279
0.01187
0.4115
LiteVSR D
23.06
0.6179
0.3576
0.1784
0.01470
0.5923
23.35
0.6334
0.3392
0.1712
0.01434
0.5772
LUVE N
16.64
0.4558
0.4398
0.1571
0.02030
0.9688
16.32
0.4302
0.4540
0.1651
0.02222
0.9889
VEnhancer N
19.90
0.5406
0.3274
0.1395
0.01736
0.7017
20.15
0.5525
0.3254
0.1377
0.01798
0.6844
Appendix
Table S9 : Six metrics on YouHQ40 for the configurations of main-paper Table 1. Full-frame post-SR RGB against original HR, 40 videos per scale. Best and second-best scores are bold and underlined .
Real-time diffusion-based video super-resolution (VSR) is in high demand for online streaming, yet stringent latency requirements often compromise generative fidelity. We propose ReCaVSR, a Wan2.2-based, one-step framework for streaming VSR that builds on two observations: recycled SR latents retain local temporal context, reducing the need for full historical Key-Value (KV) caches; and individual transformer layers benefit from distinct temporal scopes. ReCaVSR combines three complementary designs: (i) layer-wise cache routing with recycled SR latents: each DiT layer learns its KV-cache temporal scope under a cache budget and exports a static inference schedule, while recycled SR latents propagate local context by conditioning each new block on the model's own preceding predictions. (ii) Multi-Scope Query (MSQ) Discriminator: a compositional discriminator combining global, spatial-window, and temporal-tube feedback for holistic realism, local texture generation, and temporal stability. (iii) LR-conditioned adaptation of FlashDecoder: a VAE decoder that incorporates LR observations for efficient latent decoding. ReCaVSR enables streaming VSR without iterative sampling or full historical KV-cache materialization. Experiments on synthetic and real-world VSR benchmarks show better perceptual quality, temporal consistency, and streaming efficiency than representative VSR baselines. At 1080×1920 output resolution on a single NVIDIA A100-80GB, ReCaVSR achieves 21.20 FPS with 15.16 GB peak allocated GPU memory, running 2.72× faster while using 38.0% less peak allocated memory than FlashVSR Tiny. The code is available at https://github.com/kopperx/ReCaVSR.
Large generative models can recover realistic detail in real-world video super-resolution (VSR), but processing an entire video with them is computationally expensive. In this work, we present RelayVSR, a streaming VSR framework built on the Sparse Generative Relay mechanism. A large generative model generates reference latents for sparse keyframes, while a lightweight VSR network uses these references and low-resolution video to super-resolve every frame. The lightweight VSR network, implemented as a Dual-Memory Video Transformer, reuses keyframe information across frames and updates recent video context, supporting first-keyframe conditioning and dual-endpoint conditioning with bounded lookahead. However, errors in shared keyframes can propagate and accumulate across output frames, making keyframe quality alone an insufficient optimization target. We address this collaboration gap with Video-Aware Reference Optimization (VARO), which uses reinforcement learning to update the large generative model with two reward levels: a system-level reward evaluates videos produced by the fixed lightweight VSR network, while a reference-level reward evaluates decoded keyframe quality. VARO improves final video quality over direct joint training, and its dual-level rewards outperform a system-level reward alone. At 1080p on a single NVIDIA A100 80GB, dual-endpoint RelayVSR with a 15-frame keyframe interval reaches 29.29 FPS, 13.82 GB peak GPU memory, and 0.327 s first-frame model latency, compared with 7.80 FPS, 24.447 GB, and 2.83 s for FlashVSR-Tiny. The code is available at https://github.com/kopperx/RelayVSR.
Generative video super-resolution (VSR) can recover visually compelling high-frequency detail by leveraging diffusion priors, but iterative sampling and heavy spatio-temporal coupling make practical deployments difficult. We study a different operating point: practical HD generative VSR on a single high-end consumer GPU without end-to-end retraining. To this end, we present a zero-shot framework built entirely from pretrained components. The proposed system decouples spatial enhancement from temporal stabilization. A time-distilled one-step diffusion backbone handles spatial upscaling, while temporal coherence is promoted by a lightweight pretrained flow-based interpolator that synthesizes intermediate frames between super-resolved keyframes. To further improve scalability, we introduce a resolution-aware hybrid attention scheme that, in a zero-shot manner, replaces selected high-resolution global-attention layers with neighborhood attention while preserving global attention in semantically critical low-resolution layers, while also improving temporal consistency. Experiments show that the proposed framework reaches up to ~32 FPS at 720p resolution under interleaved inference on a single NVIDIA RTX 4090. At 4K, the proposed attention substitution can yield up to a 44% speedup over the original attention implementation. Across standard VSR benchmarks, the method consistently shifts the speed-quality trade-off decisively toward runtime efficiency, trading reconstruction fidelity for strong no-reference perceptual quality, while remaining reasonably competitive against generative approaches in temporal consistency. It also exposes clear limitations in handling more challenging motion, which a simple motion gate partially mitigates. Code will be released upon acceptance.
Georgios Panaretou, Sotirios Chatzis
Ethical AI Novelties Limassol, Cyprus · Cyprus University of Technology Limassol, Cyprus