Transformer-based video diffusion models rely on 3D attention over spatial and temporal tokens, which incurs quadratic time and memory complexity and makes end-to-end training for ultra-high-resolution videos prohibitively expensive. To overcome this bottleneck, we propose a pure image adaptation framework that upgrades a video Diffusion Transformer pre-trained at its native scale to synthesize higher-resolution videos. Unfortunately, naively fine-tuning with high-resolution images alone often introduces noticeable noise due to the image-video modality gap. To address this, we decouple the learning objective to separately handle modality alignment and spatial extrapolation. At the core of our approach is Relay LoRA, a two-stage adaptation strategy. In the first stage, the video diffusion model is adapted to the image domain using low-resolution images to bridge the modality gap. In the second stage, the model is further adapted with high-resolution images to acquire spatial extrapolation capability. During inference, only the high-resolution adaptation is retained to preserve the video generation modality while enabling high-resolution video synthesis. To enhance fine-grained detail synthesis, we further propose a High-Frequency-Awareness-Training-Objective, which explicitly encourages the model to recover high-frequency components from degraded latent representations via a dedicated reconstruction loss. Extensive experiments demonstrate that our method produces ultra-high-resolution videos with rich visual details without requiring any video training data, even outperforming previous state-of-the-art models trained on high-resolution videos by 0.8 on the VBench benchmark. Code will be available at https://github.com/WillWu111/ViBe.
Figures & tables
Figure 2: The two plots show how training VRAM consumption and iteration time scale with the number of frames under different resolutions.
Figure 3: The plots on the left shows the general pipeline of our Relay LoRA. The two panels on the right provide a qualitative analysis on Wan2.2 [ 39 ] : naive high-resolution image fine-tuning (Naive LoRA) introduces noticeable artifacts, whereas ours methods (Relay LoRA) effectively removes them.
Figure 4: Upper Left : Stage-1 fine-tuning trains LoRA1 to adapt the DiT backbone to single low-resolution frame generation. Upper Middle : Stage-2 first merges LoRA1 into the base model and freezes the merged weights, then trains a Relay LoRA2 to adapt the DiT backbone to single high-resolution frame generation. Upper Right : During inference, only LoRA2 is loaded on the base model for video generation. Bottom Left : Global-Coarse-Local-Fine-Attention combines an inward sliding-window local attention with pooled coarse attention. Bottom Right : The training objective first degrades the latents via a downsample–upsample operation, then add noise on the degraded sample. The model applies the predicted flow and computes the loss against the clean latents.
Figure 5: Qualitative comparison. ViBe yields high-resolution videos characterized by high-fidelity details and coherent structure. The red boxes highlight regions with incorrect semantics or layout. The blue boxes provide zoomed-in views.
Resolution (W × H)
Model
Subject Consistency
Background Consistency
Motion Smoothness
Aesthetic Quality
Imaging Quality
Overall Consistency
Overall Score
1920 × 1088
Wan2.2 [ 39 ]
94.9%
95.4%
98.5%
59.1%
64.0%
23.3%
72.5%
ESRGAN [ 40 ]
95.3%
93.8%
96.9%
60.4%
63.6%
25.7%
72.6%
UAV [ 45 ]
95.2%
93.2%
96.8%
58.7%
62.5%
25.2%
71.9%
I-Max [ 8 ]
95.7%
95.3%
97.7%
60.2%
62.0%
24.0%
72.5%
HiFlow [ 3 ]
95.5%
95.6%
97.4%
60.1%
59.6%
25.4%
72.3%
CineScale [ 32 ]
93.9%
96.8%
97.4%
59.1%
60.7%
23.5%
71.9%
Table 1: Video comparison with both training-based and training-free models at 1080P and 4K resolutions. The highest value is bold , and the second-highest is underlined . Overall Score is computed as the mean of the six metrics in each row.
Figure 6: Results of user study for high-resolution video generation. Participants were expected to select the best method based on Text Alignment, Aesthetic Quality, and Video Quality.
Figure 7: Qualitative ablations. We perform controlled comparisons of our method with alternative variants under controlled settings. Best viewed zoomed in. The red boxes highlight regions with more noticeable noise, and the blue boxes indicate the corresponding zoomed-in views.
Methods
Subject Consistency
Background Consistency
Motion Smoothness
Aesthetic Quality
Imaging Quality
Overall Consistency
Overall Score
w/o Relay LoRA
94.6%
97.9%
98.3%
46.7%
37.4%
21.1%
66.0%
w/o GCLFA
94.9%
95.4%
99.0%
52.2%
49.0%
24.3%
69.1%
w/o HFATO
95.0%
94.2%
99.1%
53.7%
59.0%
24.0%
70.8%
w/o X0 Loss
96.3%
95.3%
99.2%
57.3%
59.2%
26.7%
72.3%
Ours (DR = 1/2)
95.2%
97.1%
99.2%
61.4%
66.1%
27.1%
74.4%
Ours (DR = 1/4)
95.0%
97.2%
99.3%
61.3%
65.6%
26.5%
74.2%
Table 2: Quantitative ablations. Ours Methods achieves the optimal overall video quality. The highest value is bold , and the second-highest is underlined .
Figure 8: Left : Illustration 4 steps generation results integrated with our framework. Right : Illustration Image-to-Video generation results integrated with our framework.
Figure 9: More Qualitative ablations. We perform controlled comparisons of our method with alternative variants under controlled settings. Best viewed zoomed in. The red boxes highlight regions with more noticeable noise, and the blue boxes indicate the corresponding zoomed-in views.
Figure 10: A fair comparison of Local Attention and GCLFA. We perform controlled comparisons of our method with attnetion variants under controlled settings. Best viewed zoomed in. The red boxes highlight regions with incorrect semantics or layout.
Figure 11: Left : Illustration 4 steps generation results integrated with our framework. Right : Illustration Image-to-Video generation results integrated with our framework.
Figure 12: Two examples of style transfer with ViBe. In each case, the top row shows the original video, while the bottom row presents the stylized result.
Figure 13: Qualitative results of 1080P and 2K generated by our method ViBe integrated with Wan2.2 [ 39 ] .
Figure 14: Qualitative results of 3K and 4K generated by our method ViBe integrated with Wan2.2 [ 39 ] .
Recent diffusion-based video generators have achieved remarkable visual fidelity and prompt controllability, yet scaling them to ultra-high-resolution (UHR) long videos remains prohibitively expensive. The difficulty is especially pronounced for long single-shot generation where a continuous scene must preserve global temporal coherence, and fine-grained spatial details without relying on clip transitions or autoregressive shot stitching. In this work, we revisit this challenge from the perspective of decoupled modeling. We argue that existing video diffusion models already encode strong local visual priors, while the main bottleneck lies in efficiently extending global spatiotemporal modeling as resolution and duration increase. Based on this insight, we propose AtlaVid, a decoupled global-local framework for efficient UHR long video generation. AtlaVid first generates a low-resolution and low-FPS global semantic proxy via temporally scaled RoPE, thereby extending the temporal horizon without increasing the training token count. Guided by this proxy, a high-resolution detail branch performs joint denoising with hierarchical locality-preserving attention. Reordered spatiotemporal windows preserve geometric locality and asymmetric global-local attention injects aligned semantic guidance and preserves the model's pretrained ability. This design enables resolution-agnostic training: the model is trained only at 720P with lightweight LoRA adaptation, yet generalizes directly to 4K and beyond for longer (>10s) video synthesis. Experiments show that AtlaVid substantially improves the efficiency of ultra-high-resolution long video generation, achieving high-quality UHR long video generation with 60.9x speed up and significantly less training cost and even better performance than native 4K video generators.
Adapting large-scale pre-trained video generators for Video Super-Resolution (VSR) in novel domains remains computationally prohibitive. Methods that reformulate generation as direct Low-Quality to High-Quality mappings deviate from the original generative formulation, demanding extensive fine-tuning. ControlNet-style adapters lose their efficiency under modern Diffusion Transformers since the absence of encoder-decoder hierarchy forces duplication of the entire backbone. We observe that flow matching offers a principled alternative for cross-domain VSR adaptation. By predicting a constant velocity field across all timesteps, the adaptation task reduces to learning a fixed injection pattern rather than time-varying transformations. Building on this insight, we propose LiteVSR, a minimalist framework that performs VSR using a completely frozen Diffusion Transformer with a lightweight State-Aware Adapter. The adapter employs a dual-stream architecture that extracts static structural cues from the LQ input and dynamic cues from intermediate denoising states, aligning them through time-dependent cross-attention to enable adaptive transition from structural alignment to texture refinement as denoising proceeds. LiteVSR achieves competitive restoration quality with only 11.25% trainable parameters and 12 GPU-hours of training on a single A100, while maintaining fast sampling (down to a single step) compatibility.
Yu Cao, Ziquan Liu, Zhensong Zhang +3
Queen Mary University of London · Huawei Darwin Research Center · Imperial College London
Generative video super-resolution (VSR) can recover visually compelling high-frequency detail by leveraging diffusion priors, but iterative sampling and heavy spatio-temporal coupling make practical deployments difficult. We study a different operating point: practical HD generative VSR on a single high-end consumer GPU without end-to-end retraining. To this end, we present a zero-shot framework built entirely from pretrained components. The proposed system decouples spatial enhancement from temporal stabilization. A time-distilled one-step diffusion backbone handles spatial upscaling, while temporal coherence is promoted by a lightweight pretrained flow-based interpolator that synthesizes intermediate frames between super-resolved keyframes. To further improve scalability, we introduce a resolution-aware hybrid attention scheme that, in a zero-shot manner, replaces selected high-resolution global-attention layers with neighborhood attention while preserving global attention in semantically critical low-resolution layers, while also improving temporal consistency. Experiments show that the proposed framework reaches up to ~32 FPS at 720p resolution under interleaved inference on a single NVIDIA RTX 4090. At 4K, the proposed attention substitution can yield up to a 44% speedup over the original attention implementation. Across standard VSR benchmarks, the method consistently shifts the speed-quality trade-off decisively toward runtime efficiency, trading reconstruction fidelity for strong no-reference perceptual quality, while remaining reasonably competitive against generative approaches in temporal consistency. It also exposes clear limitations in handling more challenging motion, which a simple motion gate partially mitigates. Code will be released upon acceptance.
Georgios Panaretou, Sotirios Chatzis
Ethical AI Novelties Limassol, Cyprus · Cyprus University of Technology Limassol, Cyprus