cs.CVMar 4, 2026
SaveHelios: Real Real-Time Long Video Generation Model
Organizations: Peking University · ByteDance China · Chengdu Anu Intelligence · Canva
Abstract
We introduce Helios, the first 14B video generation model that runs at 19.5 FPS on a single NVIDIA H100 GPU and supports minute-scale generation while matching the quality of a strong baseline. We make breakthroughs along three key dimensions: (1) robustness to long-video drifting without commonly used anti-drifting heuristics such as self-forcing, error-banks, or keyframe sampling; (2) real-time generation without standard acceleration techniques such as KV-cache, sparse/linear attention, or quantization; and (3) training without parallelism or sharding frameworks, enabling image-diffusion-scale batch sizes while fitting up to four 14B models within 80 GB of GPU memory. Specifically, Helios is a 14B autoregressive diffusion model with a unified input representation that natively supports T2V, I2V, and V2V tasks. To mitigate drifting in long-video generation, we characterize typical failure modes and propose simple yet effective training strategies that explicitly simulate drifting during training, while eliminating repetitive motion at its source. For efficiency, we heavily compress the historical and noisy context and reduce the number of sampling steps, yielding computational costs comparable to -- or lower than -- those of 1.3B video generative models. Moreover, we introduce infrastructure-level optimizations that accelerate both inference and training while reducing memory consumption. Extensive experiments demonstrate that Helios consistently outperforms prior methods on both short- and long-video generation. We plan to release the code, base model, and distilled model to support further development by the community.
Figures & tables
Figure 1: End-to-end throughput (FPS) of various video generation models on a single H100. The results are obtained at the same resolution with all official acceleration techniques, including FlashAttention, torch compile, and KV-cache. Helios is substantially faster than models at the same scale and matches the speed of smaller distilled ones.
Figure 2: Benchmark performance of Helios and its counterparts. For both short- and long-video generation, Helios consistently outperforms existing distilled models while achieving performance comparable to that of base models.
Figure 3: Showcases of infinite videos generated by Helios. Despite overhead comparable to that of the 1.3B models [ 90 , 100 , 59 , 60 , 126 ] , Helios still excels in visual quality, text alignment, and motion dynamics.
Figure 4: Architecture of Helios. Helios is an autoregressive video diffusion transformer built with Guidance Attention blocks. It reduces overhead by compressing historical and noisy context through Multi-Term Memory Patchification and Pyramid Unified Predictor Corrector, while unifying T2V, I2V, and V2V tasks via Representation Control.
Figure 5: Visualization of three representative drifting patterns in long-video generation.
Figure 6: Temporal trends of saturation, aesthetic, and RGB statistics for normal videos versus drifting videos. Normal videos are stable, while drifting videos initially follow a similar trajectory but shift abruptly and remain unstable.
Figure 7: Overhead reduction with Multi-Term Memory Patchification. A hierarchical history window uses progressively larger kernels, keeping the token budget constant while extending the context length.
Figure 8: Outline of Pyramid Unified Predictor Corrector. The process consists of three stages: (i) Low-stage focuses on efficiency, (ii) Mid-stage balances quality and efficiency, and (iii) High-stage prioritizes quality.
Figure 9: Pipeline of Adversarial Hierarchical Distillation. The framework is based on DMD [ 105 ] , with improvements such as Pure Teacher Forcing, Staged Backward Simulation, Coarse-to-Fine Learning and Adversarial Post-Training.
Figure 10: Execution of Cache Grad for GAN. We cache discriminator gradients w.r.t. inputs to decouple backpropagation and free intermediate activations early, substantially reducing peak memory.
| Configuration | Stage-1-init | Stage-1-post | Stage-2-init | Stage-2-post |
| Global Batch Size | 128 | 128 | 256 | 192 |
| Optimizer | AdamW, , , weight_decay=1e-04 | AdamW, , , weight_decay=1e-04 | AdamW, , , weight_decay=1e-04 | AdamW, , , weight_decay=1e-04 |
| Learning Rate | 5e-5 | 3e-5 | 1e-4 | 3e-5 |
| Learning Rate Schedule | Constant | Constant | Constant | Constant |
| Training Steps | 5.5k | 7.5k | 16k | 20k |
| Gradient Clipping | 1.0 | 1.0 | 1.0 | 1.0 |
Table 1: The detailed training hyperparameters of Stage-1 and Stage-2.
| Configuration | Stage-3-ode | Stage-3-post |
| Global Batch Size | 128 | 128 |
| Real-score initialization | - | Helios-Base |
| Fake-score initialization | - | Helios-Base |
| Real-score CFG weight | - | 3.0 |
| Optimizer ( & ) | AdamW, , , weight_decay=1e-03 | AdamW, , , weight_decay=1e-03 |
| Learning Rate | 2.0e-6 | 2.0e-6 |
Table 2: The detailed training hyperparameters of Stage-3.
| Model | #Params | Throughput (FPS) | Total | Aesthetic | Dynamic | Smoothness | Semantic | Naturalness |
| Bidirectional Models | ||||||||
| SANA Video [ 9 ] | 2B | 1.34 | 4.60 | 9 | 7 | 9 | 5 | 1 |
| CogVideoX [ 103 ] | 2B | 1.95 | 5.55 | 7 | 10 | 7 | 4 | 5 |
| CogVideoX 1.5 [ 103 ] | 5B | 1.47 | 5.15 | 7 | 5 | 8 | 5 | 4 |
| Mochi-1 [ 80 ] | 10B | 0.53 | 5.35 | 6 | 7 | 9 | 5 | 4 |
| HV Video [ 41 ] | 13B | 0.36 | 6.00 | 8 | 8 | 9 | 5 | 5 |
Table 3: Quantitative comparisons on 81-frames short videos. Helios achieves a superior speed-quality trade-off over existing methods, which are either computationally prohibitive or yield suboptimal results. “ ” higher is better.
Figure 11: Qualitative comparisons on 81-frames short videos (Part-1). Even as a distilled model, Helios matches or even surpasses the base models in terms of visual quality, motion dynamics, and naturalness.
Figure 12: Qualitative comparisons on 81-frames short videos (Part-2). Despite being a distilled model, Helios matches or surpasses the base models in visual fidelity, text alignment, and overall realism.
| Model | #Params | Throughput (FPS) | Total | Throughput Score | Total* | Aesthetic | Dynamic | Smoothness | Semantic | Naturalness | Drifting Aesthetic | Drifting Smoothness | Drifting Semantic | Drifting Naturalness |
| Autoregressive Models | ||||||||||||||
| NOVA [ 18 ] | 0.6B | 1.18 | 2.48 | 1 | 2.38 | 1 | 1 | 9 | 1 | 2 | 1 | 9 | 1 | 2 |
| Pyramid Flow [ 38 ] | 2B | 3.11 | 2.85 | 1 | 2.75 | 6 | 3 | 9 | 3 | 1 | 1 | 8 | 1 | 2 |
| MAGI-1 [ 86 ] | 4.5B | 0.37 | 4.92 | 1 | 4.82 | 7 | 3 | 10 | 6 | 2 | 1 | 10 | 6 | 5 |
| InfinityStar [ 58 ] | 8B | 3.38 | 2.63 | 2 | 2.43 | 4 | 4 | 8 | 2 | 1 | 1 | 9 | 1 | 1 |
| SkyReelsV2-DF [ 8 ] | 1.3B | 0.55 | 3.29 | 1 | 3.19 | 6 | 6 | 9 | 4 | 1 | 1 | 9 | 2 | 1 |
Table 4: Quantitative comparisons on 120, 240, 720 and 1440-frames long videos. Helios consistently exceeds existing real-time long video generation methods. “ ” denotes higher is better.
Figure 13: Qualitative comparisons on 120, 240, 720 and 1440-frames long videos (Part-1). It is clear that Helios consistently outperforms the baseline models in terms of realism and naturalness.
Figure 14: Qualitative comparisons on 120, 240, 720 and 1440-frames long videos (Part-2). It is clear that Helios consistently outperforms the baseline models in terms of text alignment and dynamics.
Figure 15: Side-by-side human evaluation of Helios versus counterparts. Left: Long Video; Right: Short Video
| Model | Total | Aesthetic | Dynamic | Smoothness | Semantic | Naturalness | Drifting Aesthetic | Drifting Smoothness | Drifting Semantic | Drifting Naturalness |
| Helios-Base | 6.47 | 8 | 6 | 9 | 6 | 5 | 7 | 10 | 8 | 5 |
| w Guidance Attention* | unstable training process | |||||||||
| w/o Guidance Attention | 6.23 | 9 | 4 | 7 | 7 | 5 | 6 | 10 | 8 | 2 |
| w/o First Frame Anchor | 5.51 | 8 | 5 | 8 | 7 | 4 | 3 | 10 | 6 | 2 |
| w/o Frame-Aware Corrupt | 4.70 | 7 | 4 | 8 | 6 | 4 | 2 | 10 | 3 | 1 |
| Helios-Distilled | 6.34 | 8 | 6 | 10 | 5 | 5 | 7 | 10 | 7 | 7 |
Table 5: Quantitative ablation of key components. Changing any component leads to a significant degradation in quality while exacerbating temporal drifting, thereby impairing Helios ’s ability to generate stable long videos. w Guidance Attention* indicates the additional use of a causal mask for self-attention. w Staged Backward Simulation* means incorporating multi-scale into real/fake-score estimators during training.
Figure 16: Qualitative ablation on Guidance Attention with Helios -Base. Introducing causal mask hinders the ability to learn temporal coherence, causing each generated section to appear independent. Conversely, removing Guidance Attention results in excessive semantic accumulation over time (e.g., a progressively enlarged bird crest). * indicates the additional usage of causal mask to self-attention to prevent noisy context from affecting historical context.
Figure 17: Qualitative ablation on First Frame Anchor and Frame-Aware Corrupt with Helios -Base. Removing the First Frame Anchor not only introduces drifting but also causes the subject to deviate from the one in the initial frame. Removing Frame-Aware Corruption leads to noticeable drifting even in videos as short as 240 frames.
Figure 18: Qualitative ablation on Stage Backward Simulation and Pure Teacher Forcing with Helios -Distilled. Feeding multi-scale into the fake-score estimator causes the model to converge toward incorrect directions. Pure Teacher Forcing achieves comparable robustness against long-video drifting as Self-Forcing [ 34 ] . * indicates that multi-scale , rather than only the full-resolution , is provided to the real/fake-score estimator.
Figure 19: Qualitative ablation on Coarse-to-Fine Learning and Adversarial Post-Training with Helios -Distilled. Removing Coarse-to-Fine Learning prevents the model from converging, with particularly unacceptable quality in the first generated section. Removing Adversarial Post-Training leads to degradation in visual quality.
Figure 20: Qualitative ablation on Decouple DMD [ 54 ] and Reward-weighted Regression [ 60 ] with Helios -Distilled. The former may hinder convergence and cause grayish outputs, whereas the latter may intensify video flickering.
Figure 21: Text-to-Video showcases. Part of the prompts are sourced from [ 34 ] .
Figure 22: Image-to-Video showcases. Part of the images are sourced from [ 35 , 45 ] .
Figure 23: Interactive-to-Video showcases. Part of the prompts are sourced from [ 100 , 91 ] .
| Inference Time (s) | Training Time (s) | |
| Wan-2.1-T2V-14B [ 90 ] | 98.68 | 398.03 |
| w Flash Normalization | 89.91 | 360.77 |
| w Flash RoPE | 93.39 | 378.77 |
| w Flash Normalization and Flash RoPE | 84.41 | 340.38 |
Table 6: Quantitative ablation on Flash Normalization and Flash RoPE. We report the total runtime of the DiT component in Helios, measured over 50 forward passes and 50 forward-backward passes on a single NVIDIA H100 GPU.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Metric ( ) | Type | |||||||||
| Aesthetic | Higher Better | 0.70 | 0.65 | 0.60 | 0.55 | 0.50 | 0.45 | 0.40 | 0.35 | 0.30 |
| Motion Amplitude | Higher Better | 0.45 | 0.40 | 0.35 | 0.30 | 0.25 | 0.20 | 0.15 | 0.10 | 0.05 |
| Motion Smoothness | Higher Better | 0.99 | 0.98 | 0.97 | 0.96 | 0.95 | 0.94 | 0.93 | 0.92 | 0.91 |
| Semantic | Higher Better | 0.30 | 0.29 | 0.28 | 0.27 | 0.26 | 0.25 | 0.24 | 0.23 | 0.22 |
| Naturalness | Higher Better | 0.65 | 0.60 | 0.55 | 0.50 | 0.45 | 0.40 | 0.30 | 0.25 | 0.20 |
| Drifting Aesthetic | Lower Better | 0.01 | 0.02 | 0.03 | 0.04 | 0.05 | 0.06 | 0.07 | 0.08 | 0.09 |
Table 7: Empirical Thresholds ( ) for Metric Discretization. For the Aesthetic metric, the final two thresholds are marked with an asterisk (*) to indicate a potential anomaly in the empirical data or source configuration.