Video generation has recently emerged as a central task in the field of generative AI. However, the substantial computational cost inherent in video synthesis makes model distillation a critical technique for efficient deployment. Despite its significance, there is a scarcity of methods specifically designed for video diffusion models. Prevailing approaches often directly adapt image distillation techniques, which frequently lead to artifacts such as oversaturation, temporal inconsistency, and mode collapse. To address these challenges, we propose a novel distillation framework tailored specifically for video diffusion models. Its core innovations include: (1) an adaptive regression loss that dynamically adjusts spatial supervision weights to prevent artifacts arising from excessive distribution shifts; (2) a temporal regularization loss to counteract temporal collapse, promoting smooth and physically plausible sampling trajectories; and (3) an inference-time frame interpolation strategy that reduces sampling overhead while preserving perceptual quality. Extensive experiments and ablation studies on the VBench and VBench2 benchmarks demonstrate that our method achieves stable few-step video synthesis, significantly enhancing perceptual fidelity and motion realism. It consistently outperforms existing distillation baselines across multiple metrics.
Figures & tables
Figure 1: Baselines like DMD and rCM show severe color oversaturation (left) and reduced motion indicative of temporal collapse (right). Our method achieves appropriate saturation and enhances motion dynamics beyond the teacher model.
Figure 2: Our method distills a pre-trained teacher model, denoted as sdata , into a few-step video generator Gϕ . The training procedure consists of the following steps:(1) A batch of real video-text pairs is sampled from the dataset. After applying noise perturbations to the videos, the student model performs denoising reconstruction. A regression loss is computed between the reconstructed video and the ground-truth video. Subsequently, this loss is adaptively weighted using our Loss Mean Cache to produce the final adaptive regression loss (see Sec. 3.3 for details). (2) Text conditions are sampled from the dataset to guide the student model in generating a video from pure noise. The denoised output from this process is used to compute a temporal regularization loss (Eq. 8 ) and a distribution matching loss (Eq. 4 ).(3) Finally, the generator Gϕ is updated via gradient descent using the combined losses. The sgen,ξ in DMD are updated separately, following the methodology of DMD2 (this particular update step is not depicted in the figure for clarity).
Figure 3: This figure explains the origin of oversaturation in distribution-matching distillation. At a given denoising timestep, sreal denotes the teacher’s ground-truth clean-sample distribution and sfake the student’s estimate obtained via an online multi-step model; the teacher’s overemphasis on fine-grained detail biases the student toward an oversaturated, suboptimal distribution, degrading perceptual video quality.
Figure 4: A naive regression loss (Row 3) causes object fusion artifacts (t=2.5s) absent in the teacher (Row 1) and baseline (Row 2). Our adaptive loss (Row 4) resolves this artifact, improving generation quality.
Method
Steps
Resolution
Params
Time
VBench2 Evaluation Dimension
VBench1 Evaluation Dimension
Creati- vity
Common- sense
Controll- ability
Human Fidelity
Physics
Total
Quality Score
Semantic Score
Total
Teacher
50 × 2
832 × 480 × 81
1.3B
270s
48.01
58.51
21.82
80.65
45.97
50.99
84.11
64.20
80.13
DMD ∗ [ 65 ]
4
832 × 480 × 81
1.3B
10.8s
55.90
56.51
25.29
86.75
43.72
53.63
85.55
61.10
80.66
LCM [ 31 ]
6
832 × 480 × 81
1.3B
16.2s
27.66
45.53
10.95
76.07
40.40
40.12
77.62
50.15
72.12
PCM [ 57 ]
6
832 × 480 × 81
1.3B
16.2s
44.54
50.42
17.02
81.48
43.53
47.40
79.36
51.40
73.70
DCM [ 37 ]
6
832 × 480 × 81
1.3B
16.2s
61.31
52.71
16.93
83.99
44.07
51.80
79.27
52.16
73.92
Table 1: Quantitative comparison on two benchmarks (VBench2 and VBench1). Our method achieves the best overall score across all metrics and datasets. DMD ∗ is denoted as our baseline method. In all result tables, bold text indicates the optimal result, while underlined text represents the suboptimal result.
Figure 5: Qualitative comparison. Baseline methods tend to produce oversaturated colors (left case) and exhibit stiff or static motion (right case), while our method generates videos with more natural color tones, smoother temporal dynamics, and scene transitions that better align with the prompt. The highlighted text in the prompt indicates key visual changes described in the scene. Zoom in for better visualization.
Figure 6: The figure demonstrates that the adaptive-regression-loss enables effective distribution transfer during distillation. ”GT” denotes ground-truth frames; both the Teacher and DMD models fail to generate specific anime-style videos consistent with the target domain.
Figure 7: The figure presents a user study comparing our distilled student model against its teacher (Wan2.1) and other existing video diffusion models. The results demonstrate that our model achieves superior video quality, even surpassing its teacher model in terms of user preference, while operating at a significantly lower inference cost.
Method
Steps
Resolution
Time
Evaluation Score
Instance Preservation
Dynamic Degree
Teacher
50×2
832×480
270s
92.39
85.56
DMD
4
832×480
10.8s
88.88
72.22
+TR
4
832×480
10.8s
85.38
100.00
+TR+RegLoss
4
832×480
10.8s
83.04
78.61
+TR+AdaLoss
4
832×480
10.8s
92.39
99.72
Table 2: Ablation study on the effectiveness of the proposed Adaptive Regression Loss and Temporal Regularization. DMD serves as the baseline method, while TR and AdaLoss denote models equipped with the Temporal Regularization and Adaptive Regression Loss, respectively. Full+VIF refers to the model incorporating both components and frame-interpolation.
Figure 8: …
Figure 9: …
Figure 10: Comparison with baselines on MovieGen prompts. While DMD and rCM (a strong performer from Sec. 4 ) produce videos with low motion dynamics and oversaturated colors, our method resolves both issues. It achieves superior motion, well-calibrated colors, and excellent detail and stability.More video examples are included in the supplementary zip file. (Note: We have confirmed that any visual artifacts in some videos are not code-level issues.)
Figure 11: Caption model prompt and caption style examples.
Figure 12: This figure illustrates the impact of the truncation threshold of the temporal regularization loss on model performance. The horizontal axis represents the truncation threshold of the regularization loss, while the vertical axis shows the motion score and the instance preservation score of the videos generated by the student model, respectively.
Figure 13: Visualization of the adaptive weight function in Eq. ( 7 ) for different values of k . The x-axis represents the deviation Ls−Lˉs , while the y-axis represents the resulting adaptive weight.
Figure 14: Failure cases resulting from unclipped temporal regularization. Without clipping, the student generator produces severe artifacts late in training. (Top) After the first second, a drastic content shift occurs, accompanied by a noticeable distortion of the building (highlighted by the red box). (Bottom) The scene content abruptly vanishes at the two-second mark and is replaced by another major content shift at the three-second mark (highlighted by the red box). These phenomena, inconsistent with plausible camera motion, are clear manifestations of hallucinations. This highlights the necessity of clipping the temporal loss to prevent it from excessively amplifying inter-frame variance.
Figure 15: Violin plot showing the distribution of adjacent-frame cosine similarity across different denoising stages. The overall similarity is notably higher during the high-noise stage, which motivates our approach of halving the inference frame rate in this phase to reduce computational cost.
Large pretrained diffusion models have significantly enhanced the quality of generated videos, and yet their use in real-time streaming remains limited. Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity. Diffusion distillation can compress these models into efficient few-step variants, but existing video distillation approaches largely adapt image-specific methods that neglect temporal dependencies. These techniques often excel in image generation but underperform in video synthesis, exhibiting reduced motion coherence, error accumulation over long sequences, and a latency-quality trade-off. We identify two factors that result in these limitations: insufficient utilization of temporal context during step reduction and implicit prediction of subsequent noise levels in next-chunk prediction (i.e., exposure bias). To address these issues, we propose Diagonal Distillation, which operates orthogonally to existing approaches and better exploits temporal information across both video chunks and denoising steps. Central to our approach is an asymmetric generation strategy: more steps early, fewer steps later. This design allows later chunks to inherit rich appearance information from thoroughly processed early chunks, while using partially denoised chunks as conditional inputs for subsequent synthesis. By aligning the implicit prediction of subsequent noise levels during chunk generation with the actual inference conditions, our approach mitigates error propagation and reduces oversaturation in long-range sequences. We further incorporate implicit optical flow modeling to preserve motion quality under strict step constraints. Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.
Jinxiu Liu, Xuanming Liu, Kangfu Mei +3
South China University of Technology · Westlake University · Johns Hopkins University +2
Video Diffusion Models (VDMs) have demonstrated superior generation quality but suffer from prohibitive computational costs. While recent few-step distillation techniques significantly accelerate inference, they typically enforce a static model architecture across all denoising stages, ignoring the varying computational demands inherent to different noise levels. In this work, we propose a novel post-training acceleration framework that exploits this redundancy by integrating dynamic structural sparsification directly into the distillation process. Unlike conventional post-hoc compression applied to a fixed diffusion pipeline, our approach jointly optimizes the denoising steps and structured model sparsity, transforming a pre-trained VDM into a compact, step-specific Mixture-of-Models (MoM). To address the training instability arising from this joint optimization, we introduce a Progressive Training Strategy coupled with an Output Rollout Mechanism, which ensures the coherent learning of structural decisions across timesteps. Furthermore, we develop a specialized inference engine to deploy the resulting MoM efficiently. Our method is orthogonal to existing acceleration techniques and highly effective: On Wan-14B, it removes 24% of the per-step FLOPs on top of 4-step distillation, adding a 1.2x wall-clock gain and reaching a 30x speedup over the 50-step teacher while preserving competitive generation quality.
Yu Cheng, Siyue Yao, Zhongang Qi +3
Zhejiang University · BlueImage, vivo · Westlake University
Recent progress has shown promise in distilling multi-step video diffusion models into efficient few-step students. Among them, Distribution Matching Distillation (DMD) and its successor DMD2 achieved strong generation quality and fast convergence. However, due to the nature of the reverse Kullback--Leibler (KL) objective, these methods exhibit two persistent failure modes: a substantial drop in sample diversity, and visibly over-saturated outputs that deviate from real-video appearance. In this work, we propose Data-Forcing Distillation (DFD), a simple post-training framework that restores diversity and fidelity in DMD with only a single-line of code change. At its core is the teacher score discrepancy to guide the student toward the real-data distribution, pulling it to missing modes (mitigating mode collapse) and away from problematic modes absent in real data (avoiding over-saturation). We provide an in-depth theoretical analysis of our framework and validate our approach on text-to-video, image-to-video, and autoregressive video generation. With only 100--300 steps of finetuning, DFD effectively restores diversity and fidelity on both Wan2.1-1.3B and Cosmos-Predict2.5-2B model, resolving the over-saturation artifacts with significantly better video dynamics and appearance, and even outperforms the teacher model.
Siyi Chen, Shaowei Liu, Yixuan Jia +4
University of Michigan · NVIDIA · University of Illinois Urbana-Champaign