Few-step distillation is essential for deploying modern video diffusion models. Among existing approaches, Distribution Matching Distillation (DMD) has emerged as a highly promising paradigm by achieving strong output-space fidelity without adversarial training, while remaining flexible across various student architectures. We observe, however, that DMD fundamentally relies on an output-space objective, lacking explicit regularization on the model's internal representations. As the distillation step count aggressively shrinks, this output-only supervision becomes insufficient, causing the distilled student to degrade in both fine-grained spatial structures and coherent temporal dynamics. To address this limitation, we introduce Self Flow Forcing (SF2), which incorporates a predictive representation loss into the DMD framework without altering its output objective. By providing the student and an Exponential Moving Average (EMA) teacher with heterogeneous noise levels, we establish an information asymmetry. This formulation forces the student to predict the teacher's internal representations from a more corrupted input, effectively integrating self-supervised representation alignment into distribution matching. Extensive experiments demonstrate that SF2 effectively mitigates temporal flickering and consistently improves both spatial fidelity and motion quality. Furthermore, the proposed mechanism is architecture-agnostic - seamlessly supporting both bidirectional and causal models - and requires no external feature extractors, introducing minimal computational overhead.
Figures & tables
Figure 1: Overview of SF 2 . From the student’s denoising trajectory, the cleanest prediction drives the standard DMD loss LDMD , while two intermediate states (xt,xs) with t>s feed the SF 2 branch: a spatial mask mixes them into an asymmetric, highly corrupted input for the student, while the EMA teacher sees only the cleaner xs . Hidden states from asymmetric layers are aligned via learnable projectors with a stop-gradient cosine loss Lrep . The teacher is updated by EMA and requires no external pre-trained features.
Generator update ( ×1 )
Critic update ( ×5 )
Total
Component
Rollout
DMD
Ours
Backward
Rollout
MSE
Backward
% of cycle
4.4
20.1
6.2
6.4
25.0
8.7
29.2
100.0
Table 1: Per-cycle training-time breakdown (% of one cycle). One cycle = 1 generator update + 5 critic updates. The SF 2 branch is only ∼6% of the cycle.
Figure 2: Qualitative comparison on causal few-step T2V generation. Top row: multi-step bidirectional Wan2.1 teacher. Rows 2–5: few-step distilled causal students (CausVid, Self-Forcing, Causal-Forcing, SF 2 ). The four panels expose four facets of representation-level coherence that the rep loss is designed to compress into the student: articulated-motion plausibility (top-left), event irreversibility (top-right), object persistence (bottom-left), and scene-structural stability (bottom-right). SF 2 satisfies all four, whereas baselines fail in distinct ways on each.
Figure 3: Hyperparameter selection for asymmetric inputs. (a) Masking ratio ρmax . The two paradigms peak at different optima — ρmax=0.4 for bidirectional and ρmax=0.2 for causal — which we adopt as defaults in the following experiments. (b) Mask sampling pattern. Space-only sampling (a single Hp×Wp mask broadcast across all frames) versus agnostic sampling (independent per-token masking) on VBench’s Quality Score, Dynamic Degree, and Background Consistency.
Model
Params
Throughput ↑
Latency ↓
Resolution
Total ↑
Quality ↑
Semantic ↑
Vision. ↑
Bidirectional Video Diffusion Models
LTX-Video [ 13 ]
1.9B
8.98
13.5
768 × 512
79.83
81.88
71.62
–6.218
Wan2.1 [ 41 ]
1.3B
0.78
103
848 × 480
83.37
84.30
79.65
5.275
Distilled Bidirectional Video Models
CausVid [ 50 ]
1.3B
16.2
4.22
848 × 480
83.19
84.08
79.63
5.310
Ours
1.3B
16.2
4.22
848 × 480
84.15
84.92
81.07
7.060
Table 2: Quantitative comparison with existing methods on VBench [ 17 ] and VisionReward [ 46 ] . Models are grouped by sampling regime — bidirectional multi-step, distilled bidirectional, autoregressive multi-step, and distilled autoregressive — and bold marks the best score within each comparable group. Throughput is in FPS and latency in second, both measured on a single H100. All distilled models (in both distilled groups) share the same Wan2.1-T2V-1.3B backbone.
Figure 4: Pairwise quantitative comparison. All compared methods share the same Wan2.1-T2V-1.3B backbone. Each bar shows the per-prompt preference of VisionReward [ 46 ] ; the dark-blue segment is the fraction of prompts on which SF 2 is preferred. SF 2 wins the majority of prompts against all four baselines, on both bidirectional (DMD, multi-step Wan2.1 base model) and causal (Causal-Forcing, Self-Forcing) sides.
Figure 5: Qualitative comparison on bidirectional few-step T2V generation. Both compared methods are distilled from the same HunyuanVideo-1.5 backbone. Each pair shows the DMD student (top) and SF 2 (bottom) on the same prompt and noise; a zoomed crop on the right of each panel highlights the rendered text. Across all four prompts, SF 2 produces sharp, legible glyphs where DMD collapses into blurred or scrambled characters, indicating that the rep loss strengthens the student’s spatial representation.
Figure 6: Quantitative evaluation on interactive video generation. We evaluate the adherence of the generated video to the input camera and action conditions. Compared to the WorldPlay baseline, SF 2 achieves notably higher translation, rotation, and action accuracy across training steps, demonstrating that a stronger internal representation leads to more precise state transitions in interactive environments.
Figure 7: Qualitative camera trajectory alignment. Reconstructed 3D point clouds and camera trajectories from the generated videos. The trajectory produced by SF 2 ( blue ) tightly tracks the input condition ( green ), whereas the WorldPlay baseline ( red ) exhibits severe drift. This visualizes how the stronger internal representation of SF 2 enforces strict adherence to complex control signals.
Model
Params
Throughput
PSNR ↑
SSIM ↑
LPIPS ↓
Dynamic ↑
Flickering ↑
Consistency ↑
DMD (Ours w/o Lrep )
8.3B
2.21
20.64
0.6982
0.2144
90.00
96.39
94.11
Ours
8.3B
2.21
21.24
0.7110
0.2096
90.00
96.86
94.65
Table 3: Probing representation quality via re-rendering and frame-level coherence. We report PSNR / SSIM / LPIPS against the generated frames, plus VBench Temporal Flickering and Background Consistency. SF 2 improves over vanilla DMD on every reconstruction and coherence metric at matched Dynamic Degree, despite using no 3D supervision during training.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Wan2.1-1.3B (Bid.)
HunyuanVideo-1.5 (Bid.)
Wan2.1-1.3B (Causal)
Backbones
Student backbone
Wan2.1-T2V-1.3B
HunyuanVideo-1.5
Wan2.1-T2V-1.3B
Real-score critic (frozen)
Wan2.1-T2V-14B
HunyuanVideo-1.5
Wan2.1-T2V-14B
Fake-score critic (init.)
Wan2.1-T2V-1.3B
HunyuanVideo-1.5
Wan2.1-T2V-1.3B
Student init. ckpt
Wan2.1-T2V-1.3B
HunyuanVideo-1.5
Causal-Forcing ODE-regression
Backbone depth Nb
30
54
30
Appendix
Table 4: Full hyperparameters for the three SF 2 instantiations. The SF 2 branch is identical in spirit across settings; the only paradigm-dependent quantities are the masking ratio ρmax and the chunk size Fb (causal only). All bidirectional models use K=4 denoising steps; the causal student also uses K=4 unless otherwise noted.
Recent progress has shown promise in distilling multi-step video diffusion models into efficient few-step students. Among them, Distribution Matching Distillation (DMD) and its successor DMD2 achieved strong generation quality and fast convergence. However, due to the nature of the reverse Kullback--Leibler (KL) objective, these methods exhibit two persistent failure modes: a substantial drop in sample diversity, and visibly over-saturated outputs that deviate from real-video appearance. In this work, we propose Data-Forcing Distillation (DFD), a simple post-training framework that restores diversity and fidelity in DMD with only a single-line of code change. At its core is the teacher score discrepancy to guide the student toward the real-data distribution, pulling it to missing modes (mitigating mode collapse) and away from problematic modes absent in real data (avoiding over-saturation). We provide an in-depth theoretical analysis of our framework and validate our approach on text-to-video, image-to-video, and autoregressive video generation. With only 100--300 steps of finetuning, DFD effectively restores diversity and fidelity on both Wan2.1-1.3B and Cosmos-Predict2.5-2B model, resolving the over-saturation artifacts with significantly better video dynamics and appearance, and even outperforms the teacher model.
Siyi Chen, Shaowei Liu, Yixuan Jia +4
University of Michigan · NVIDIA · University of Illinois Urbana-Champaign
Distilling pretrained video diffusion models into few-step autoregressive (AR) generators typically combines student initialization with distribution matching distillation (DMD). An intermediate AR teacher can provide causally compatible targets for ODE initialization. However, causally compatible targets alone do not ensure high-quality generation under a short sampling schedule. We analyze the posterior-mean predictions favored by direct latent regression and empirically find that causal ODE initialization alleviates blur but still leaves fine detail unresolved. To address this limitation, we propose Representation Forcing, which learns few-step causal initialization through representation-space supervision rather than intermediate-teacher trajectory regression. The student is trained with teacher forcing by comparing clean predictions and training targets through normalized features of a frozen video diffusion transformer. This phase reuses the synthetic videos generated by a bidirectional model for AR teacher training and replaces both teacher training and causal ODE distillation, without generating additional causal ODE trajectories. The subsequent asymmetric DMD procedure remains unchanged. Empirical results show that, at four sampling steps, RepForcing better preserves generation quality than intermediate-teacher-based initialization before DMD. After DMD, our four-step chunk-wise model improves VBench Total from 84.04 to 84.66 (+0.62 points) and VisionReward from 6.326 to 10.074 (+59.25%) over the reported causal-ODE-based baseline.
Few-step distillation for video diffusion models has attracted significant attention, driven by the urgent demand for efficient deployment in real-world scenarios. However, Distribution Matching Distillation (DMD), a leading paradigm, tends to degrade under limited NFE budgets, manifesting in video generation as layout instability, oversaturation, and broken motion dynamics. We trace this failure to a structural limitation: standard DMD is an intra-sample distribution-matching objective with coordinate-wise gradients, and thus imposes no explicit constraint on the relational geometry across batch elements or temporal frames, leaving the underlying copula largely unregulated. Combined with the mode-seeking tendency of its reverse-KL objective, this absence of relational guidance makes DMD prone to collapsing into local optima in the few-step regime. Motivated by this insight, we propose Copula-aware DMD (CoDMD), a lightweight relational regularizer that reuses score estimates already produced by the frozen teacher and the online fake model to construct pairwise relation matrices across samples and frames. These are matched through a supplementary distributional objective that requires no additional networks, datasets, or sampling trajectories. On the Wan-2.1-T2V model series at 1.3B & 14B scales, CoDMD distills 50-step teachers into 4-step students, achieving an approximate 25× speed-up while attaining VBench scores of 84.46 & 84.87, outperforming prior trajectory-based (rCM 82.81 & 84.05) and distribution-based (DMD 83.38 & 83.81) methods.