Distilling pretrained video diffusion models into few-step autoregressive (AR) generators typically combines student initialization with distribution matching distillation (DMD). An intermediate AR teacher can provide causally compatible targets for ODE initialization. However, causally compatible targets alone do not ensure high-quality generation under a short sampling schedule. We analyze the posterior-mean predictions favored by direct latent regression and empirically find that causal ODE initialization alleviates blur but still leaves fine detail unresolved. To address this limitation, we propose Representation Forcing, which learns few-step causal initialization through representation-space supervision rather than intermediate-teacher trajectory regression. The student is trained with teacher forcing by comparing clean predictions and training targets through normalized features of a frozen video diffusion transformer. This phase reuses the synthetic videos generated by a bidirectional model for AR teacher training and replaces both teacher training and causal ODE distillation, without generating additional causal ODE trajectories. The subsequent asymmetric DMD procedure remains unchanged. Empirical results show that, at four sampling steps, RepForcing better preserves generation quality than intermediate-teacher-based initialization before DMD. After DMD, our four-step chunk-wise model improves VBench Total from 84.04 to 84.66 (+0.62 points) and VisionReward from 6.326 to 10.074 (+59.25%) over the reported causal-ODE-based baseline.
Figures & tables
Figure 1: High-Quality Videos Generated by RepForcing. The examples span human activities, animal behavior, and stylized scenes, showing details such as clothing folds, fur, and vegetation. Across the displayed frames, subjects change pose and position while retaining their appearance and surrounding scene structure.
Figure 2: RepForcing Initialization Better Preserves Generation Quality as Sampling Steps Decrease. Rows compare three initialization strategies: teacher-forced AR training followed by ODE distillation, teacher-forced AR training followed by consistency distillation, and direct representation-space initialization, from top to bottom. All results are shown before subsequent refinement. Columns show 4, 8, and 12 sampling steps from left to right, with two frames per setting for the same prompt. With more denoising steps, ODE initialization approaches the visual quality of RepForcing. At four steps, however, the intermediate-teacher initializations exhibit blur or structural artifacts, while RepForcing maintains coherent subject and scene structure.
Figure 3: Representation Forcing Initialization Pipeline. Training videos are encoded into clean latent blocks by a VAE encoder. The trainable causal DiT reconstructs clean blocks from noisy inputs, conditioned on preceding clean blocks under teacher forcing. The predicted and target video latents are passed separately through the same truncated, frozen Wan2.1-14B to obtain features f^ and f . A token-wise cosine loss compares these features, with stop-gradient ( sg ) on the target branch. Gradients propagate through the frozen extractor to update the causal DiT. This initialization replaces intermediate AR teacher training and causal ODE distillation.
Figure 4: Qualitative comparison of chunk-wise distilled autoregressive models. For each of four prompts, three sampled frames are shown for CausVid, Self Forcing, Causal Forcing, Causal-rCM, and RepForcing. The RepForcing examples retain visible fur strands and facial detail in the character scene, together with clothing folds, hanging blossoms, and rock textures in the human scenes. Changes in body and hand pose are visible across the sampled frames.
Model
Total ↑
Quality ↑
Semantic ↑
Dynamic. ↑
Vision. ↑
Instruct. ↑
Bidirectional video models
LTX-1.9B ( HaCohen et al., 2024 )
79.83
81.88
71.62
46
−6.218
−38
Wan2.1-1.3B ( Wan et al., 2025 )
83.37
84.30
79.65
61
5.275
42
Autoregressive video models
NOVA ( Deng et al., 2025 )
80.31
80.66
78.92
46
−7.381
−16
Pyramid Flow ( Jin et al., 2025 )
80.75
83.41
70.11
16
4.055
−2
Table 1: Comparison with existing methods. Dynamic., Vision., and Instruct. denote Dynamic Degree, VisionReward, and Instruction Following. Bold marks column-wise best results within each distilled AR group, including ties; empty cells are unreported. Baseline scores follow Zhu et al. (2026) ; frame-wise four-step Causal Forcing++ scores come from its own report ( Zhao et al., 2026 ) , and Causal-rCM scores are obtained from evaluations of pretrained weights ( Zheng et al., 2026 ) .
Ablation
Setting
Total ↑
Quality ↑
Semantic ↑
Dynamic. ↑
Vision. ↑
Instruct. ↑
Repr. Model
DINOv2
82.82
83.73
79.15
75
5.947
53
InternVid
83.08
84.33
78.05
64
7.653
62
Wan2.1-14B
84.66
85.55
81.10
69
10.074
78
Relative Depth
1/8
84.49
85.45
80.64
61
9.217
69
1/5
84.66
85.55
81.10
69
10.074
78
1/2
84.58
85.51
80.86
65
9.893
77
Table 2: Ablation Study on Representation Models and Extraction Depth. All variants are evaluated with chunk-wise four-step generation after matched DMD refinement. Repr. Model compares feature extractors, while Relative Depth varies the extraction position within Wan2.1-14B. The default representation model is shown in bold. Dynamic., Vision., and Instruct. denote Dynamic Degree, VisionReward, and Instruction Following.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Qualitative comparison of frame-wise autoregressive models. Each quadrant shows three sampled frames for the same prompt across Causal Forcing, Causal Forcing++, and RepForcing (top to bottom). The examples depict a girl peeling an orange (top left), a mountaineer overlooking a frozen fjord (top right), a mother bird feeding her chicks (bottom left), and a woman holding flowers above a lake (bottom right).
Real-time interactive video generation requires low-latency, streaming, and controllable rollout. Existing autoregressive (AR) diffusion distillation methods have achieved strong results in the chunk-wise 4-step regime by distilling bidirectional base models into few-step AR students, but they remain limited by coarse response granularity and non-negligible sampling latency. In this paper, we study a more aggressive setting: frame-wise autoregression with only 1--2 sampling steps. In this regime, we identify the initialization of a few-step AR student as the key bottleneck: existing strategies are either target-misaligned, incapable of few-step generation, or too costly to scale. We propose Causal Forcing++, a principled and scalable pipeline that uses causal consistency distillation (causal CD) for few-step AR initialization. The core idea is that causal CD learns the same AR-conditional flow map as causal ODE distillation, but obtains supervision from a single online teacher ODE step between adjacent timesteps, avoiding the need to precompute and store full PF-ODE trajectories. This makes the initialization both more efficient and easier to optimize. The resulting pipeline, \ours, surpasses the SOTA 4-step chunk-wise Causal Forcing under the frame-wise 2-step setting by 0.1 in VBench Total, 0.3 in VBench Quality, and 0.335 in VisionReward, while reducing first-frame latency by 50% and Stage 2 training cost by \sim$$4\times. We further extend the pipeline to action-conditioned world model generation in the spirit of Genie3. Project Page: https://github.com/thu-ml/Causal-Forcing and https://github.com/shengshu-ai/minWM .
Min Zhao, Hongzhou Zhu, Kaiwen Zheng +6
Tsinghua University · ShengShu · Renmin University of China
Few-step autoregressive video generation commonly relies on Distribution Matching Distillation (DMD), requiring a bidirectional diffusion teacher and an online fake-score model. We instead learn the rollout distribution directly from reference videos, eliminating both score models during post-training. Our framework minimizes maximum mean discrepancy (MMD) in frozen self-supervised video representation spaces, using a hybrid Nyström--Monte Carlo estimator to balance approximation bias and sampling variance. Memory-efficient replay and gradient subsampling make this objective practical. Using the same architecture and initialization as Self-Forcing, our 1.3B model improves the VBench Total score from 83.80 to 84.64 while retaining 17 FPS. Removing auxiliary score models also enables 14B post-training on eight H200 GPUs. Beyond distillation, learning from reference videos enables the acquisition of new visual styles, semantic concepts, and spatial priors without a target-specific diffusion teacher.
Chi Zhang, Yueyi Liu, Shi Haoyang +5
College of AI, Tsinghua University · IAIR, Xi’an Jiaotong University · Xianghui Academy, Fudan University +1
Few-step autoregressive (AR) video diffusion enables low-latency streaming generation, but existing post-training methods predominantly rely on Distribution Matching Distillation (DMD), requiring both a large pretrained teacher and an online critic to estimate distributional discrepancies through diffusion scores. In this work, we ask whether this resource-intensive teacher--critic stack can be eliminated by post-training only the generator against a precomputed target distribution. Drawing inspiration from representation distribution matching (RDM) for one-step image generation, we systematically study its transfer to few-step causal video generation and identify three key barriers: a memory-intractable gradient path, a distinct video optimization regime, and representation distributions that underconstrain temporal dynamics. We introduce ViRDM, a teacher- and critic-free video post-training recipe that addresses these barriers sequentially. By coupling RDM with stochastically truncated clean-exit supervision, a lightweight VAE decoder, and staged vector--Jacobian products, ViRDM makes representation distribution matching memory-feasible for multi-step causal video rollouts. We further establish effective generated-population and initialization regimes for video RDM, and introduce lightweight dynamics regularization to compensate for the underconstrained temporal dynamics. ViRDM turns three-network distillation into generator-only post-training, reducing GPU memory use and training time while improving video quality. With only 20 generator updates, the recipe reaches 84.87 on the official VBench evaluation, outperforming the previous best few-step causal baseline by 0.36, while requiring 16 A100 GPU-hours. We additionally report exploratory results demonstrating the potential of the same recipe for lower causal sampling budget and for one-, two-, and four-step bidirectional generation.