Direct Forcing: Single-Stage Training with Representation-Space Supervision for Few-Step Autoregressive Video Generation

Haojun Xu; Jie Huang; Chuyang Zhao; Xin Lu; Mingchen Zhong; Zihao Fan; Haoyang Huang; Linjiang Huang

cs.CVOct 22, 2026

Direct Forcing: Single-Stage Training with Representation-Space Supervision for Few-Step Autoregressive Video Generation

Authors: Haojun Xu, Jie Huang, Chuyang Zhao, Xin Lu, Mingchen Zhong, Zihao Fan, Haoyang Huang, Linjiang Huang,

Organizations: Beihang University · JD Future Academy

Abstract

Few-step autoregressive video generation enables low-latency streaming, yet prevailing training pipelines separate causal adaptation from distribution matching. Recent representation-based text-to-video methods retain this separation by starting with an ODE initialization stage, even when the target distribution is defined directly by reference videos. We introduce \method, the first single-stage framework that uses reference videos both as paired examples of how a history continues and as a target population for self-generated rollouts. Specifically, Direct Forcing is trained with two complementary signals: (1) Causal Continuation Alignment learns paired spatiotemporal correspondence under reference histories; and (2) Rollout Distribution Matching employs maximum mean discrepancy to align the representation distributions of self-generated and reference videos. To coordinate these signals, we introduce Dynamic Gradient Balancing, which adapts their relative gradient contributions during training. Experiments with Wan2.1-1.3B show that Direct Forcing learns a four-step autoregressive generator in 250 updates without a separate ODE initialization stage or online diffusion-score supervision, achieving 84.63 VBench Total and 86.01 Quality.

Figures & tables

Figure 1: Single-stage training for few-step AR generation. Direct Forcing jointly learns causal alignment and distribution matching, achieving 84.63 VBench Total in 250 updates. Baseline counts include inherited initialization (Table 1 ).

Figure 2: Temporal discontinuity under MMD-only training. Consecutive frames 8–10 show an abrupt change at the first chunk boundary.

Method	Training path	AR teacher training	ODE init.	Distribution matching	Total
Self Forcing [ 16 , 41 ]	ODE init. (BID teacher) $\rightarrow$ DMD	–	3,000	750	3,750
Causal Forcing [ 41 ]	ODE init. (AR teacher) $\rightarrow$ DMD	2,000	1,000	750	3,750
ViRDM [ 24 ]	ODE init. (AR teacher) $\rightarrow$ MMD	2,000	1,000	20	3,020
Elastic Forcing [ 38 ]	ODE init. (BID teacher) $\rightarrow$ MMD	–	3,000	150	3,150
Direct Forcing (ours)	Joint CCA + RDM	–	–	250	250

Table 1: Training paths from bidirectional Wan2.1-1.3B to four-step AR generation. BID denotes bidirectional. Numeric entries report optimization updates for causal-teacher training, ODE initialization, and distribution matching; Total includes inherited stages. Our 250 updates jointly optimize Causal Continuation Alignment (CCA) and Rollout Distribution Matching (RDM). Backbone pretraining and offline data/feature preparation are excluded.

Figure 3: Direct Forcing overview. A shared few-step causal generator learns from reference histories with Causal Continuation Alignment (CCA; Sec. 3.2 ) and self-generated histories with Rollout Distribution Matching (RDM; Sec. 3.3 ). Frozen encoders provide paired cosine alignment and video–text MMD, respectively. Dynamic Gradient Balancing (Sec. 3.4 ) first matches gradient norms, then weights the CCA gradient by its current fitting residual before combining both gradients to update the shared generator.

Model	# Params	Resolution	Throughput $\uparrow$	Latency (s) $\downarrow$	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$
Bidirectional Video Diffusion Models
LTX-Video-1.9B [ 14 ]	1.9B	$768\times 512$	8.98	13.5	79.83	81.88	71.62
Wan2.1-1.3B [ 30 ]	1.3B	$832\times 480$	0.78	103	83.37	84.30	79.65
Autoregressive Video Diffusion Models
NOVA [ 7 ]	0.6B	$768\times 480$	0.88	4.1	80.31	80.66	78.92
Pyramid Flow [ 19 ]	2.0B	$640\times 384$	6.7	2.5	80.75	83.41	70.11

Table 2: Text-to-video generation on VBench. Models are grouped by generation scheme. Scores are percentages. The highlighted row is our 250-update model. ∗ ViRDM denotes its reported variant without optical-flow regularization. Our main model also excludes this auxiliary objective; Appendix C.4 presents our optical-flow regularization ablation.

Figure 4: Effect of training duration. VBench scores of our main configuration (RDM batch size 128) from 50 to 250 updates. Each marker denotes an evaluated checkpoint.

Configuration	Subj. $\uparrow$	Smooth. $\uparrow$	Aes. $\uparrow$	Dyn. $\uparrow$	Sem. $\uparrow$
CCA only	97.08	99.50	59.13	13.89	72.21
RDM only	84.17	96.82	63.95	11.11	71.93
CCA init $\rightarrow$ RDM-only	94.08	98.56	64.35	74.44	77.88
CCA + RDM jointly	95.69	98.77	65.07	54.72	77.99

Table 3: CCA and RDM training. Subj./Smooth./Aes./Dyn.: Subject Consistency/Motion Smoothness/Aesthetic Quality/Dynamic Degree (VBench quality); Sem.: aggregate Semantic. Sequential uses

150+150

updates; others use 150.

Figure 5: Gradient scales and fitting progress. (a) Fixed-weight CCA/RDM gradient norm ratio. (b) GNM-only losses relative to update 1; RDM is measured before self-normalization.

Batch size	32	64	128	256	512	1024
Total $\uparrow$	81.27	82.40	82.36	82.96	82.89	82.81
Quality $\uparrow$	82.10	83.51	83.46	83.95	84.06	83.89
Semantic $\uparrow$	77.95	77.99	77.99	78.97	78.21	78.47
Dynamic $\uparrow$	33.33	54.72	47.22	47.78	54.17	51.67

Table 4: RDM batch size with VideoMAE v2 L16 for both components. CCA batch size is 32; all runs use 150 updates.

Representation	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$	Dynamic $\uparrow$
Latent FM	81.76	82.77	77.75	51.39
V-JEPA 2.1 L16	81.77	83.12	76.35	40.56
V-JEPA 2.1 L24	82.05	83.30	77.08	58.06
DiT L9	82.56	83.53	78.68	58.89
VideoMAE v2 L16	82.40	83.51	77.99	54.72
VideoMAE v2 L24	82.80	84.00	78.01	58.06

Table 6: Representation for causal continuation alignment, with rollout distribution matching fixed to VideoMAE v2 L16. Latent FM denotes velocity regression in latent space.

Continuation	Distribution	Total $\uparrow$	Qual. $\uparrow$	Sem. $\uparrow$	Dyn. $\uparrow$
$V_{24}$	$V_{16}$	82.80	84.00	78.01	58.06
$V_{24}$	$D_{9}$	83.49	84.96	77.58	68.61
$V_{24}$	$V_{16}+D_{9}$	84.11	85.47	78.67	74.72
$V_{24}+D_{9}$	$V_{16}+D_{9}$	83.68	84.99	78.43	71.94
$V_{24}+D_{9}$	$V_{16}+D_{9}+V_{24}$	83.46	84.71	78.45	75.00

Table 8: Representation combinations after 150 updates.

V_{k}

: VideoMAE v2 L

k

;

D_{9}

: DiT L9. Gray marks the selected combination.

Figure 7: Qualitative comparison on four text-to-video prompts. For each case, rows show ViRDM, Elastic Forcing, and Direct Forcing from top to bottom; columns show frames 0, 40, and 80. These lengthy prompts are abbreviated above the panels for readability.

Metric	Self-Forcing++	Direct Forcing
Subject consistency	97.00	97.07
Background consistency	95.55	96.11
Motion smoothness	98.39	98.61
Dynamic degree	55.36	53.13
Aesthetic quality	53.76	56.14
Imaging quality	67.87	67.15

Table 9: Long-video generation on MovieGen-128 at 50 seconds. Self-Forcing++ scores follow its reported results [ 6 ] . All scores are percentages; higher is better.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Shared space	Total $\uparrow$	Qual. $\uparrow$	Sem. $\uparrow$	Dyn. $\uparrow$
VideoMAE v2 L16	82.40	83.51	77.99	54.72
VideoMAE v2 L24	82.69	84.06	77.24	55.83
V-JEPA 2.1 L16	79.69	81.06	74.21	25.28
V-JEPA 2.1 L24	80.51	81.79	75.39	25.56
DiT L9	82.32	83.76	76.55	66.94

Appendix

Table 10: Shared representations for continuation alignment and rollout distribution matching. VideoMAE v2 L24 yields the highest Total and Quality, while DiT L9 gives the highest Dynamic.

Training decoder	Total $\uparrow$	Qual. $\uparrow$	Sem. $\uparrow$	Dyn. $\uparrow$
TAEW2.1	82.40	83.51	77.99	54.72
Wan VAE (truncated)	81.70	82.62	78.03	51.11

Appendix

Table 11: Training decoders with both components using VideoMAE v2 L16. Evaluation uses the original Wan VAE.

Steps	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$	Dynamic $\uparrow$
4	84.11	85.47	78.67	74.72
2	83.77	85.20	78.03	70.00
1	81.58	82.75	76.94	69.17

Appendix

Table 12: Denoising-step ablation of Direct Forcing after 150 updates. Steps refer to later chunks in RDM rollouts and inference; the first chunk uses four steps. CCA retains the four-level noise grid for all chunks.

Flow weight $w$	Total $\uparrow$	Qual. $\uparrow$	Sem. $\uparrow$	Dyn. $\uparrow$
0	82.40	83.51	77.99	54.72
$10^{-4}$	82.93	84.12	78.19	77.50
$5\times 10^{-4}$	83.46	84.87	77.84	90.56

Appendix

Table 13: Optical-flow regularization with both components using VideoMAE v2 L16. Increasing the weight improves Total and Dynamic.

Metric	0	$10^{-4}$	$5\times 10^{-4}$
Subject Consistency	95.69	95.51	95.47
Background Consistency	95.62	94.92	94.87
Temporal Flickering	99.15	97.72	97.73
Motion Smoothness	98.77	98.44	98.32
Dynamic Degree	54.72	77.50	90.56
Aesthetic Quality	65.07	64.85	64.72

Appendix

Table 14: Full VBench dimension scores for optical-flow regularization. Columns give flow weight

w

. Scores are multiplied by 100; higher is better for every metric.

Representation	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$	Dynamic $\uparrow$
Global pooling	80.75	81.84	76.39	55.56
Tokens	82.40	83.51	77.99	54.72

Appendix

Table 15: Pooling ablation for rollout distribution matching, with VideoMAE v2 L16 for both training components.

CCA representation	Total $\uparrow$	Qual. $\uparrow$	Sem. $\uparrow$	Dyn. $\uparrow$
V-JEPA 2.1 L16	81.14	82.01	77.65	28.61
V-JEPA 2.1 L24	81.62	82.47	78.24	35.56

Appendix

Table 16: V-JEPA 2.1 features for continuation alignment, with rollout distribution matching fixed to VideoMAE v2 L24.

Figure 8: Qualitative effects of the training schedule. Rows show CCA-only, sequential CCA initialization followed by RDM-only, and joint training. CCA-only and joint training use 150 updates; sequential training uses 150+150. All rows use frames 0/16/32 for the school case and 0/40/80 for the airliner case. Prompts are abbreviated.

CCA batch size	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$	Dynamic $\uparrow$
16	81.84	82.64	78.61	38.06
32	82.40	83.51	77.99	54.72
64	82.03	83.21	77.32	51.67

Appendix

Table 17: Sensitivity to the CCA batch size, using VideoMAE v2 L16 for both CCA and RDM. The default batch size is 32.

Continuation	Distribution	Total $\uparrow$	Qual. $\uparrow$	Sem. $\uparrow$	Dyn. $\uparrow$
$V_{24}$	$V_{16}$	82.80	84.00	78.01	58.06
$V_{24}$	$D_{9}$	83.49	84.96	77.58	68.61
$V_{24}$	$V_{16}+D_{9}$	84.11	85.47	78.67	74.72
$V_{24}$	$V_{16}+D_{9}+V_{24}$	83.86	85.31	78.05	77.78
$V_{24}+D_{9}$	$V_{16}$	83.35	84.36	79.32	56.39
$V_{24}+D_{9}$	$D_{9}$	83.04	84.37	77.72	65.00

Appendix

Table 18: Additional representation combinations after 150 updates.

V_{k}

: VideoMAE v2 L

k

;

D_{9}

: DiT L9. Gray marks the representation configuration used by our main model.

Perturbation	Clean	Noised	Reduction
Cross-video splicing	24.75	17.99	27.29%
Temporal freezing: 50% retained	4.31	3.33	22.68%

Appendix

Table 19: Response of clean and noised DiT L9 features at

\tau=300

. Distances are multiplied by 100; Reduction gives the relative decrease after noising.

CCA: RDM uses VideoMAE v2 L16
Layer	$\boldsymbol{\tau}$	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$	Dynamic $\uparrow$
L9	0	81.88	82.54	79.25	44.72
L9	300	82.56	83.53	78.68	58.89
L9	600	82.10	82.88	78.99	48.33
L9	900	82.02	83.02	78.05	65.28
L25	300	82.90	83.78	79.36	56.39

Appendix

Table 20: DiT layer and encoder time condition

\tau

for CCA and RDM. All DiT inputs are clean latents; the other component uses VideoMAE v2 L16. Gray marks L9 at

\tau=300

; best scores are bold within each component.

	ViRDM	Elastic Forcing	Direct Forcing
Dynamic $\uparrow$	48.61	63.33	75.56

Appendix

Table 21: Dynamic Degree in the main comparison.

Batch size	Update	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$	Dynamic $\uparrow$
64	150	84.11	85.47	78.67	74.72
128	250	84.63	86.01	79.08	75.56
256	175	83.87	85.24	78.36	71.94
512	75	83.63	85.00	78.15	65.28

Appendix

Table 22: RDM batch size with VideoMAE v2 L24 for CCA and VideoMAE v2 L16 plus DiT L9 for RDM. Each row reports the highest-Total checkpoint within the first 250 updates; CCA batch size is 32. Gray marks the main setting.

Method	Chunk boundary ratio ( $\approx 1$ )	Motion Smoothness $\uparrow$
ViRDM [ 24 ]	1.393 [1.332, 1.461]	98.12
Direct Forcing	1.040 [0.973, 1.129]	98.52

Appendix

Table 23: Temporal continuity of the ODE-initialized ViRDM pipeline and our single-stage pipeline, both without optical-flow regularization. Brackets denote 95% prompt-bootstrap confidence intervals. Boundary ratios closer to one are better.

Metric	Original data	Seedance 2.5
Dynamic Degree $\uparrow$	54.72	62.50
Motion Smoothness $\uparrow$	98.77	98.90
Subject Consistency $\uparrow$	95.69	93.81
Background Consistency $\uparrow$	95.62	93.80

Appendix

Table 24: Full VBench scores of Direct Forcing trained on the indicated reference data. Scores are percentages.

T2V Generation

Text-to-video generation creates moving visual scenes from natural-language prompts, with growing control over motion, identity, events, and style.

See all 113 papers

From Scores to Samples: Elastic Forcing for Autoregressive Video Generation
Sep 28, 2026Chi Zhang, Yueyi Liu, Shi Haoyang +5Autoregressive Video Generation
College of AI, Tsinghua University · IAIR, Xi’an Jiaotong University · Xianghui Academy, Fudan University +1
One-Forcing: Towards Stable One-Step Autoregressive Video Generation
May 22, 2026Jiaqi Feng, Justin Cui, Yuanhao Ban +1Autoregressive Video Generation One-Step Generative Modeling
Tsinghua University · UCLA
Custom Forcing: Training-Free Subject Customization for Autoregressive Video Generation
Oct 2, 2026Yunseung Ok, Hyunsoo Kim, Minseo Kim +1Identity-Preserving Video Generation Streaming Video Generation
Kyung Hee University · The University of Texas at Austin

Sep 28, 2026J/K move · Enter open · S save

From Scores to Samples: Elastic Forcing for Autoregressive Video Generation

Chi Zhang, Yueyi Liu, Shi Haoyang, Ruichuan An+4

College of AI, Tsinghua University · IAIR, Xi’an Jiaotong University · Xianghui Academy, Fudan University+1

Autoregressive Video Generation

Open paper AskPDF arXiv

Few-step autoregressive video generation commonly relies on Distribution Matching Distillation (DMD), requiring a bidirectional diffusion teacher and an online fake-score model. We instead learn the rollout distribution directly from reference videos, eliminating both score models during post-training. Our framework minimizes maximum mean discrepancy (MMD) in frozen self-supervised video representation spaces, using a hybrid Nyström--Monte Carlo estimator to balance approximation bias and sampling variance. Memory-efficient replay and gradient subsampling make this objective practical. Using the same architecture and initialization as Self-Forcing, our 1.3B model improves the VBench Total score from 83.80 to 84.64 while retaining 17 FPS. Removing auxiliary score models also enables 14B post-training on eight H200 GPUs. Beyond distillation, learning from reference videos enables the acquisition of new visual styles, semantic concepts, and spatial priors without a target-specific diffusion teacher.

Abstract

T2V Generation

Explore similar work

From Scores to Samples: Elastic Forcing for Autoregressive Video Generation

One-Forcing: Towards Stable One-Step Autoregressive Video Generation

Custom Forcing: Training-Free Subject Customization for Autoregressive Video Generation