Few-step autoregressive video diffusion enables efficient generation, yet maintaining consistency over long rollouts remains challenging, especially when subjects or scenes reappear after going out of sight. Despite progress in fidelity and rollout robustness, existing distillation methods do not fully exploit richer historical information to constrain the student's current prediction. To address this gap, we introduce History Forcing, a unified on-policy distillation framework that strengthens historical supervision for few-step autoregressive video generators. The framework uses the residual between teacher predictions with and without informative historical context to decouple this supervision. Such a historical context can be obtained using earlier generated chunks with less accumulated drift or a fixed-view teacher rollout that keeps the scene visible while the student looks away. The residual is then weighted and added to the score or velocity prediction used for distillation, forming a corrected target. By aligning the student with this corrected target through distillation, History Forcing incorporates long-range historical constraints into autoregressive generation. Our method achieves state-of-the-art performance among 1.3B models, with a VBench quality score of 85.07 for 15s video generation and a VBench-Long aggregate score of 79.82 for 60s generation. For out-of-sight world modeling, evaluations further demonstrate more plausible scene evolution and more consistent states when the camera returns, with Evolution Quality improving from 2.35 to 2.70 on PlayWorld and Object Consistency from 58.43 to 62.18 on MemoBench.
Figures & tables
Figure 2: Comparison of distillation strategies. (a) DMD uses a bidirectional teacher’s prediction as the reference for the current chunk. (b) AR distillation directly uses the history-conditioned prediction of an AR teacher as the full target. (c) History Forcing uses the context residual between the AR teacher’s predictions with and without history to correct the reference prediction.
Figure 3: Overview of History Forcing. An AR teacher predicts the same noisy student chunk with and without additional history. The resulting history contrast modifies the reference score in DMD and the velocity target in on-policy self-distillation.
Figure 4: Comparison of distillation targets, where + and − denote distributions with and without additional history. (a) DMD uses a reference close to the data distribution without history. (b) AR Distillation uses the history-conditioned AR distribution, with larger modeling error. (c) History Forcing transfers the history-induced shift from the AR teacher to the reference, producing a target closer to the data distribution with history.
Method
Student Size
Teacher Size
Aggregate Score ↑
Subject Consistency ↑
Background Consistency ↑
Motion Smoothness ↑
Dynamic Degree ↑
Aesthetic Quality ↑
Imaging Quality ↑
Other model sizes
NOVA
0.6B
—
64.22
77.50
88.06
98.94
12.00
47.53
44.97
MAGI-1
4.5B
—
71.33
79.46
87.76
99.26
56.00
52.10
54.54
LongLive-2.0
5B
5B
79.90
97.48
97.00
98.86
60.62
53.68
65.51
1.3B models
Causal Forcing
1.3B
14B
76.43
93.52
94.12
95.74
72.32
51.24
62.30
Table 2: Comparison on VBench-Long for 60s video generation. Results for competing methods are taken from LongLive-2.0 [ 11 ] . Aggregate Score uses VBench normalization and weights over the six metrics. Teacher Size refers to the DMD teacher. History Forcing uses the AR teacher fine-tuned on 10s videos. Best and second-best results among 1.3B models are bold and underlined.
Table 5
Table 6: Ablation study on historical supervision strength for video world modeling.
Figure 5: Qualitative comparisons of consistency. (a) Temporal consistency in 60s autoregressive video generation with Self-Forcing, LongLive, Rolling Forcing, and History Forcing. (b,c) Revisit consistency in video world models, with each pair showing the baseline and History Forcing.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Additional qualitative comparisons of out-of-sight world modeling with the baseline and History Forcing: (a) sand filling, (b) tomatoes into a bowl, (c) coffee pouring, and (d) purple solution pouring.
Figure 7: Additional qualitative comparisons of 15s video generation: (a) a violin performance in a candle-lit chapel, (b) a desert carnival, (c) bread baking, (d) a Tokyo street, (e) a glacial cave, and (f) a Hong Kong tram.
Figure 8: Additional qualitative comparisons of 60s video generation: brushing teeth in front of a mirror.
Figure 9: Additional qualitative examples of 60s video generation with History Forcing: (a) making dumplings, (b) playing an acoustic guitar, and (c) a chef preparing a dish in a kitchen.
Autoregressive video diffusion models have emerged as a foundation for long-horizon world rollout. However, when extended far beyond their training windows, these models suffer from progressive drift: visual quality deteriorates, motion becomes inconsistent, and object identity gradually collapses as generated frames are recursively reused as context. We study a supervision-horizon gap: short-horizon distillation leaves later rollout states without direct supervision. We show that matching short-window marginals need not determine long-horizon behavior, while longer on-policy trajectories expose delayed failures to training. This motivates temporal manifold matching at scale for long-horizon world models. Building on this insight, we introduce LongForcing, a distillation approach that scales teacher supervision to long student self-rollouts. LongForcing preserves the short-horizon teacher-forcing and causal ODE-distillation stages, while replacing the realscore teacher in the final distribution-matching stage with an extended-horizon bidirectional teacher. The long-horizon teacher supervises extended student self-rollouts using the standard distribution-matching formulation, without modifying the student architecture or increasing inference cost. We evaluate LongForcing on text-to-video generation and interactive action-conditioned world models, achieving a VBench Quality score of 85.52 and improving WorldRoamBench 60s action score by 23.9 points in first-person ablations. Qualitative stress tests further sample interactive rollouts up to 24 hours, far beyond the training horizon. These results suggest that scaling supervision along the temporal axis is a practical step toward long-horizon world models. The project website and code are at https://github.com/Fantasy-AMAP/LongForcing.
We propose OPSD-V, an on-policy self-distillation paradigm for post-training few-step autoregressive (AR) video diffusion models. Existing few-step AR video generators can produce long videos with low latency, but still suffer from error accumulation and weakened motion dynamics during long autoregressive rollout. OPSD-V reduces long-horizon degradation while preserving the original few-step inference path. The key idea is to introduce real long-video data as temporal context during training and use it to provide dense trajectory-level supervision. Specifically, the student follows the exact inference-time rollout, generating each chunk conditioned on its own previously generated KV cache. In parallel, the teacher is evaluated at the same student-visited denoising states, but uses a cleaner AR-consistent temporal cache in which older history can be replaced by real-video context. This provides dense denoising-level corrective targets under on-policy AR cache dynamics, without changing the sampler, number of denoising steps, or inference-time cache mechanism. We apply OPSD-V to representative few-step AR video models, including Self-Forcing and LongLive. Experiments show consistent improvements in visual quality, motion dynamics, and VBenchLong scores. A user study with 10 participants comparing 20 video pairs shows that OPSD-V is preferred over the base models in 66.0% of overall-preference judgments (82.5% excluding ties).
Autoregressive video diffusion models hold promise for world simulation but are vulnerable to exposure bias arising from the train-test mismatch. While recent works address this via post-training, they typically rely on a bidirectional teacher model or discriminator. To achieve an end-to-end solution, we introduce Resampling Forcing, a teacher-free framework that enables training autoregressive video models from scratch and at scale. Central to our approach is a self-resampling scheme that simulates inference-time model errors on history frames during training. Conditioned on these degraded histories, a sparse causal mask enforces temporal causality while enabling parallel training with frame-level diffusion loss. To facilitate efficient long-horizon generation, we further introduce history routing, a parameter-free mechanism that dynamically retrieves the top-k most relevant history frames for each query. Experiments demonstrate that our approach achieves performance comparable to distillation-based baselines while exhibiting superior temporal consistency on longer videos owing to native-length training.
Yuwei Guo, Ceyuan Yang, Hao He +5
The Chinese University of Hong Kong, Hong Kong, China · ByteDance Seed, China · ByteDance, China