TriTD: Tri-Partite Trajectory-Distribution Distillation for Real-Time Autoregressive Video Generation

Autoregressive (AR) video diffusion enables low-latency, streamable video generation, but prediction errors often accumulate over long rollouts. Training the generator on its own rollouts exposes it to these imperfect histories. However, existing video-level distribution matching distillation (DMD) scores the whole rollout jointly. Because a chunk is evaluated together with its past and future, its correction can favor matching artifacts in the surrounding context merely to preserve temporal consistency. To provide a clearer visual-quality signal, we introduce Rollout-Marginal Distillation (RMD). RMD retains the generated history for AR prediction but scores each chunk independently against a chunk teacher, ensuring its quality correction is not compromised by an imperfect temporal context. To compensate for the lack of temporal context in independent chunk scoring, RMD subsequently applies video-level DMD to restore temporal coherence. Extensive experiments demonstrate that RMD maintains high visual quality far beyond its training horizon and outperforms video-level DMD baselines. Code and video results are available at https://cjeen.github.io/RMD

Autoregressive (AR) video generation enables scalable long-video synthesis by sequentially generating fixed-length chunks, but incurs substantial computational cost from repeated denoising and attention over growing historical contexts. Recent cache reuse methods exploit temporal redundancy to reduce this cost, yet are mainly designed for single-trajectory generation, making them less suitable for sequentially coupled AR trajectories where approximation errors can accumulate and propagate across chunks. To address this challenge, we propose TRAC, a Trajectory-aware Reuse and Adaptive Correction framework for efficient and robust AR video generation. TRAC integrates three complementary components: (1) Robust Cumulative Scheduling (RCS), which selects cache reuse schedules by accounting for cumulative rollout errors and their variation across chunks and prompts; (2) Autoregressive Trajectory-aware Guidance Scheduling (ATGS), which coordinates classifier-free guidance refreshes along the global AR trajectory; and (3) Spectral Structure Correction (SSC), which adaptively restores low-frequency structural information to mitigate long-term error accumulation. By combining these innovations, TRAC substantially improves the efficiency-quality trade-off of AR video generation. For example, on VBench with SkyReels-V2, it achieves a 6.03x speedup over the vanilla model with only ~0.6% VBench score degradation, while consistently outperforming existing AR acceleration methods in both latency and generation quality. The code is publicly available at https://github.com/yolosjxhhhhhh/trac.

Large pretrained diffusion models have significantly enhanced the quality of generated videos, and yet their use in real-time streaming remains limited. Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity. Diffusion distillation can compress these models into efficient few-step variants, but existing video distillation approaches largely adapt image-specific methods that neglect temporal dependencies. These techniques often excel in image generation but underperform in video synthesis, exhibiting reduced motion coherence, error accumulation over long sequences, and a latency-quality trade-off. We identify two factors that result in these limitations: insufficient utilization of temporal context during step reduction and implicit prediction of subsequent noise levels in next-chunk prediction (i.e., exposure bias). To address these issues, we propose Diagonal Distillation, which operates orthogonally to existing approaches and better exploits temporal information across both video chunks and denoising steps. Central to our approach is an asymmetric generation strategy: more steps early, fewer steps later. This design allows later chunks to inherit rich appearance information from thoroughly processed early chunks, while using partially denoised chunks as conditional inputs for subsequent synthesis. By aligning the implicit prediction of subsequent noise levels during chunk generation with the actual inference conditions, our approach mitigates error propagation and reduces oversaturation in long-range sequences. We further incorporate implicit optical flow modeling to preserve motion quality under strict step constraints. Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.