Autoregressive video diffusion models generate frames based on new observations and inputs, making them a promising medium for interactive visual experiences. However, existing models are orders of magnitude too slow for fluid interaction. This paper introduces FastFrames, a family of fast autoregressive video diffusion models that achieves interactive latency without compromising quality. We profile the inference pipeline of state-of-the-art models and find that operations often treated as fixed implementation details, such as VAE decoding and KV cache updates, in fact account for over half of end-to-end inference latency. We challenge the assumption that these components are indispensable and replace them with lightweight alternatives. Our 1.3B variant runs at 36 ms per frame, or 28 responses per second (RPS), on a single B200 GPU while matching the generation quality of a state-of-the-art model that is nearly 8x slower. Our 14B variant runs at 45 ms per frame, or 22 RPS, on an 8x B200 GPU node and yields significantly better visual quality and realism. We show that FastFrames enables real-time interaction in applications such as camera-conditioned and physics-conditioned video generation.
Figures & tables
Figure 1: Quality-speed tradeoff. VBench scores vs. per-frame latency. Upper left is better. Only FastFrames meets an acceptable threshold ( Deber et al., 2015 ) for seamless interactivity ( ≤50 ms latency, equivalently ≥20 responses per second (RPS)) while matching or exceeding comparable baselines in visual quality.
Figure 2: Inference pipeline and latency profiling for one autoregressive generation. State-of-the-art few-step autoregressive video diffusion models generate each new frame by first updating the KV cache using the previous frame’s denoised latent ( z0i−1 ) through a DiT evaluation, then performing multiple denoising steps followed by VAE decoding. The color gradient of the model components denotes speed: from white (fastest operations) to red (slowest). (a) The fastest baseline, Causal Forcing++ ( Zhao et al., 2026a ) , takes 271 ms to generate a frame at 480p but only 30% of wall time is spent on diffusion sampling. (b) FastFrames reduces latency nearly 8× by replacing the non-denoising “accessories” with lightweight alternatives and performing systems-level optimizations to speed up each DiT evaluation.
Figure 3: Comparing pixel decoders. Our pixel decoder (right) renders thin structures and details like hair and raindrops while being two orders of magnitude faster than Wan’s VAE (left). A single linear projection (middle) cannot render sharp outputs, but is sufficient for recognizing the scene and suggests that pixel rendering does not require an expensive decoder.
VBench ↑
Model
Size (B)
Latency (ms) ↓
Total
Quality
Semantic
Bidirectional Video Models
Wan2.1 ( Wang et al., 2025 )
1.3
30,444
83.80
84.56
80.73
Wan2.2 ( Wang et al., 2025 )
14.3
117,734
84.65
85.41
81.60
MiniMax-H3 ( MiniMax, 2026 )
33.1
38,454
84.43
85.17
81.47
HunyuanVideo1.5 ( Wu et al., 2025a )
8.3
1,555,508
83.18
84.14
79.35
Table 1: Quantitative comparisons in terms of end-to-end latency and VBench scores. All methods use the same evaluation prompts and seeds, run on one B200 GPU unless otherwise indicated, and are evaluated at approximately 480p using their officially supported dimensions. Green indicates latency that meets the interactive target ( ≤50 ms); yellow indicates sub-second latency ( 50<latency<1,000 ms).
Figure 4: Qualitative comparisons. We compare FastFrames -S with Causal Forcing++ at 1.3B parameters and FastFrames -L with Krea-Realtime at 14B. Each example shows frames at 0 and 5 seconds. FastFrames -S achieves comparable visual sharpness to Causal Forcing++, such as the details in the sculpture (first example) and leaves (second example), while being nearly 8× faster. FastFrames -L achieves comparable visual realism to Krea-Realtime, including composition of “paper origami dragon” (third), and legible text of “stay curious” (fourth) while being 10× faster. See our website for video playback.
Figure 5: Interactive generation and latency results. Left: example of interaction with LingBot’s official implementation vs. with FastFrames optimizations. Given a single keypress, FastFrames is 16× faster to respond. Right: generation latency of officially released implementations and our optimized implementations. We measure steady-state latency on NVIDIA B200 GPUs at 832×480 resolution with a full KV cache and include distributed optimizations such as Ulysses where officially supported. Times include diffusion sampling (four steps for LingBot and three for RealWonder), action conditioning, KV-cache updates, and pixel decoding. Our optimizations provide speedups of 3× to 22×. See the project website for the video demonstration.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
Resolution
B200 Latency (ms)
H100 Latency (ms)
FastFrames -S (1.3B) on 1 GPU
DiT evaluation
832×480
16
25
DiT evaluation
1280×720
60
101
KV refinement network
832×480
2
3
KV refinement network
1280×720
5
7
VAE
832×480
147
175
Appendix
Table 2: Additional latency profiling at 720p resolution and on H100 GPUs . FastFrames -S runs in 36 ms at 480p and in 127 ms at 720p on a B200 GPU. FastFrames -L runs in 45 ms at 480p and in 116 ms at 720p on 8 B200 GPUs. Savings from our lightweight network replacements are more noticeable at 720p. H100s are approximately 25% to 50% slower depending on the configuration.
Figure 6: KV refinement ablation. Autoregressive baselines refine the KV cache using an additional DiT evaluation, which is costly but effective. Removing that refinement naively (third column) substantially degrades the scene’s consistency; the objects in the scenes change over time. Even training without the refinement (fourth column) does not fix the problem. Either the subject changes over time (the face changes in the second example), or noticeable artifacts appear (clothing in the first example, fingers in the second example). Our KV refinement network (second column) replaces the DiT evaluation and performs equally while being an order of magnitude faster.
Video diffusion models generate or edit temporally coherent video by iteratively denoising latent sequences conditioned on text, images, motion, or other signals.
Real-time interactive video generation requires low-latency, streaming, and controllable rollout. Existing autoregressive (AR) diffusion distillation methods have achieved strong results in the chunk-wise 4-step regime by distilling bidirectional base models into few-step AR students, but they remain limited by coarse response granularity and non-negligible sampling latency. In this paper, we study a more aggressive setting: frame-wise autoregression with only 1--2 sampling steps. In this regime, we identify the initialization of a few-step AR student as the key bottleneck: existing strategies are either target-misaligned, incapable of few-step generation, or too costly to scale. We propose Causal Forcing++, a principled and scalable pipeline that uses causal consistency distillation (causal CD) for few-step AR initialization. The core idea is that causal CD learns the same AR-conditional flow map as causal ODE distillation, but obtains supervision from a single online teacher ODE step between adjacent timesteps, avoiding the need to precompute and store full PF-ODE trajectories. This makes the initialization both more efficient and easier to optimize. The resulting pipeline, \ours, surpasses the SOTA 4-step chunk-wise Causal Forcing under the frame-wise 2-step setting by 0.1 in VBench Total, 0.3 in VBench Quality, and 0.335 in VisionReward, while reducing first-frame latency by 50% and Stage 2 training cost by \sim$$4\times. We further extend the pipeline to action-conditioned world model generation in the spirit of Genie3. Project Page: https://github.com/thu-ml/Causal-Forcing and https://github.com/shengshu-ai/minWM .
Min Zhao, Hongzhou Zhu, Kaiwen Zheng +6
Tsinghua University · ShengShu · Renmin University of China
Recent progress in large-scale generative models has substantially advanced video generation, yet existing methods remain constrained by a rigid inference paradigm. Bidirectional diffusion models excel at global coherence and visual fidelity but suffer from slow inference, while autoregressive models offer efficient and streaming generation at the cost of long-range consistency and exposure bias. We introduce Flex-Forcing, a unified training and inference framework that enables a video diffusion model to seamlessly operate under both bidirectional and autoregressive generation regimes. The core idea is a flexible chunking mechanism jointly defined over the temporal axis and denoising steps. This design allows the model to (1) perform flexible chunking according to different device budgets, (2) perform bidirectional inference across chunks for global structure planning, while generating frames autoregressively within each chunk for efficient and fine-grained synthesis, and (3) perform any-order, any-timestep autoregressive generation without the strict causal constraint. Extensive experiments on multiple video generation benchmarks demonstrate that Flex-Forcing achieves consistently better video quality, long-video stability than strong baselines with a rigid inference schedule, while offering faster inference.
Xinyin Ma, Julius Berner, Chao Liu +3
Electrical and Computer Engineering, National University of Singapore, Singapore · NVIDIA Research, California, USA
Few-step autoregressive video generators enable streaming generation, yet they run every token through the final denoising step, regardless of how much refinement it needs. We find this uniform computation is often unnecessary: many tokens already closely match their eventual full-step outputs in the initial steps, while others require refinement until the end. We introduce Speculative Forcing, a framework that treats intermediate clean estimates as drafts and predicts when individual tokens can be safely accepted early. Once accepted, a token is frozen as its final output and undergoes no further denoising, while its cached keys and values remain available as context for the remaining tokens. We explore two alternative verifier designs: a training-free one based on similarity to clean content and stability across steps, and a learned one that reads the frozen generator's intermediate features to estimate each token's remaining gap to its full-step output. Across short-video, long-video, and world generation, Speculative Forcing preserves the quality of full-step sampling while substantially reducing computation. On LongLive, it matches the four-step generator with ~1.7 denoising steps per token, running over 1.5x faster at 41.7 and 37.8 fps. On Lingbot-World 2.0, where camera motion continually changes the visible content, it maintains quality with ~2.5 denoising steps per token and runs 1.35x faster at 24.5 fps.