Can autoregressive video diffusion be accelerated further when only four denoising steps remain? Existing feature caching approaches struggle in this regime, as large feature changes between denoising timesteps undermine both direct reuse and lightweight predictors. We observe that features across video chunks remain strongly correlated at matching denoising timesteps, offering a complementary source of guidance. Building on this insight, we introduce TGuide, a framework that accelerates autoregressive video diffusion through a lightweight Anchor-Transport-Correct (ATC) predictor guided by historical sampling trajectories. Our ATC combines two complementary references: the current chunk's preceding-step features preserve its spatial structure, while the previous chunk's same-timestep features provide information at the target denoising stage. By integrating these references through spatial retrieval and adaptive correction, ATC improves velocity prediction accuracy despite the large gaps between denoising steps. To mitigate error accumulation across both denoising steps and video chunks, we train the predictor on its own autoregressive rollouts, allowing it to adapt to states shaped by its own earlier predictions. An optional confidence-aware scheduler further adjusts the quality-efficiency trade-off. Experiments on Self Forcing, Causal Forcing, and HY-WorldPlay show that TGuide preserves generation quality while achieving denoising speedups of up to 1.89× over the original 4-step generators. On Self Forcing, TGuide generalizes from 81-frame training videos to 333-frame generation, approximately four times the training length, while exceeding the original model's VBench score.
Figures & tables
Figure 1: Motivation and overview of TGuide. Curves and arrows schematically illustrate sampling trajectories and velocity relationships. (a) With many denoising steps, nearby velocities can be similar, making direct reuse a useful approximation. (b) Step distillation increases the discrepancy between adjacent step velocities, leading to large errors in both direct reuse and existing prediction methods. (c) Past sampling trajectories guide current velocity prediction: ATC combines the previous chunk’s velocity at the matching timestep with the current chunk’s velocity from an earlier step to improve prediction accuracy in 4-step generation. (d) On Self Forcing with 333-frame generation, our TGuide achieves a 1.89× speedup over the original 4-step baseline while improving the VBench score by 0.16 points. Bubble sizes indicate computational cost in TFLOPs.
Figure 2: (a) Correlation along denoising time and video time. The 4-step autoregressive generators exhibit lower cosine similarity of final hidden states across denoising timesteps than the 50-step teacher references, while correlations across video chunks remain largely preserved. Optical-flow alignment further improves cross-chunk similarity. (b) Global impact of local approximation. Velocity reuse in a single chunk generally causes larger full-video deviations from a full-compute reference when applied earlier in the rollout, motivating more conservative approximation decisions for earlier chunks. The horizontal axis indicates the index of the chunk perturbed by velocity reuse. See Appendix E for diagnostic protocols.
Figure 3: Overview of TGuide. (a) ATC replaces selected full DiT forward passes by predicting final hidden states from current tokens xemb , the current chunk’s anchor a=hiaj , and the previous chunk’s reference p=hij−1 at the same timestep. Optional confidence-aware scheduling accepts predictions when r≤τ and otherwise invokes the full DiT. (b) The anchor predictor estimates the current hidden state, Transport retrieves relevant historical features through cross-attention, and Correct applies a token-wise gated correction to produce h^ .
Figure 4: Qualitative comparisons on Self Forcing, Causal Forcing, and HY-WorldPlay.
Method
81 frames
333 frames
Latency (s) ↓
Speedup ↑
Total ↑
Quality ↑
Semantic ↑
Latency (s) ↓
Speedup ↑
Total ↑
Quality ↑
Semantic ↑
Original 4 steps
6.53
1.00 ×
84.28
84.63
82.87
32.93
1.00 ×
82.55
82.93
81.04
Original 2 steps
3.48
1.88 ×
84.09
84.46
82.59
16.53
1.99 ×
82.35
82.76
80.71
TeaCache
3.83
1.71 ×
83.91
84.14
83.01
17.50
1.88 ×
81.47
81.82
80.06
TaylorSeer
3.78
1.73 ×
83.68
83.93
82.69
18.67
1.76 ×
80.69
81.05
79.26
MotionCache
3.96
1.65 ×
83.97
84.25
82.85
19.12
1.72 ×
81.56
81.89
80.24
Table 1: Performance comparison on Self Forcing using the full VBench benchmark.
Method
Quality and efficiency
vs. Original 4 steps
Self-comparison
VBench ↑
Latency (s) ↓
Speedup ↑
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Original 4 steps
84.26
49.61
1.00 ×
—
—
—
16.75
0.5107
0.2820
Original 2 steps
84.18
26.81
1.85 ×
19.72
0.7122
0.1480
16.68
0.5124
0.2963
TeaCache
84.20
30.06
1.65 ×
18.63
0.7230
0.1413
16.44
0.5088
0.3026
TaylorSeer
84.21
29.76
1.67 ×
18.60
0.6845
0.1670
16.60
0.5128
0.2983
MotionCache
84.24
30.02
1.65 ×
18.59
0.6843
0.1672
16.60
0.5135
0.2986
Table 2: Performance comparison on HY-WorldPlay at 125 frames.
Method
Total ↑
TFLOPs ↓
Latency (s) ↓
Speedup ↑
Original 4 steps
84.28
793
6.53
1.00 ×
FRRF
83.14
429
3.42
1.91 ×
w/o p
83.86
444
3.59
1.82 ×
Concat p
83.98
445
3.59
1.82 ×
Anchor + Transport
84.05
448
3.60
1.81 ×
ATC
84.08
452
3.62
1.80 ×
Table 3: ATC ablation under Stage 1 training.
Training
Predictor
81 frames
333 frames
Total ↑
Speedup ↑
Total ↑
Speedup ↑
Original 4 steps
84.28
1.00 ×
82.55
1.00 ×
Stage 1
DisCa
83.86
1.82 ×
82.12
1.90 ×
ATC
84.08
1.80 ×
82.29
1.89 ×
+ Stage 2
DisCa
84.09
1.82 ×
82.37
1.90 ×
ATC
84.23
1.80 ×
82.71
1.89 ×
Table 4: Effect of Stage 2 training.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Self Forcing: Slow
Causal Forcing
VBench ↑
TFLOPs ↓
Latency (s) ↓
Speedup ↑
VBench ↑
TFLOPs ↓
Latency (s) ↓
Speedup ↑
Original 4-step
80.68
793
6.53
1.00 ×
81.58
793
6.48
1.00 ×
Original 3 / 2-step
80.42
611
4.84
1.35 ×
81.05
429
3.49
1.86 ×
TeaCache
80.47
611
4.98
1.31 ×
81.17
493
3.99
1.62 ×
TaylorSeer
80.40
611
5.06
1.29 ×
80.60
457
3.76
1.72 ×
MotionCache
80.35
626
5.04
1.30 ×
80.85
451
3.75
1.73 ×
Appendix
Table 5: Additional comparisons on 81-frame videos using the VBench subset. Ours uses dynamic K08 on Self Forcing and fixed FPPF on Causal Forcing. Naive step reduction uses three and two steps, respectively.
81 frames
Dimension
Original 4-step
Original 2-step
TeaCache
TaylorSeer
MotionCache
DisCa
Ours
Subject Consistency
96.19
96.16
96.15
95.52
95.79
95.80
96.17
Background Consistency
96.96
96.90
97.01
95.72
96.88
96.93
97.22
Temporal Flickering
98.98
99.12
99.17
99.08
99.08
99.10
99.19
Motion Smoothness
98.63
98.58
98.70
98.72
98.68
98.58
98.68
Dynamic Degree
58.33
56.94
51.39
52.78
54.17
54.17
55.56
Appendix
Table 6: Per-dimension and aggregate VBench results on 81- and 333-frame Self Forcing generation. Dimension scores are raw values multiplied by 100 ; Quality, Semantic, and Total Scores are normalized weighted percentages. Higher is better for all metrics. Original denotes the original 4-step model. Bold indicates the best accelerated method in each frame setting; aggregate rankings use scores before final rounding.
Figure 5: Additional qualitative comparisons on Self Forcing. Each example shows frames at 0 , 10 , and 20 seconds under matched generation conditions, with denoising speedups indicated for each method.
Method
VBench ↑
TFLOPs ↓
Latency (s) ↓
Speedup ↑
Original 4 steps
80.68
793
6.53
1.00 ×
FPFF
80.55
622
4.99
1.31 ×
FFPF
80.57
622
4.99
1.31 ×
Dynamic K06
80.62
613
5.10
1.28 ×
Dynamic K08
80.65
559
4.67
1.40 ×
Dynamic K10
80.58
507
4.24
1.54 ×
Appendix
Table 7: Dynamic schedule ablation on 81-frame Self Forcing videos, evaluated on the VBench subset.
Source layer
Val. MSE ↓
Source layer
Val. MSE ↓
Source layer
Val. MSE ↓
0
0.052675
10
0.049378
20
0.047325
1
0.051242
11
0.048483
21
0.046994
2
0.048375
12
0.049627
22
0.058777
3
0.048407
13
0.048251
23
0.048755
4
0.048115
14
0.049588
24
0.047499
5
0.048193
15
0.049718
25
0.048287
Appendix
Table 8: Validation velocity MSE across all source layers.
Initialization
Val. MSE ↓
Complete teacher block
0.046459
Teacher + zeroed branch outputs
0.055906
Xavier + zeroed branch outputs
0.057737
Xavier
0.057085
All-zero block
0.060460
Appendix
Table 9: Initialization controls with layer-17 K/V fixed, evaluated by validation velocity MSE.
Autoregressive video generation creates videos frame by frame or in chunks, conditioning each prediction on prior context for streaming and interactive use.
Autoregressive video diffusion models have become a leading approach to real-time interactive video generation and video world modeling. However, existing methods typically allocate computation uniformly across space when generating the next video chunk, updating all patches with the same denoising schedule and number of function evaluations. This ignores the spatial heterogeneity of video content: static, simple, or recurring regions are often easier to denoise, while other regions require further refinement or additional context. A uniform budget thus leads to redundant computation on easy patches, whereas reducing the fixed step count can leave difficult patches insufficiently refined. We introduce \methodname{}, a general framework that dynamically allocates denoising steps to individual patches: easy patches stop early to improve efficiency, while difficult patches receive more refinement steps. This design accelerates generation with negligible quality degradation. Our method is compatible with most existing few-step autoregressive diffusion models, requiring only a lightweight routing module with modest training. Experiments demonstrate that \methodname{} reduces the total number of denoising steps applied across video patches by about 50% while maintaining visual quality comparable to the four-step baseline.
Zekai Gu, Peng Wang, Jiahao Lu +3
HKUST · VAST · Macau University of Science and Technology
Autoregressive video diffusion models have enabled the generation of arbitrarily long videos by removing conditioning on future frames, thus greatly improving computational efficiency. Yet, they suffer from error accumulation over time, as the denoised sequence gradually drifts away from the conditioning distribution seen during training. Recent advances attempt to reduce this error by anchoring each generated frame to the learned manifold of real ones. However, even when all generated individual frames lie close to the real manifold, there are trajectories which the model lacks sufficient knowledge to continue without exiting it, thus reaching a terminal point. To prevent the model from being trapped in terminal points, we start from the hypothesis that for well-modeled future trajectories the distribution of the predicted noise should match the one of the forward noising process. To enforce such a prior at test time, we introduce Terminal points Avoidance through Noise Guided Optimization (TANGO), which uses the diffusion model as a critic of its own outputs, by predicting one step forward and requiring an isotropic Gaussian noise prediction. We use the deviation from this expected noise distribution to search for an alternative trajectory that does not lead to a terminal point. Our approach achieves a 3.1% absolute improvement on VBench over state-of-the-art, while reducing Fréchet Video Distance by 28.3% on average across 15s videos. Our code is available on https://mever-team.github.io/tango.
Dimitrios Karageorgiou, Symeon Papadopoulos, Ioannis Kompatsiaris +1
Information Technologies Institute, CERTH, Thessaloniki, Greece · University of Amsterdam, Amsterdam, The Netherlands
Few-step autoregressive video generators enable streaming generation, yet they run every token through the final denoising step, regardless of how much refinement it needs. We find this uniform computation is often unnecessary: many tokens already closely match their eventual full-step outputs in the initial steps, while others require refinement until the end. We introduce Speculative Forcing, a framework that treats intermediate clean estimates as drafts and predicts when individual tokens can be safely accepted early. Once accepted, a token is frozen as its final output and undergoes no further denoising, while its cached keys and values remain available as context for the remaining tokens. We explore two alternative verifier designs: a training-free one based on similarity to clean content and stability across steps, and a learned one that reads the frozen generator's intermediate features to estimate each token's remaining gap to its full-step output. Across short-video, long-video, and world generation, Speculative Forcing preserves the quality of full-step sampling while substantially reducing computation. On LongLive, it matches the four-step generator with ~1.7 denoising steps per token, running over 1.5x faster at 41.7 and 37.8 fps. On Lingbot-World 2.0, where camera motion continually changes the visible content, it maintains quality with ~2.5 denoising steps per token and runs 1.35x faster at 24.5 fps.