TGuide: Faster Autoregressive Video Diffusion via Historical Trajectory Guidance

Zhuo Chen; Bin Zou; Fanyue Wei; Lixin Duan; Angela Yao; Wen Li

cs.CVOct 10, 2026

TGuide: Faster Autoregressive Video Diffusion via Historical Trajectory Guidance

Authors: Zhuo Chen, Bin Zou, Fanyue Wei, Lixin Duan, Angela Yao, Wen Li

Organizations: University of Electronic Science and Technology of China · University of Hong Kong · National University of Singapore

Abstract

Can autoregressive video diffusion be accelerated further when only four denoising steps remain? Existing feature caching approaches struggle in this regime, as large feature changes between denoising timesteps undermine both direct reuse and lightweight predictors. We observe that features across video chunks remain strongly correlated at matching denoising timesteps, offering a complementary source of guidance. Building on this insight, we introduce TGuide, a framework that accelerates autoregressive video diffusion through a lightweight Anchor-Transport-Correct (ATC) predictor guided by historical sampling trajectories. Our ATC combines two complementary references: the current chunk's preceding-step features preserve its spatial structure, while the previous chunk's same-timestep features provide information at the target denoising stage. By integrating these references through spatial retrieval and adaptive correction, ATC improves velocity prediction accuracy despite the large gaps between denoising steps. To mitigate error accumulation across both denoising steps and video chunks, we train the predictor on its own autoregressive rollouts, allowing it to adapt to states shaped by its own earlier predictions. An optional confidence-aware scheduler further adjusts the quality-efficiency trade-off. Experiments on Self Forcing, Causal Forcing, and HY-WorldPlay show that TGuide preserves generation quality while achieving denoising speedups of up to $1.89\times$ over the original 4-step generators. On Self Forcing, TGuide generalizes from 81-frame training videos to 333-frame generation, approximately four times the training length, while exceeding the original model's VBench score.

Figures & tables

Figure 1: Motivation and overview of TGuide. Curves and arrows schematically illustrate sampling trajectories and velocity relationships. (a) With many denoising steps, nearby velocities can be similar, making direct reuse a useful approximation. (b) Step distillation increases the discrepancy between adjacent step velocities, leading to large errors in both direct reuse and existing prediction methods. (c) Past sampling trajectories guide current velocity prediction: ATC combines the previous chunk’s velocity at the matching timestep with the current chunk’s velocity from an earlier step to improve prediction accuracy in 4-step generation. (d) On Self Forcing with 333-frame generation, our TGuide achieves a $1.89\times$ speedup over the original 4-step baseline while improving the VBench score by $0.16$ points. Bubble sizes indicate computational cost in TFLOPs.

Figure 2: (a) Correlation along denoising time and video time. The 4-step autoregressive generators exhibit lower cosine similarity of final hidden states across denoising timesteps than the 50-step teacher references, while correlations across video chunks remain largely preserved. Optical-flow alignment further improves cross-chunk similarity. (b) Global impact of local approximation. Velocity reuse in a single chunk generally causes larger full-video deviations from a full-compute reference when applied earlier in the rollout, motivating more conservative approximation decisions for earlier chunks. The horizontal axis indicates the index of the chunk perturbed by velocity reuse. See Appendix E for diagnostic protocols.

Figure 3: Overview of TGuide. (a) ATC replaces selected full DiT forward passes by predicting final hidden states from current tokens $x_{\mathrm{emb}}$ , the current chunk’s anchor $a=h_{i_{a}}^{j}$ , and the previous chunk’s reference $p=h_{i}^{j-1}$ at the same timestep. Optional confidence-aware scheduling accepts predictions when $r\leq\tau$ and otherwise invokes the full DiT. (b) The anchor predictor estimates the current hidden state, Transport retrieves relevant historical features through cross-attention, and Correct applies a token-wise gated correction to produce $\hat{h}$ .

Figure 4: Qualitative comparisons on Self Forcing, Causal Forcing, and HY-WorldPlay.

Method	81 frames					333 frames
Method	Latency (s) $\downarrow$	Speedup $\uparrow$	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$	Latency (s) $\downarrow$	Speedup $\uparrow$	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$
Original 4 steps	6.53	1.00 $\times$	84.28	84.63	82.87	32.93	1.00 $\times$	82.55	82.93	81.04
Original 2 steps	3.48	1.88 $\times$	84.09	84.46	82.59	16.53	1.99 $\times$	82.35	82.76	80.71
TeaCache	3.83	1.71 $\times$	83.91	84.14	83.01	17.50	1.88 $\times$	81.47	81.82	80.06
TaylorSeer	3.78	1.73 $\times$	83.68	83.93	82.69	18.67	1.76 $\times$	80.69	81.05	79.26
MotionCache	3.96	1.65 $\times$	83.97	84.25	82.85	19.12	1.72 $\times$	81.56	81.89	80.24

Table 1: Performance comparison on Self Forcing using the full VBench benchmark.

Method	Quality and efficiency			vs. Original 4 steps			Self-comparison
Method	VBench $\uparrow$	Latency (s) $\downarrow$	Speedup $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Original 4 steps	84.26	49.61	1.00 $\times$	—	—	—	16.75	0.5107	0.2820
Original 2 steps	84.18	26.81	1.85 $\times$	19.72	0.7122	0.1480	16.68	0.5124	0.2963
TeaCache	84.20	30.06	1.65 $\times$	18.63	0.7230	0.1413	16.44	0.5088	0.3026
TaylorSeer	84.21	29.76	1.67 $\times$	18.60	0.6845	0.1670	16.60	0.5128	0.2983
MotionCache	84.24	30.02	1.65 $\times$	18.59	0.6843	0.1672	16.60	0.5135	0.2986

Table 2: Performance comparison on HY-WorldPlay at 125 frames.

Method	Total $\uparrow$	TFLOPs $\downarrow$	Latency (s) $\downarrow$	Speedup $\uparrow$
Original 4 steps	84.28	793	6.53	1.00 $\times$
FRRF	83.14	429	3.42	1.91 $\times$
w/o $p$	83.86	444	3.59	1.82 $\times$
Concat $p$	83.98	445	3.59	1.82 $\times$
Anchor + Transport	84.05	448	3.60	1.81 $\times$
ATC	84.08	452	3.62	1.80 $\times$

Table 3: ATC ablation under Stage 1 training.

Training	Predictor	81 frames		333 frames
Training	Predictor	Total $\uparrow$	Speedup $\uparrow$	Total $\uparrow$	Speedup $\uparrow$
Original 4 steps		84.28	1.00 $\times$	82.55	1.00 $\times$
Stage 1	DisCa	83.86	1.82 $\times$	82.12	1.90 $\times$
Stage 1	ATC	84.08	1.80 $\times$	82.29	1.89 $\times$
+ Stage 2	DisCa	84.09	1.82 $\times$	82.37	1.90 $\times$
+ Stage 2	ATC	84.23	1.80 $\times$	82.71	1.89 $\times$

Table 4: Effect of Stage 2 training.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Self Forcing: Slow				Causal Forcing
Method	VBench $\uparrow$	TFLOPs $\downarrow$	Latency (s) $\downarrow$	Speedup $\uparrow$	VBench $\uparrow$	TFLOPs $\downarrow$	Latency (s) $\downarrow$	Speedup $\uparrow$
Original 4-step	80.68	793	6.53	1.00 $\times$	81.58	793	6.48	1.00 $\times$
Original 3 / 2-step	80.42	611	4.84	1.35 $\times$	81.05	429	3.49	1.86 $\times$
TeaCache	80.47	611	4.98	1.31 $\times$	81.17	493	3.99	1.62 $\times$
TaylorSeer	80.40	611	5.06	1.29 $\times$	80.60	457	3.76	1.72 $\times$
MotionCache	80.35	626	5.04	1.30 $\times$	80.85	451	3.75	1.73 $\times$

Appendix

Table 5: Additional comparisons on 81-frame videos using the VBench subset. Ours uses dynamic K08 on Self Forcing and fixed FPPF on Causal Forcing. Naive step reduction uses three and two steps, respectively.

81 frames
Dimension	Original 4-step	Original 2-step	TeaCache	TaylorSeer	MotionCache	DisCa	Ours
Subject Consistency	96.19	96.16	96.15	95.52	95.79	95.80	96.17
Background Consistency	96.96	96.90	97.01	95.72	96.88	96.93	97.22
Temporal Flickering	98.98	99.12	99.17	99.08	99.08	99.10	99.19
Motion Smoothness	98.63	98.58	98.70	98.72	98.68	98.58	98.68
Dynamic Degree	58.33	56.94	51.39	52.78	54.17	54.17	55.56

Appendix

Table 6: Per-dimension and aggregate VBench results on 81- and 333-frame Self Forcing generation. Dimension scores are raw values multiplied by

100

; Quality, Semantic, and Total Scores are normalized weighted percentages. Higher is better for all metrics. Original denotes the original 4-step model. Bold indicates the best accelerated method in each frame setting; aggregate rankings use scores before final rounding.

Figure 5: Additional qualitative comparisons on Self Forcing. Each example shows frames at $0$ , $10$ , and $20$ seconds under matched generation conditions, with denoising speedups indicated for each method.

Method	VBench $\uparrow$	TFLOPs $\downarrow$	Latency (s) $\downarrow$	Speedup $\uparrow$
Original 4 steps	80.68	793	6.53	1.00 $\times$
FPFF	80.55	622	4.99	1.31 $\times$
FFPF	80.57	622	4.99	1.31 $\times$
Dynamic K06	80.62	613	5.10	1.28 $\times$
Dynamic K08	80.65	559	4.67	1.40 $\times$
Dynamic K10	80.58	507	4.24	1.54 $\times$

Appendix

Table 7: Dynamic schedule ablation on 81-frame Self Forcing videos, evaluated on the VBench subset.

Source layer	Val. MSE $\downarrow$	Source layer	Val. MSE $\downarrow$	Source layer	Val. MSE $\downarrow$
0	0.052675	10	0.049378	20	0.047325
1	0.051242	11	0.048483	21	0.046994
2	0.048375	12	0.049627	22	0.058777
3	0.048407	13	0.048251	23	0.048755
4	0.048115	14	0.049588	24	0.047499
5	0.048193	15	0.049718	25	0.048287

Appendix

Table 8: Validation velocity MSE across all source layers.

Initialization	Val. MSE $\downarrow$
Complete teacher block	0.046459
Teacher + zeroed branch outputs	0.055906
Xavier + zeroed branch outputs	0.057737
Xavier	0.057085
All-zero block	0.060460

Appendix

Table 9: Initialization controls with layer-17 K/V fixed, evaluated by validation velocity MSE.

Autoregressive Video Generation

Autoregressive video generation creates videos frame by frame or in chunks, conditioning each prediction on prior context for streaming and interactive use.

See all 200 papers

ForcingBooster: Adaptive Patch-wise Denoising for Fast Autoregressive Video Diffusion
Oct 8, 2026Zekai Gu, Peng Wang, Jiahao Lu +3Adaptive Computation for Diffusion Models Video Diffusion Models
HKUST · VAST · Macau University of Science and Technology
Test-Time Noise Guided Adaptation for Realistic Autoregressive Video Generation
Jul 17, 2026Dimitrios Karageorgiou, Symeon Papadopoulos, Ioannis Kompatsiaris +1Diffusion Model Guidance Video Diffusion Models
Information Technologies Institute, CERTH, Thessaloniki, Greece · University of Amsterdam, Amsterdam, The Netherlands
Speculative Forcing for Efficient Autoregressive Video Generation
Oct 15, 2026Hyunwook Choi, Seungryong Kim, Dongyoon Han +1Efficient Video Generation Autoregressive Video Generation
KAIST AI · NAVER AI Lab

Oct 8, 2026J/K move · Enter open · S save

ForcingBooster: Adaptive Patch-wise Denoising for Fast Autoregressive Video Diffusion

Zekai Gu, Peng Wang, Jiahao Lu, Yongsen Mao+2

HKUST · VAST · Macau University of Science and Technology

Adaptive Computation for Diffusion Models Video Diffusion Models Diffusion Model Inference Acceleration Autoregressive Video Generation Diffusion Models Few-Step Diffusion Sampling

Open paper AskPDF arXiv

Autoregressive video diffusion models have become a leading approach to real-time interactive video generation and video world modeling. However, existing methods typically allocate computation uniformly across space when generating the next video chunk, updating all patches with the same denoising schedule and number of function evaluations. This ignores the spatial heterogeneity of video content: static, simple, or recurring regions are often easier to denoise, while other regions require further refinement or additional context. A uniform budget thus leads to redundant computation on easy patches, whereas reducing the fixed step count can leave difficult patches insufficiently refined. We introduce \methodname{}, a general framework that dynamically allocates denoising steps to individual patches: easy patches stop early to improve efficiency, while difficult patches receive more refinement steps. This design accelerates generation with negligible quality degradation. Our method is compatible with most existing few-step autoregressive diffusion models, requiring only a lightweight routing module with modest training. Experiments demonstrate that \methodname{} reduces the total number of denoising steps applied across video patches by about 50% while maintaining visual quality comparable to the four-step baseline.

Abstract

Autoregressive Video Generation

Explore similar work

ForcingBooster: Adaptive Patch-wise Denoising for Fast Autoregressive Video Diffusion

Test-Time Noise Guided Adaptation for Realistic Autoregressive Video Generation

Speculative Forcing for Efficient Autoregressive Video Generation