Looking Ahead: Modeling Future Evolution in Autoregressive Video Generation

Ruopeng Gao; Yukun Huang; Jiarui Hu; Shuai Wang; Wenbo Hu; Ying Shan; Limin Wang

cs.CVOct 21, 2026

Looking Ahead: Modeling Future Evolution in Autoregressive Video Generation

Authors: Ruopeng Gao, Yukun Huang, Jiarui Hu, Shuai Wang, Wenbo Hu, Ying Shan, Limin Wang

Organizations: Nanjing University · ARC Lab, Tencent IEG · Zhejiang University

Abstract

Autoregressive video diffusion models generate videos chunk by chunk, yet producing plausible continuations does not guarantee that they capture how scenes evolve. Under teacher forcing, strong temporal redundancy may allow local visual continuity to serve as a shortcut for current-chunk denoising, reducing the pressure to model visual evolution. We introduce Future Chunk Prediction (FCP), an auxiliary objective jointly optimized with teacher-forced denoising that predicts temporally separated chunks without access to the intervening visual context. Drawing supervision directly from training videos, FCP requires no additional pretrained visual encoder and leaves autoregressive inference unchanged. Our method achieves superior physical plausibility over standard teacher-forced autoregressive training with a lower overall training compute budget, highlighting the effectiveness and efficiency of future-chunk supervision for modeling visual evolution. Intriguingly, our visualizations suggest that FCP fosters a more structured representation space, which may be valuable for building autoregressive world models.

Figures & tables

Figure 1: Overview of Future Chunk Prediction (FCP). Intermediate representations from the autoregressive backbone condition auxiliary transformer blocks for future-chunk prediction.

Panel of Figure 2 — Figure 2: Two variants of future-prediction supervision, illustrated through the green paths from $\mathbf{h}_{t}$ to $f_{t+3}$ . The chained design passes hidden states between predictors, whereas the parallel design conditions each predictor directly on the source representations.

Bidirectional
Model	Params	IQ $\uparrow$	SP $\uparrow$	ST $\uparrow$	WS $\uparrow$	MSE $\uparrow$
Wan2.2-5B ( Wang et al., 2025a )	5B	24.9	34.2	22.1	21.1	22.1
Sora 2 ( OpenAI, 2025 )	–	26.5	37.3	27.0	26.9	14.8
Cosmos3-Nano ( Aditi et al., 2026 )	A8B	29.1	40.4	22.0	24.6	29.5
Wan2.2-14B ( Wang et al., 2025a )	A14B	32.2	51.1	20.5	28.5	28.9
HunyuanVideo-1.5 ( Team, 2025 )	8.3B	33.4	47.1	26.9	29.7	30.0

Table 1: Physics-IQ Verified results for image-to-video generation ( Rädsch et al., 2026 ) . Baseline scores are primarily from the benchmark. IQ is the overall score. Params counts visual-denoiser parameters. Bold and underlined mark the best and second-best autoregressive scores.

ID	Method	I2V				V2V
ID	Method	IQ $\uparrow$	SP $\uparrow$	ST $\uparrow$	WS $\uparrow$	IQ $\uparrow$	SP $\uparrow$	ST $\uparrow$	WS $\uparrow$
(a)	Teacher Forcing	30.4	34.8	32.8	24.0	40.6	48.5	40.8	37.8
(b)	+ FCP (3 offsets)	32.1	33.6	39.3	23.2	43.0	50.3	44.6	39.8
(c)	+ FCP (3 offsets, chained)	30.7	34.6	33.0	23.3	40.5	48.5	39.3	37.9
(d)	+ FCP (5 offsets)	33.8	36.8	41.1	25.2	44.4	51.1	47.2	40.7
(e)	+ FCP (5 offsets, chained)	31.0	35.6	33.0	23.9	39.9	47.6	38.9	36.6

Table 2: Controlled Physics-IQ Verified ( Rädsch et al., 2026 ) comparisons on Wan2.2-TI2V-5B. The shaded row and columns mark our default configuration and overall IQ scores, respectively.

Panel of Figure 3 — Figure 3: Qualitative comparison of teacher forcing (top) and FCP (bottom). Left: FCP predicts domino toppling along the expected direction (green arrows), whereas teacher forcing produces an anomalous collapse at the center (red box). Right: FCP captures the rise in water level as the ice melts (green $\Delta$ ), a detail missed by teacher forcing.

Benchmark	Metric	Teacher Forcing	+ FCP (Ours)
WorldModelBench ( Li et al., 2025a )	Instruction $\uparrow$	1.77	1.81 (+0.04)
	Physical Laws $\uparrow$	4.28	4.47 (+0.19)
	Common Sense $\uparrow$	1.17	1.36 (+0.19)
	Total $\uparrow$	7.22	7.64 (+0.42)
PAI-Bench-G ( Zhou et al., 2025 )	AV $\uparrow$	63.54	64.81 (+1.27)
	Common Sense $\uparrow$	90.90	92.10 (+1.20)

Table 3: Results on WorldModelBench, PAI-Bench-G, and PhyGround. All variants are based on Wan2.2-TI2V-5B and use matched training settings.

Panel of Figure 5 — Figure 5: Spatial Feature Similarity Maps. Each map shows within-frame similarity to a selected query token. The top and bottom rows correspond to teacher forcing and FCP, respectively.

Panel of Figure 6 — Figure 6: Spatiotemporal Feature Similarity Maps. The top row shows video frames, while the bottom row visualizes similarity to a selected query token across space and time.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	Wan2.1 T2V-1.3B	Wan2.2 TI2V-5B
Backbone depth	30	30
Hidden dimension $d$	1536	3072
FFN intermediate dimension	8960	14336
Source feature layers	9, 15, 21, 27	3, 12, 21, 30
Auxiliary DiT depth	3	3
Auxiliary initialization layers	17, 18, 19	28, 29, 30

Appendix

Table 5: Model configurations. All layer indices are one-based. Hidden and FFN dimensions apply to both the backbone and its auxiliary DiT blocks.

clean	noisy	text	chained	IQ $\uparrow$	SP $\uparrow$	ST $\uparrow$	WS $\uparrow$	MSE $\uparrow$
source	source	condition	connection	IQ $\uparrow$	SP $\uparrow$	ST $\uparrow$	WS $\uparrow$	MSE $\uparrow$
				27.6	32.8	25.5	20.4	31.6
$\checkmark$				28.8	33.0	29.5	21.0	31.9
	$\checkmark$			29.9	34.7	28.2	21.6	35.0
$\checkmark$	$\checkmark$			30.9	36.2	29.6	23.2	34.5
$\checkmark$	$\checkmark$	$\checkmark$		29.8	34.8	28.4	22.0	34.2

Appendix

Table 6: Ablation study of the main components on Physics-IQ Verified ( Rädsch et al., 2026 ) . All variants are implemented based on Wan2.1 1.3B. The shaded row denotes our default configuration.

Fusion strategy	IQ $\uparrow$	SP $\uparrow$	ST $\uparrow$	WS $\uparrow$	MSE $\uparrow$
Token Concat	28.8	34.1	28.1	21.0	32.0
Dim Concat	29.5	32.2	31.2	20.6	34.1
AdaLN	30.9	36.2	29.6	23.2	34.5

Appendix

Table 7: Ablation study of feature-fusion strategies for injecting features into the future-prediction branch on Physics-IQ Verified ( Rädsch et al., 2026 ) . All variants are implemented based on Wan2.1 1.3B. The shaded row denotes our default configuration.

Panel of Figure 8 — Figure 8: Additional Spatial Feature Similarity Maps (continued). Two further examples using the same query-to-token similarity visualization.

Panel of Figure 9 — Figure 9: Additional Spatiotemporal Feature Similarity Maps. In each example, the top row shows video frames, while the bottom row visualizes similarity to a selected query token across space and time.

Panel of Figure 10 — Figure 10: Detailed Spatiotemporal Feature Similarity Maps. In each example, the top row shows video frames, while the middle and bottom rows show similarity to a selected query token under teacher forcing and FCP, respectively.

Panel of Figure 11 — Figure 11: Auxiliary Future Predictions. In each example, the top row shows the ground-truth video, while the middle and bottom rows show outputs from auxiliary branches at the future offsets indicated on the left.

Autoregressive Video Generation

Autoregressive video generation creates videos frame by frame or in chunks, conditioning each prediction on prior context for streaming and interactive use.

See all 211 papers

Video-Mirai: Autoregressive Video Diffusion Models Need Foresight
Jun 2, 2026Yonghao Yu, Lang Huang, Runyi Li +2Temporal Consistency in Video Generation Autoregressive Diffusion
The University of Tokyo · National Institute of Informatics · Peking University
Prospective Forcing: Learning to Look Ahead for Autoregressive Long Video Diffusion
Oct 10, 2026Yifan Shen, Yewei Xia, Gongxu Luo +6Autoregressive Video Generation Video Diffusion Models
Mohamed bin Zayed University of AI · Institute of Foundation Models · Carnegie Mellon University
Next Forcing: Causal World Modeling with Multi-Chunk Prediction
Jun 9, 2026Gangwei Xu, Qihang Zhang, Jiaming Zhou +4Causal World Models Autoregressive Video Generation
Robbyant · HUST · HKUST (GZ) +1

Jun 2, 2026J/K move · Enter open · S save

Video-Mirai: Autoregressive Video Diffusion Models Need Foresight

Yonghao Yu, Lang Huang, Runyi Li, Zerun Wang+1

The University of Tokyo · National Institute of Informatics · Peking University

Temporal Consistency in Video Generation Autoregressive Diffusion Long Video Generation Streaming Video Generation Predictive Representation Learning Autoregressive Video Generation+2

Open paper AskPDF arXiv

Causal video generators must predict from the past, but they need not learn only from it. In streaming autoregressive video diffusion, each emitted segment becomes a commitment that future segments must preserve. Standard training, however, only asks each causal state to explain the present. This creates what we call a representation-level planning gap: states that fit the current segment may discard identity, layout, and motion information needed for a consistent future. We introduce Video-Mirai, a training-only method that closes this gap without changing causal inference: the generator rolls out causally, a frozen foresight encoder reads the completed rollout non-causally, and a lightweight predictor distills the resulting stopped-gradient targets into causal states. Future frames supervise representations, never generator inputs. At inference, the encoder and predictor are discarded, leaving the original architecture, per-step FLOPs, and KV-cache behavior unchanged. Video-Mirai improves a strong Causal-Forcing baseline on 5-second VBench from 83.8 to 84.6 in terms of Total Score. On 30-second rollouts beyond the training horizon, subject consistency improves from 84.9 to 88.5 and background consistency from 90.2 to 91.9. Ablations identify future-conditioned targets as the key ingredient, and probes show that future frames become more decodable from current features. Causality should constrain inference, not representation supervision. Our study highlights that visual autoregressive models need foresight. Project page: https://y0uroy.github.io/Video-Mirai.

Abstract

Autoregressive Video Generation

Explore similar work

Video-Mirai: Autoregressive Video Diffusion Models Need Foresight

Prospective Forcing: Learning to Look Ahead for Autoregressive Long Video Diffusion

Next Forcing: Causal World Modeling with Multi-Chunk Prediction