ForgeWM: Progressive Causal Training for Few-Step Action-Conditioned Video World Models

Xinye Li; Lingshuai Lin; Lei Wang; Liuzhou Zhang; Jialin Cui; Qingshan Li; Guanchu Wang; Qingbin Liu

cs.CVAug 14, 2026

ForgeWM: Progressive Causal Training for Few-Step Action-Conditioned Video World Models

Authors: Xinye Li, Lingshuai Lin, Lei Wang, Liuzhou Zhang, Jialin Cui, Qingshan Li, Guanchu Wang, Qingbin Liu,

Organizations: CUHK · Tencent PCG · FDU · Shanghai AI Laboratory ·

Abstract

Action-conditioned video world models require low-latency causal generation and reliable responses to game-native controls. Although causal distillation enables one- or few-step video synthesis, extending it to interactive world models remains challenging, as discrete keyboard states and continuous mouse motion must remain aligned with temporally compressed latent chunks during causal training and autoregressive rollout. We introduce ForgeWM, a progressive framework that transforms a bidirectional action-conditioned video generator into efficient few-step world models through domain adaptation, teacher-forced causal training, causal consistency distillation, and on-policy distribution matching with a bidirectional teacher. The resulting budget-specialized students operate at steady-state denoising budgets of 1, 2, and 4 steps. ForgeWM further supports a dual-path deployment protocol combining latency-critical interaction with optional replay-time refinement, where the one-step student re-noises and refines its saved draft. On paired Minecraft trajectories, ForgeWM leads the evaluated systems in Imaging Quality, reference-aligned motion-profile agreement, action-sign accuracy, and mouse-control accuracy, while achieving the lowest reference LPIPS; the same four-stage recipe transfers to gamepad-controlled FPS gameplay. Replay-time refinement matches four-step reference quality while remaining roughly three times closer to the experienced trajectory than regeneration from noise. These results demonstrate ForgeWM's effectiveness for controllable few-step video generation.

Figures & tables

Figure 2: Overview of ForgeWM. (A) Frame-aligned keyboard and mouse controls condition latent chunks. (B) A shared base yields a bidirectional teacher and budget-specialized causal students through four-stage training. (C) Deployment separates low-latency interaction from optional Replay-Time Refinement (Figure 6 ).

Stage	Initialization	Training Context	Objective	Output
0	Base	Full-clip bidirectional	FM	Domain teacher
1	Base	Clean causal history	Causal FM	Causal teacher
2	Stage 1	Clean causal history	Online CD	Few-step initializer
3	Stage 2	Self-generated history	DMD	1/2/4-step students

Table 1: Training stages. Each stage changes either the temporal execution pattern, sampling objective, or history distribution.

Figure 3: Qualitative comparison. Rollouts at frames 0, 25, 51, and 76. Rows show the reference and three models. Controls are annotated once on the reference: each overlay shows dominant WASD and accumulated mouse-look to the next frame; the final frame has no outgoing control overlay. Left/right: daytime forest stream/rainy riverbank at night.

Model	Visual Quality			Temporal Quality		Action Controllability		Efficiency
Model	IQ $\uparrow$	LPIPS $\downarrow$	AQ $\uparrow$	Subj. Cons. $\uparrow$	Flow Prof. $\uparrow$	KCtrl $\uparrow$	Mouse Acc. $\uparrow$	Latency (ms) $\downarrow$	FPS $\uparrow$
Matrix-Game 2.0	0.6282	0.6443	0.4583	0.7349	0.9343	0.9156	0.7061	370.9	32.35
HY-WorldPlay	0.6133	0.6172	0.4855	0.9466	0.8288	0.9286	0.5818	2164.3	7.54
ForgeWM-1 (1-step)	0.6776	0.6529	0.4807	0.8279	0.9403	0.9545	0.7848	168.2	72.10
ForgeWM-2 (2-step)	0.6865	0.6171	0.4814	0.8349	0.9429	0.9740	0.8268	239.7	50.31
ForgeWM-4 (4-step)	0.6788	0.6168	0.4860	0.7613	0.9420	0.9740	0.8102	369.6	32.47

Table 2: Comparison with interactive world models. Bold marks the best quality/control value in each column; efficiency entries are not bolded.

Figure 4: Human preferences.

Figure 5: Test-time step scaling. ForgeWM-1 under different denoising budgets; shading denotes bootstrap 95% confidence intervals.

Figure 6: Replay-Time Refinement. The deployed student re-noises and denoises its own saved chunks under aligned actions and the refined causal prefix; each refined chunk is committed before the next is processed.

Figure 8: CrossFPS rollouts. ForgeWM-CrossFPS on (A) Halo Infinite, (B) Modern Warfare, and (C) a science-fiction shooter. Colored overlays indicate active gamepad controls.

Figure 9: Extended qualitative rollouts from the four-step Minecraft checkpoint. Each row shows one $22$ s causal rollout sampled at evenly spaced timestamps, with elapsed time marked per frame. Rows span biome and time of day and include both sustained forward motion and sustained turning.

	Stage 0	Stage 1	Stage 2	Stage 3
Objective	$\mathcal{L}_{\mathrm{FM}}$	$\mathcal{L}_{1}$	$\mathcal{L}_{2}$	$\mathcal{L}_{3}$
Trainer	FM	causal FM	consist. distill.	DMD
Attention	bidir.	causal	causal	causal
Init	base	base	Stage 1	Stage 2
Blk. (latents)	21	3	3	3
Trainer iterations	4k	20k	6k	4k

Table F1: Per-stage optimization for the reported Minecraft ForgeWM lineage. “Blk.” is the number of latent frames per attention block (

21

= full-clip bidirectional;

3

= one causal chunk). “EMA” is the generator exponential-moving-average decay and the step it starts (“–” when disabled). Global batch is the effective batch after data-parallel accumulation. All learning rates are constant.

Figure F1: Training losses across stages. Raw values are shown as light traces, with exponential-moving-average trends overlaid; the horizontal axis denotes trainer iterations. In Stage 3, the critic is updated every iteration and the generator every fifth iteration. Top-left : Stage 0 bidirectional flow matching. Top-right : Stage 1 teacher-forced causal flow matching. Bottom-left : Stage 2 online causal consistency distillation. Bottom-right : Stage 3 distribution matching, with the generator (DMD) loss on the left axis and the critic (fake-denoiser) loss on the right axis. Because the stages optimize different objectives, absolute loss magnitudes are not comparable across panels; the curves document optimization behavior within each stage only.

Stage	Inference regime	LPIPS $\downarrow$	IQ $\uparrow$	AQ $\uparrow$	SC $\uparrow$
0	bidirectional teacher (ref.)	$0.814_{[.809,.819]}$	$0.455$	$0.463$	$0.677$
1	teacher-forced causal	$0.806_{[.799,.812]}$	$0.508$	$0.454$	$0.700$
2	causal consistency	$\mathbf{0.605}_{[.600,.610]}$	$0.659$	$0.483$	$\mathbf{0.760}$
3	distribution matching	$0.617_{[.613,.620]}$	$\mathbf{0.716}$	$\mathbf{0.489}$	$\mathbf{0.760}$

Table F2: Stage-wise inference ablation on

1{,}000

paired trajectories. LPIPS uses the AlexNet backbone on

16

evaluated frames; subscripts denote trajectory-level bootstrap 95% confidence intervals. IQ, AQ, and SC are the corresponding VBench metrics. Stage 0 is a non-causal domain-teacher reference evaluated in its native full-clip regime.

Game	LPIPS $\downarrow$	PSNR $\uparrow$	Flow	Ratio
Xonotic	0.5828	11.21	13.88	1.66
Modern Warfare III	0.6352	11.61	8.15	1.35
Modern Warfare	0.6479	10.99	8.29	1.46
Warzone	0.6695	10.66	9.13	1.78
Halo Infinite	0.6730	9.75	11.23	1.46
Halo	0.6920	9.31	13.28	1.29

Table H1: Per-game CrossFPS results, 25 clips per game. Flow is the mean dense optical-flow magnitude in pixels; ratio is generated over reference, where

1.0

matches the reference motion scale.

Single-Step Video Generation

See all 11 papers

minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models
May 28, 2026Min Zhao, Hongzhou Zhu, Bokai Yan +9Interactive World Models Video Diffusion Models
1ShengShu 2THU · 1ShengShu 3RUC · 1ShengShu +4
Next Forcing: Causal World Modeling with Multi-Chunk Prediction
Jun 9, 2026Gangwei Xu, Qihang Zhang, Jiaming Zhou +4Causal World Models Autoregressive Video Generation
Robbyant · HUST · HKUST (GZ) +1
BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression
Jun 8, 2026Shaohao Rui, Xiaofeng Mao, Zhanyu Zhang +5Action-Conditioned Video Generation Camera-Controlled Video Generation
LynnReal AI · Shanghai Innovation Institute · Shanghai Jiao Tong University +1

May 28, 2026J/K move · Enter open · S save

minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models

Min Zhao, Hongzhou Zhu, Bokai Yan, Zihan Zhou+8

1ShengShu 2THU · 1ShengShu 3RUC · 1ShengShu+4

Interactive World Models Video Diffusion Models Causal World Models Real-Time Interactive Video Generation Streaming Video Generation Autoregressive Video Generation+2

Open paper AskPDF arXiv

Recent video diffusion foundation models have achieved remarkable progress in high-quality video generation, yet turning them into real-time interactive video world models remains challenging. Interactive world models require controllable, causal, and low-latency rollout, which in practice demands a full pipeline spanning data construction, controllable fine-tuning, autoregressive training, few-step distillation, and streaming inference. In this work, we present minWM, a full-stack open-source framework for building real-time interactive video world models. minWM provides an end-to-end pipeline that converts existing bidirectional T2V/TI2V video foundation models into camera-controllable few-step autoregressive world models. Specifically, minWM first fine-tunes a bidirectional video diffusion model with camera control, and then applies the Causal Forcing / Causal Forcing++ pipeline, including AR diffusion training, causal ODE or causal consistency distillation, and asymmetric DMD, to distill it into a few-step autoregressive generator for low-latency rollout. The framework is modular and architecture-extensible: we instantiate it on representative open backbones, including Wan2.1-T2V-1.3B and HY1.5-TI2V-8B, covering both cross-attention-based condition injection and MMDiT-style architectures. minWM also supports adapting existing video world models, such as HY-WorldPlay, to new data distributions, training recipes, and latency targets. Beyond releasing runnable scripts, checkpoints, documentation, and inference code, we provide practical ablations on camera trajectory quality, controllability training steps, and minimal batch-size requirements. We hope minWM serves as a reproducible and extensible recipe for building and adapting real-time interactive video world models. Project Page: https://github.com/shengshu-ai/minWM

Abstract

Single-Step Video Generation

Explore similar work

minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models

Next Forcing: Causal World Modeling with Multi-Chunk Prediction

BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression