Distilling Dense Trajectory Teachers For Large-Motion Video World Models

Shuofeng Sun; Linqing Zhao; Haibin Yan

cs.CVOct 24, 2026

Distilling Dense Trajectory Teachers For Large-Motion Video World Models

Authors: Shuofeng Sun, Linqing Zhao, Haibin Yan

Organizations: Beijing University of Posts and Telecommunications

Abstract

Video world models are expected to generate videos that accurately follow specified camera trajectories while maintaining consistent 3D structures. However, existing models are predominantly trained on densely sampled videos with small inter-frame camera motions. When applied to sparse, large step trajectories, they often produce frozen frames, inaccurate camera motion, and severe geometric drift. We attribute this degradation to a mismatch between the gradual viewpoint transitions seen during training and the large viewpoint changes required at inference time. To address this problem, we propose Dense Trajectory Teacher Distillation (DTTD), a post-training framework that transfers a model's generation capability from dense trajectories to large camera motions. Given a sparse trajectory, DTTD interpolates additional camera poses to construct a dense trajectory covering the same path. The frozen base model then generates reliable teacher videos along this familiar dense trajectory, and pose aligned teacher observations are used to evaluate candidate videos generated under the original sparse trajectory. In this way, the model serves as its own generation teacher, requiring neither additional video data nor a stronger external video generator. We further combine dense teacher supervision with geometry aware rewards that assess cross frame consistency and camera trajectory following, and use the resulting preferences for offline preference optimization. To stably expand the model's motion range, we introduce a cumulative curriculum that progressively incorporates larger motion steps while retaining simpler trajectories to reduce capability forgetting.

Figures & tables

Figure 1: Generation video examples. When the camera trajectory is sparse and the motion magnitude is large, the model may produce static frames, geometric collapse, camera-following failure, and other issues. Furthermore, we can see that by densifying the trajectory and reducing the motion magnitude, the model’s output quality can be substantially improved, which demonstrates the rationality of our dense trajectory evaluation. After alignment with our method, the model’s geometric consistency, following capability, and visual quality are effectively improved when faced with large-motion trajectories.

Figure 2: Pipeline of DTTD. We construct camera trajectories with different motion magnitudes through interpolation, and design dense trajectory evaluation and geometric evaluation to improve the model’s performance when faced with large-motion trajectories.

Motion	Method	Subject Cons. $\uparrow$	Background Cons. $\uparrow$	Motion Smooth. $\uparrow$	Dynamic Degree $\uparrow$	Aesthetic Quality $\uparrow$	Imaging Quality $\uparrow$
s=0.125	Base	0.870108	0.928226	0.989094	0.934535	0.339647	0.439500
	Epipolar-DPO	0.885960	0.935076	0.991633	0.903226	0.345566	0.378853
	VideoGPA	0.889117	0.936558	0.989903	0.920304	0.349204	0.455657
	DTTD (Ours)	0.903036	0.942309	0.991436	0.924099	0.354848	0.449773
s=0.5	Base	0.815244	0.907723	0.975771	0.988615	0.330851	0.388325
	Epipolar-DPO	0.844630	0.919677	0.980837	0.972486	0.336900	0.353353

Table 1: Visual-quality comparison on the ScanNet test set. s = 0.125 denotes the densest reference trajectory. Relative to this trajectory, s = 0.5 and s = 1 correspond to 4x and 8x larger sampling intervals, respectively. The best and second-best results are highlighted in bold and underline .

Motion	Method	CamMC $\downarrow$	CamMC Rot. $\downarrow$	CamMC Trans. $\downarrow$	ATE. Trans. $\downarrow$	ATE. Rot. $\downarrow$
$s=0.125$	Base	0.518166	0.094355	0.488861	0.158650	1.730458
	Epipolar-DPO Kupyn et al. (2025)	0.499797	0.085560	0.474069	0.146549	1.667481
	VideoGPA Du et al. (2026)	0.492305	0.085704	0.466349	0.149542	1.651467
	DTTD (Ours)	0.488283	0.076850	0.452347	0.158356	1.609799
$s=0.5$	Base	0.660112	0.152059	0.601717	0.181642	2.029051
	Epipolar-DPO Kupyn et al. (2025)	0.658929	0.140667	0.609915	0.172629	2.020727

Table 2: Camera-following performance on the ScanNet test set. The best and second-best results in each setting are highlighted in bold and underline .

Figure 4: Qualitative comparison of Different Methods.

Method	s=0.125			s=0.5			s=1
Method	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$
Base	0.7003	0.5182	0.4395	0.6962	0.6601	0.3883	0.6913	0.7129	0.3824
+ Dense-Trajectory Evaluation	0.6254	0.5018	0.4352	0.6368	0.6372	0.3995	0.6342	0.6861	0.3936
+ Geometry Evaluation	0.5967	0.4919	0.4376	0.5979	0.6216	0.4078	0.5948	0.6675	0.4018
+ Cumulative Curriculum	0.5868	0.4883	0.4398	0.5866	0.6154	0.4137	0.5825	0.6597	0.4071

Table 4: Cumulative ablation study under different trajectory sampling intervals. Each row adds one component to the preceding configuration. IQ means the Imaging Quality.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

	$s=0.125$			$s=0.5$			$s=1$
Method	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$
Base	0.7003	0.5182	0.4395	0.6962	0.6601	0.3883	0.6913	0.7129	0.3824
+ Feature only	0.6578	0.5074	0.4371	0.6549	0.6468	0.3972	0.6521	0.6964	0.3915
+ Depth only	0.6397	0.5031	0.4368	0.6425	0.6349	0.4008	0.6381	0.6817	0.3952
+ Depth + Feature	0.6254	0.5018	0.4352	0.6368	0.6372	0.3995	0.6342	0.6861	0.3936

Appendix

Table 5: Ablation of the dense-trajectory teacher reward. The depth-based term uses scale-aligned depth consistency, while the feature-based term measures the cosine distance between VAE latent features. Lower is better for 3DCS and CamMC, while higher is better for IQ.

	$s=0.125$			$s=0.5$			$s=1$
Method	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$
Dense teacher only	0.6254	0.5018	0.4352	0.6368	0.6372	0.3995	0.6342	0.6861	0.3936
+ Reprojection reward	0.6048	0.5009	0.4363	0.6087	0.6318	0.4054	0.6041	0.6769	0.3999
+ Relative trajectory reward	0.6172	0.4897	0.4360	0.6205	0.6168	0.4059	0.6176	0.6614	0.4007
+ Both geometry rewards	0.5967	0.4919	0.4376	0.5979	0.6216	0.4078	0.5948	0.6675	0.4018

Appendix

Table 6: Ablation of the geometry-aware rewards. Reprojection evaluates cross-frame geometric consistency, while relative trajectory evaluates camera-motion following after similarity-transform alignment. Lower is better for 3DCS and CamMC, while higher is better for IQ.

	$s=0.125$			$s=0.5$			$s=1$
Strategy	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$	3DCS $\downarrow$	CamMC $\downarrow$	IQ $\uparrow$
Direct large-step training	0.6201	0.5104	0.4301	0.6084	0.6287	0.4052	0.5907	0.6621	0.4001
Non-cumulative curriculum	0.6107	0.5028	0.4335	0.6031	0.6314	0.4059	0.5914	0.6643	0.4028
Mixed training	0.6008	0.4976	0.4312	0.5924	0.6263	0.4098	0.5869	0.6658	0.4046
Cumulative curriculum (Ours)	0.5868	0.4883	0.4398	0.5866	0.6154	0.4137	0.5825	0.6597	0.4071

Appendix

Table 7: Ablation of different motion curriculum strategies. All variants use the same dense-teacher and geometry-aware rewards. Lower is better for 3DCS and CamMC, while higher is better for IQ.

Video World Models

Video world models generate and predict action-conditioned visual futures to simulate changing scenes for embodied agents.

See all 256 papers

WorldForge V2: Eliciting Camera-Trajectory Alignment in Video Diffusion Models
Oct 22, 2026Chenxi Song, Yanming Yang, Tong Zhao +3Camera-Controlled Video Generation Trajectory-Guided Video Generation
Westlake University, Hangzhou, China · Beihang University, Beijing, China · Singapore Management University, Singapore
TrajLoom: Dense Future Trajectory Generation from Video
Mar 23, 2026Zewei Zhang, Jia Jun Cheng Xian, Kaiwen Liu +4Trajectory Prediction Video Prediction
McMaster University · University of British Columbia · Vector Institute +2
World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models
Oct 12, 2026Junyi Wu, Zekun Li, Chenhongyi Yang +9Video World Models Memory-Augmented Video Generation
Meta Reality Labs · University of Illinois Chicago · Brown University

Oct 22, 2026J/K move · Enter open · S save

WorldForge V2: Eliciting Camera-Trajectory Alignment in Video Diffusion Models

Chenxi Song, Yanming Yang, Tong Zhao, Ruibo Li+2

Westlake University, Hangzhou, China · Beihang University, Beijing, China · Singapore Management University, Singapore

Camera-Controlled Video Generation Trajectory-Guided Video Generation Video Diffusion Models Diffusion Model Guidance Diffusion Model Adaptation

Open paper AskPDF arXiv

Video diffusion models have learned rich 3D/4D priors, making them promising for scene generation and video re-rendering. However, their use in spatial tasks is limited by imprecise camera control, spatial and temporal inconsistencies, and entangled camera motion with scene dynamics. Existing approaches, such as task-specific fine-tuning and post-process warping, often introduce visual artifacts, generalize poorly, or incur high computational costs. We propose WorldForge V2 to achieve accurate camera control while preserving the generative capabilities of pretrained video models. We first develop a training-free guidance framework using warped references to iteratively refine generation along a target camera path. Optical-flow analysis identifies motion-related latent channels for selective guidance, helping separate motion control from appearance. A dual-path strategy further compares guided and unguided predictions to correct artifacts from imperfect references. Together, these components guide viewpoint changes while preserving scene appearance and object motion. To reduce refinement costs, v-cache approximates the correction using the preceding denoising prediction, avoiding an additional model evaluation. We further extend motion localization from latent channels to network blocks and introduce Trajectory LoRA, which concentrates adaptation on blocks most responsive to trajectory guidance. A two-stage training strategy teaches these adapters to denoise the states encountered during guided generation, using limited supervision effectively. This enables reliable camera control with very little training data, with a single training scene already supporting generalization to unseen scenes. Quantitative and qualitative evaluations across multiple datasets show that our method achieves state-of-the-art generation quality and trajectory accuracy in camera-controlled video generation.

Abstract

Video World Models

Explore similar work

WorldForge V2: Eliciting Camera-Trajectory Alignment in Video Diffusion Models

TrajLoom: Dense Future Trajectory Generation from Video

World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models