WorldForge V2: Eliciting Camera-Trajectory Alignment in Video Diffusion Models

Chenxi Song; Yanming Yang; Tong Zhao; Ruibo Li; Joey Tianyi Zhou; Chi Zhang

cs.CVOct 22, 2026

WorldForge V2: Eliciting Camera-Trajectory Alignment in Video Diffusion Models

Authors: Chenxi Song, Yanming Yang, Tong Zhao, Ruibo Li, Joey Tianyi Zhou, Chi Zhang

Organizations: Westlake University, Hangzhou, China · Beihang University, Beijing, China · Singapore Management University, Singapore

Abstract

Video diffusion models have learned rich 3D/4D priors, making them promising for scene generation and video re-rendering. However, their use in spatial tasks is limited by imprecise camera control, spatial and temporal inconsistencies, and entangled camera motion with scene dynamics. Existing approaches, such as task-specific fine-tuning and post-process warping, often introduce visual artifacts, generalize poorly, or incur high computational costs. We propose WorldForge V2 to achieve accurate camera control while preserving the generative capabilities of pretrained video models. We first develop a training-free guidance framework using warped references to iteratively refine generation along a target camera path. Optical-flow analysis identifies motion-related latent channels for selective guidance, helping separate motion control from appearance. A dual-path strategy further compares guided and unguided predictions to correct artifacts from imperfect references. Together, these components guide viewpoint changes while preserving scene appearance and object motion. To reduce refinement costs, v-cache approximates the correction using the preceding denoising prediction, avoiding an additional model evaluation. We further extend motion localization from latent channels to network blocks and introduce Trajectory LoRA, which concentrates adaptation on blocks most responsive to trajectory guidance. A two-stage training strategy teaches these adapters to denoise the states encountered during guided generation, using limited supervision effectively. This enables reliable camera control with very little training data, with a single training scene already supporting generalization to unseen scenes. Quantitative and qualitative evaluations across multiple datasets show that our method achieves state-of-the-art generation quality and trajectory accuracy in camera-controlled video generation.

Figures & tables

Fig. 1: Comparison with existing methods. With limited training data and iterations, our method achieves state-of-the-art visual quality and trajectory accuracy in camera-controlled video generation.

Fig. 2: Overview of WorldForge V2. Given a single image or video frame and a target camera trajectory, the scene geometry is reconstructed and rendered into trajectory-aligned warped frames. WorldForge V2 first performs training-free trajectory guidance in the predicted clean latent space using IRR, FLF, and DSG. Building on this framework, we further introduce an efficient route that directly fuses trajectory information in the noisy latent space with V-Cache, together with a trajectory-sensitive lightweight LoRA scheme for camera trajectory control using only limited training data. The resulting video follows the specified camera path while preserving the generative capability of the pretrained video diffusion model.

Fig. 3: Qualitative comparison for 3D novel-view synthesis and 4D trajectory-controlled re-rendering. For single-image 3D generation (left), our method produces more coherent geometry and plausible newly revealed content. For dynamic re-rendering (right), it preserves human appearance and object motion while reducing floating structures and flattened faces. The training-free guidance yields more realistic results in both static and dynamic scenes.

Method	FID $\downarrow$	CLIP $\uparrow$	ATE $\downarrow$	RPE-T $\downarrow$	RPE-R $\downarrow$
See3D	123.26	0.941	0.091	0.089	0.250
ViewCrafter	117.50	0.930	0.236	0.315	0.728
ViewExtrapolator	125.50	0.930	0.183	0.260	0.882
TrajectoryAttention	122.37	0.920	0.159	0.238	0.532
TrajectoryCrafter	111.49	0.910	0.090	0.152	0.267
NVS-Solver	118.64	0.937	0.224	0.268	1.056

TABLE I: Quantitative comparison on 3D static scenes using official implementations and identical source inputs. FID and CLIP assess generation quality; ATE, RPE-T, and RPE-R assess trajectory accuracy. Our training-free method leads on all metrics.

\uparrow

: higher is better;

\downarrow

: lower is better. Bold and underlining mark the best and second-best results.

Method	FVD $\downarrow$	CLIP-V $\uparrow$	ATE $\downarrow$	RPE-T $\downarrow$	RPE-R $\downarrow$
ViewExtrapolator	108.48	0.913	1.040	1.208	4.750
TrajectoryAttention	106.94	0.911	0.605	1.238	3.560
TrajectoryCrafter	97.31	0.923	0.431	1.078	8.950
Ours (Training-Free)	93.17	0.938	0.527	0.826	2.690

TABLE II: Quantitative comparison on 4D dynamic re-rendering using official implementations with matched source videos and target trajectories. FVD and CLIP-V assess video quality; ATE, RPE-T, and RPE-R assess trajectory accuracy. Our method combines the best video quality with the lowest local camera errors.

\uparrow

: higher is better;

\downarrow

: lower is better. Bold and underlining mark the best and second-best results.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Vis-LPIPS $\downarrow$	Disocc-LPIPS $\downarrow$	ATE $\downarrow$	RPE-T $\downarrow$	RPE-R $\downarrow$
UniWorld-View	17.15	0.5958	0.3370	0.2799	0.4431	0.00625	0.2333	0.1218
TrajectoryCrafter	17.41	0.5986	0.3558	0.3066	0.4512	0.00710	0.2013	0.0990
SymphoMotion	15.38	0.4760	0.4433	0.4215	0.4908	0.02292	0.4065	0.2196
Warp-as-History	15.48	0.4553	0.4206	0.3829	0.4939	0.03533	0.4856	0.2398
Ours (LongCat)	19.72	0.6663	0.2748	0.2231	0.3640	0.00905	0.2725	0.1324
Ours (H3)	20.21	0.6806	0.2698	0.2244	0.3459	0.00685	0.1647	0.0780

TABLE III: Quantitative comparison of Trajectory LoRA with camera-control baselines. PSNR, SSIM, and LPIPS assess target-view reconstruction; Vis-LPIPS and Disocc-LPIPS assess visible and newly exposed regions. Both adapted backbones improve reconstruction across these metrics, preserving observed content and completing new regions more faithfully. H3 also achieves the lowest local camera errors. Bold and underlining mark the best and second-best results.

Fig. 4: Noise-state consistency under different reference-insertion strategies. Direct Insertion leaves gaps in the inferred-noise mean and log-variance between visible and model-generated regions, accompanied by noisy patches and broken structure. IRR reduces these gaps, and V-Cache approximates its correction while retaining continuous surfaces and clear geometric detail.

Fig. 5: Qualitative comparison of camera-controlled scene generation. Our LongCat and H3 models recover coherent object shapes and material detail from incomplete warped references, including thin structures and partially revealed objects. Compared with the baselines, they produce fewer rendering remnants and more continuous geometry, preserving scene appearance as the camera reveals new content.

Fig. 6: Qualitative transfer of Trajectory LoRA across camera-control methods. Without additional fine-tuning, the same trained LongCat adapter improves surface continuity and object detail in WAH [ 18 ] and GWF [ 16 ] , demonstrating transferable trajectory control.

Method	FID $\downarrow$	CLIP $\uparrow$	FVD $\downarrow$	CLIP-V $\uparrow$
w/o DSG	109.43	0.943	95.69	0.937
w/o FLF	112.69	0.945	99.79	0.932
w/o DSG and FLF	113.12	0.943	103.17	0.931
DSG (CFG form)	120.91	0.936	109.10	0.919
Ours (Training-Free)	96.08	0.948	93.17	0.938

TABLE IV: Quantitative ablation of inference-time guidance. IRR supplies the trajectory signal; removing its reference injection leaves the model without camera guidance. With IRR fixed, we remove FLF, DSG, or both, or replace DSG with its CFG form. Both components improve generation quality, and their combination performs best on static and dynamic scenes. Bold and underlining mark the best and second-best results.

Fig. 7: Shapley analysis of block sensitivity to trajectory supervision. After approximately 400 steps of probe LoRA training, contributions are estimated from 25 trajectory pairs per backbone. Sensitive blocks are dispersed through the middle and later layers of LongCat and show a more regular concentration in the later layers of H3. This variation identifies the blocks to prioritize for adaptation.

Selection	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	ATE $\downarrow$	RPE-T $\downarrow$	RPE-R $\downarrow$
All blocks (48)	18.98	0.6140	0.2847	0.01045	0.2904	0.1693
Selected (36)	19.03	0.6176	0.2822	0.01089	0.2806	0.1678
Selected (24)	18.91	0.6106	0.2866	0.01163	0.3311	0.1719
Selected (12)	18.96	0.6138	0.2845	0.01188	0.3127	0.1696
Random (24)	18.29	0.6041	0.2949	0.01241	0.3310	0.1753
Low response (24)	17.12	0.5778	0.3336	0.01832	0.4107	0.1924

TABLE V: Quantitative comparison of LoRA adaptation with different block selections. Blocks are ranked by Shapley sensitivity; parentheses give the number adapted. Random uses half of the blocks, with metrics averaged over five training runs; Low response uses the least sensitive half. Other training settings follow the default configuration. Selecting roughly the most sensitive three quarters matches full-block performance and improves several reconstruction and camera metrics. Bold and underlining mark the best and second-best results within each backbone.

Fig. 8: Effect of training updates on generation quality and camera control. LongCat trains more stably with eight scenes and reaches its best perceptual quality near 1,200 updates; longer training shows signs of overfitting. H3 learns reliably from one scene. Removing LoRA from its less sensitive blocks accelerates convergence and improves perceptual quality compared with full-block adaptation. Additional updates yield diminishing gains.

LongCat
Scenes	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	ATE $\downarrow$	RPE-T $\downarrow$	RPE-R $\downarrow$
1	18.96	0.6108	0.2838	0.01087	0.2723	0.1886
2	19.00	0.6152	0.2851	0.01130	0.2834	0.1766
4	19.02	0.6167	0.2828	0.01048	0.2861	0.1638
8	19.03	0.6176	0.2822	0.01089	0.2806	0.1678
H3

TABLE VI: Effect of training-scene diversity on camera-controlled generation. The total number of training pairs is fixed. Both backbones already produce good results after single-scene training; covering two to four scenes captures most gains from added diversity. Further expansion brings limited benefit, showing that the adapter can learn reliable camera control from few training scenes. Bold and underlining mark the best and second-best results within each backbone.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	ATE $\downarrow$	RPE-T $\downarrow$	RPE-R $\downarrow$
Standard FM	18.55	0.5477	0.3407	0.01012	0.2590	0.1692
Only warm-up	18.92	0.6062	0.2834	0.01476	0.2985	0.1879
w/o warm-up	18.90	0.6084	0.2857	0.01276	0.2931	0.1801
w/o LoRA	18.07	0.5271	0.3640	0.01683	0.2986	0.1964
Full	19.03	0.6176	0.2822	0.01089	0.2806	0.1678

TABLE VII: Quantitative ablation of the two-stage training strategy and Trajectory LoRA. Standard FM trains on independently noised targets; Only warm-up and w/o warm-up isolate adaptation to recorded states and training across additional noise levels. Full combines both stages and improves reconstruction over Standard FM on both backbones, with the best camera accuracy on H3. Removing LoRA gives the poorest reconstruction.

Fig. 9: Qualitative comparison of Trajectory LoRA training strategies. For LongCat (a) and H3 (b), two examples per backbone place the input and warped reference beside No LoRA, Standard FM, and the full method. The full strategy reduces fragmented geometry and residual rendering artifacts, producing more continuous surfaces and finer scene detail than No LoRA and Standard FM. Red boxes and enlarged crops compare corresponding regions.

Coefficient	Value	Description
$\gamma_{k}$	$(0.4,0.4,0.2)$	Optical-flow metric weights
$\lambda_{i}$	$0.65$	FLF threshold coefficient
$\rho$	$4.0$	DSG guidance scale

Table S1: Default guidance coefficients for the Wan2.1 training-free experiments. The flow weights combine displacement, direction, and outlier errors;

\lambda_{i}

controls the adaptive channel threshold in Eq. ( S31 ), and

\rho

sets the strength of DSG correction.

Setting	LongCat	H3
Training scenes / pairs	8 / 50	1 / 50
Selected blocks	36 / 48	38 / 50
LoRA rank / $\alpha$	128 / 128	128 / 128
Optimizer	AdamW	AdamW
Learning rate	$2\times 10^{-5}$	$2\times 10^{-5}$
Warm-up iterations	200	200

Table S2: Default training settings for Trajectory LoRA. Both backbones first train on recorded guided anchors, then use a mixture of anchors, constructed states, and directly noised targets.

Figure S1: Contribution of IRR, FLF, and DSG to training-free camera control. IRR introduces the target camera motion; adding FLF reduces appearance distortions during reference fusion, and DSG further corrects the guided prediction. The complete method preserves coherent scene content along the requested camera path.

Figure S2: Channel filtering frequencies for training-free control over more than 40 static and 30 dynamic scenes. A higher frequency means that a channel more often retains its original prediction. Stable differences across channels support selecting where to inject the camera guidance according to motion relevance.

Figure S3: Combining the original and trajectory-guided predictions for training-free camera control. Standard CFG introduces structural artifacts, and removing DSG’s adaptive angular weight weakens consistency. Cosine-based projection with adaptive weighting preserves scene structure and fine detail in the guided result.

Backbone	CLIP $\uparrow$	ATE $\downarrow$	RPE-T $\downarrow$	RPE-R $\downarrow$
SVD	0.910	0.265	0.316	0.444
Wan2.1	0.948	0.077	0.086	0.221
LongCat-Video	0.949	0.095	0.076	0.230

Table S3: Training-free camera control across three video backbones on the static benchmark of more than 40 scenes. All backbones use the same rendered-reference guidance. LongCat-Video leads on CLIP and RPE-T, while Wan2.1 leads on ATE and RPE-R; both improve all reported metrics over SVD. Bold and underlining mark the best and second-best results.

Figure S4: Training-free camera control across video backbones. Comparisons under a shared SVD backbone show the effect of the guidance method. WorldForge also produces richer scene detail with LongCat-Video, using the same inference-time guidance.

Method	Backbone	Backbone params.	Training data	Training updates	Inference time (s)
UniWorld-View [ 83 ]	Wan2.1-VACE	14B	100K triplets + 100K pairs	20,000	124.5
TrajectoryCrafter [ 5 ]	CogVideoX-Fun	5B	120K multi-view + 60K pairs	15,000	169.0
SymphoMotion [ 84 ]	Wan2.1-I2V	14B	25K videos	—	442.9
Warp-as-History [ 18 ]	Helios	14B	1 scene / 4 clips	1,000	39.3
Ours (LongCat)	LongCat-Video	13.6B	8 scenes / 50 videos	1,200	274.2
Ours (LongCat+acceleration LoRA)	LongCat-Video	13.6B	8 scenes / 50 videos	1,200	67.0

Table S4: Training resources and inference efficiency for camera-controlled generation. Our trajectory adapters learn from 50 videos with 1,200 or 1,500 updates. Combining them with released acceleration LoRAs gives speedups of

4.09\times

on LongCat and

5.09\times

on H3. Times are mean seconds per generated video over 200 test cases for each method, estimated for a single A100 from local runs with native video configurations. Parameter counts refer to the backbones. Training updates exclude backbone pretraining; a dash denotes an unreported total update count.

Figure S5: Compatibility of training-free guidance with motion and geometry estimators. (a) FLF reduces rendering artifacts with both Farnebäck and RAFT optical flow. (b) Warped references from VGGT, UniDepth, Mega-SaM, and DepthCrafter support coherent scene layouts and plausible surfaces despite differences in their estimated geometry.

Figure S6: H3 generation with 50-step and eight-step sampling using the same trajectory LoRA. The accelerated results retain scene layout and object geometry in both examples. Enlargements compare corresponding details.

Figure S7: Camera-control limits under erroneous depth and complex scene dynamics. Distorted face geometry and inconsistencies around moving people show how reference errors remain visible in the generated views. Rows compare training-free WorldForge (top) with TrajectoryCrafter (bottom).

Figure S8: FLF in training-free generation under fast motion, non-rigid motion, and heavy occlusion. Comparing w/o FLF with w/ FLF shows that selecting channels for reference fusion reduces distortions in objects, water, and scene boundaries while retaining the requested viewpoint changes.

Figure S9: Training-free generation of an object-centric $360^{\circ}$ orbit from one input image. The final frame of each clip initializes the next segment, extending camera control beyond the original field of view while retaining recognizable scene content.

Figure S10: Camera and content editing with training-free WorldForge. Rendered references support camera freezing and stabilization while retaining scene content. An edited first frame and region masks propagate appearance changes through the video.

Figure S11: Additional camera-controlled generation on six held-out DL3DV cases beyond the main figure. Our LongCat and H3 results preserve scene layout and local appearance, while several baseline outputs exhibit distorted boundaries, texture loss, or rendering remnants. LongCat uses the main-table model; H3 uses the eight-step accelerated sampling evaluated in Sec. 4.4 . Inputs and warped references appear on the left, and red boxes identify corresponding enlarged regions.

Camera-Controlled Video Generation

Camera-controlled video generation creates videos that follow specified viewpoints and camera paths while preserving coherent scenes and motion.

See all 141 papers

Beyond Inpainting: Unleash 3D Understanding for Stable Camera-Controlled Video Generation
Jan 15, 2026Dong-Yu Chen, Yixin Guo, Shuojin Yang +2Camera-Controlled Video Generation Video Diffusion Models
BNRist, Department of Computer Science and Technology, Tsinghua University Beijing 100084, China
Probing into Camera Control of Video Models
May 14, 2026Chen Hou, Christian RupprechtDiffusion Model Guidance Camera-Controlled Video Generation
Visual Geometry Group · Visual Geometry Group University of Oxford
Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation
Jul 2, 2026Long Vu, Tan Ngo, Animesh Karnewar +5Camera-Controlled Video Generation Video Diffusion Models
Qualcomm AI Research · Trinity College Dublin, Ireland

Jan 15, 2026J/K move · Enter open · S save

Beyond Inpainting: Unleash 3D Understanding for Stable Camera-Controlled Video Generation

Dong-Yu Chen, Yixin Guo, Shuojin Yang, Tai-Jiang Mu+1

BNRist, Department of Computer Science and Technology, Tsinghua University Beijing 100084, China

Camera-Controlled Video Generation Video Diffusion Models Controllable Video Generation

Open paper AskPDF arXiv

Camera control has been extensively studied in conditioned video generation; however, performing precisely altering the camera trajectories while faithfully preserving the video content remains a challenging task. The mainstream approach to achieving precise camera control is warping a 3D representation according to the target trajectory. However, such methods fail to fully leverage the 3D priors of video diffusion models (VDMs) and often fall into the Inpainting Trap, resulting in subject inconsistency and degraded generation quality. To address this problem, we propose DepthDirector, a video re-rendering framework with precise camera controllability. By leveraging the depth video from explicit 3D representation as camera-control guidance, our method can faithfully reproduce the dynamic scene of an input video under novel camera trajectories. Specifically, we design a View-Content Dual-Stream Condition mechanism that injects both the source video and the warped depth sequence rendered under the target viewpoint into the pretrained video generation model. This geometric guidance signal enables VDMs to comprehend camera movements and leverage their 3D understanding capabilities, thereby facilitating precise camera control and consistent content generation. Next, we introduce a lightweight LoRA-based video diffusion adapter to train our framework, fully preserving the knowledge priors of VDMs. Additionally, we construct a large-scale multi-camera synchronized dataset named MultiCam-WarpData using Unreal Engine 5, containing 8K videos across 1K dynamic scenes. Extensive experiments show that DepthDirector outperforms existing methods in both camera controllability and visual quality. Our code and dataset will be publicly available.

Abstract

Camera-Controlled Video Generation

Explore similar work

Beyond Inpainting: Unleash 3D Understanding for Stable Camera-Controlled Video Generation

Probing into Camera Control of Video Models

Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation