Video diffusion models have learned rich 3D/4D priors, making them promising for scene generation and video re-rendering. However, their use in spatial tasks is limited by imprecise camera control, spatial and temporal inconsistencies, and entangled camera motion with scene dynamics. Existing approaches, such as task-specific fine-tuning and post-process warping, often introduce visual artifacts, generalize poorly, or incur high computational costs. We propose WorldForge V2 to achieve accurate camera control while preserving the generative capabilities of pretrained video models. We first develop a training-free guidance framework using warped references to iteratively refine generation along a target camera path. Optical-flow analysis identifies motion-related latent channels for selective guidance, helping separate motion control from appearance. A dual-path strategy further compares guided and unguided predictions to correct artifacts from imperfect references. Together, these components guide viewpoint changes while preserving scene appearance and object motion. To reduce refinement costs, v-cache approximates the correction using the preceding denoising prediction, avoiding an additional model evaluation. We further extend motion localization from latent channels to network blocks and introduce Trajectory LoRA, which concentrates adaptation on blocks most responsive to trajectory guidance. A two-stage training strategy teaches these adapters to denoise the states encountered during guided generation, using limited supervision effectively. This enables reliable camera control with very little training data, with a single training scene already supporting generalization to unseen scenes. Quantitative and qualitative evaluations across multiple datasets show that our method achieves state-of-the-art generation quality and trajectory accuracy in camera-controlled video generation.
Figures & tables
Fig. 1: Comparison with existing methods. With limited training data and iterations, our method achieves state-of-the-art visual quality and trajectory accuracy in camera-controlled video generation.
Fig. 2: Overview of WorldForge V2. Given a single image or video frame and a target camera trajectory, the scene geometry is reconstructed and rendered into trajectory-aligned warped frames. WorldForge V2 first performs training-free trajectory guidance in the predicted clean latent space using IRR, FLF, and DSG. Building on this framework, we further introduce an efficient route that directly fuses trajectory information in the noisy latent space with V-Cache, together with a trajectory-sensitive lightweight LoRA scheme for camera trajectory control using only limited training data. The resulting video follows the specified camera path while preserving the generative capability of the pretrained video diffusion model.
Fig. 3: Qualitative comparison for 3D novel-view synthesis and 4D trajectory-controlled re-rendering. For single-image 3D generation (left), our method produces more coherent geometry and plausible newly revealed content. For dynamic re-rendering (right), it preserves human appearance and object motion while reducing floating structures and flattened faces. The training-free guidance yields more realistic results in both static and dynamic scenes.
Method
FID ↓
CLIP ↑
ATE ↓
RPE-T ↓
RPE-R ↓
See3D
123.26
0.941
0.091
0.089
0.250
ViewCrafter
117.50
0.930
0.236
0.315
0.728
ViewExtrapolator
125.50
0.930
0.183
0.260
0.882
TrajectoryAttention
122.37
0.920
0.159
0.238
0.532
TrajectoryCrafter
111.49
0.910
0.090
0.152
0.267
NVS-Solver
118.64
0.937
0.224
0.268
1.056
TABLE I: Quantitative comparison on 3D static scenes using official implementations and identical source inputs. FID and CLIP assess generation quality; ATE, RPE-T, and RPE-R assess trajectory accuracy. Our training-free method leads on all metrics. ↑ : higher is better; ↓ : lower is better. Bold and underlining mark the best and second-best results.
Method
FVD ↓
CLIP-V ↑
ATE ↓
RPE-T ↓
RPE-R ↓
ViewExtrapolator
108.48
0.913
1.040
1.208
4.750
TrajectoryAttention
106.94
0.911
0.605
1.238
3.560
TrajectoryCrafter
97.31
0.923
0.431
1.078
8.950
Ours (Training-Free)
93.17
0.938
0.527
0.826
2.690
TABLE II: Quantitative comparison on 4D dynamic re-rendering using official implementations with matched source videos and target trajectories. FVD and CLIP-V assess video quality; ATE, RPE-T, and RPE-R assess trajectory accuracy. Our method combines the best video quality with the lowest local camera errors. ↑ : higher is better; ↓ : lower is better. Bold and underlining mark the best and second-best results.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Vis-LPIPS ↓
Disocc-LPIPS ↓
ATE ↓
RPE-T ↓
RPE-R ↓
UniWorld-View
17.15
0.5958
0.3370
0.2799
0.4431
0.00625
0.2333
0.1218
TrajectoryCrafter
17.41
0.5986
0.3558
0.3066
0.4512
0.00710
0.2013
0.0990
SymphoMotion
15.38
0.4760
0.4433
0.4215
0.4908
0.02292
0.4065
0.2196
Warp-as-History
15.48
0.4553
0.4206
0.3829
0.4939
0.03533
0.4856
0.2398
Ours (LongCat)
19.72
0.6663
0.2748
0.2231
0.3640
0.00905
0.2725
0.1324
Ours (H3)
20.21
0.6806
0.2698
0.2244
0.3459
0.00685
0.1647
0.0780
TABLE III: Quantitative comparison of Trajectory LoRA with camera-control baselines. PSNR, SSIM, and LPIPS assess target-view reconstruction; Vis-LPIPS and Disocc-LPIPS assess visible and newly exposed regions. Both adapted backbones improve reconstruction across these metrics, preserving observed content and completing new regions more faithfully. H3 also achieves the lowest local camera errors. Bold and underlining mark the best and second-best results.
Fig. 4: Noise-state consistency under different reference-insertion strategies. Direct Insertion leaves gaps in the inferred-noise mean and log-variance between visible and model-generated regions, accompanied by noisy patches and broken structure. IRR reduces these gaps, and V-Cache approximates its correction while retaining continuous surfaces and clear geometric detail.
Fig. 5: Qualitative comparison of camera-controlled scene generation. Our LongCat and H3 models recover coherent object shapes and material detail from incomplete warped references, including thin structures and partially revealed objects. Compared with the baselines, they produce fewer rendering remnants and more continuous geometry, preserving scene appearance as the camera reveals new content.
Fig. 6: Qualitative transfer of Trajectory LoRA across camera-control methods. Without additional fine-tuning, the same trained LongCat adapter improves surface continuity and object detail in WAH [ 18 ] and GWF [ 16 ] , demonstrating transferable trajectory control.
Method
FID ↓
CLIP ↑
FVD ↓
CLIP-V ↑
w/o DSG
109.43
0.943
95.69
0.937
w/o FLF
112.69
0.945
99.79
0.932
w/o DSG and FLF
113.12
0.943
103.17
0.931
DSG (CFG form)
120.91
0.936
109.10
0.919
Ours (Training-Free)
96.08
0.948
93.17
0.938
TABLE IV: Quantitative ablation of inference-time guidance. IRR supplies the trajectory signal; removing its reference injection leaves the model without camera guidance. With IRR fixed, we remove FLF, DSG, or both, or replace DSG with its CFG form. Both components improve generation quality, and their combination performs best on static and dynamic scenes. Bold and underlining mark the best and second-best results.
Fig. 7: Shapley analysis of block sensitivity to trajectory supervision. After approximately 400 steps of probe LoRA training, contributions are estimated from 25 trajectory pairs per backbone. Sensitive blocks are dispersed through the middle and later layers of LongCat and show a more regular concentration in the later layers of H3. This variation identifies the blocks to prioritize for adaptation.
Selection
PSNR ↑
SSIM ↑
LPIPS ↓
ATE ↓
RPE-T ↓
RPE-R ↓
All blocks (48)
18.98
0.6140
0.2847
0.01045
0.2904
0.1693
Selected (36)
19.03
0.6176
0.2822
0.01089
0.2806
0.1678
Selected (24)
18.91
0.6106
0.2866
0.01163
0.3311
0.1719
Selected (12)
18.96
0.6138
0.2845
0.01188
0.3127
0.1696
Random (24)
18.29
0.6041
0.2949
0.01241
0.3310
0.1753
Low response (24)
17.12
0.5778
0.3336
0.01832
0.4107
0.1924
TABLE V: Quantitative comparison of LoRA adaptation with different block selections. Blocks are ranked by Shapley sensitivity; parentheses give the number adapted. Random uses half of the blocks, with metrics averaged over five training runs; Low response uses the least sensitive half. Other training settings follow the default configuration. Selecting roughly the most sensitive three quarters matches full-block performance and improves several reconstruction and camera metrics. Bold and underlining mark the best and second-best results within each backbone.
Fig. 8: Effect of training updates on generation quality and camera control. LongCat trains more stably with eight scenes and reaches its best perceptual quality near 1,200 updates; longer training shows signs of overfitting. H3 learns reliably from one scene. Removing LoRA from its less sensitive blocks accelerates convergence and improves perceptual quality compared with full-block adaptation. Additional updates yield diminishing gains.
Scenes
PSNR ↑
SSIM ↑
LPIPS ↓
ATE ↓
RPE-T ↓
RPE-R ↓
LongCat
1
18.96
0.6108
0.2838
0.01087
0.2723
0.1886
2
19.00
0.6152
0.2851
0.01130
0.2834
0.1766
4
19.02
0.6167
0.2828
0.01048
0.2861
0.1638
8
19.03
0.6176
0.2822
0.01089
0.2806
0.1678
H3
TABLE VI: Effect of training-scene diversity on camera-controlled generation. The total number of training pairs is fixed. Both backbones already produce good results after single-scene training; covering two to four scenes captures most gains from added diversity. Further expansion brings limited benefit, showing that the adapter can learn reliable camera control from few training scenes. Bold and underlining mark the best and second-best results within each backbone.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
ATE ↓
RPE-T ↓
RPE-R ↓
Standard FM
18.55
0.5477
0.3407
0.01012
0.2590
0.1692
Only warm-up
18.92
0.6062
0.2834
0.01476
0.2985
0.1879
w/o warm-up
18.90
0.6084
0.2857
0.01276
0.2931
0.1801
w/o LoRA
18.07
0.5271
0.3640
0.01683
0.2986
0.1964
Full
19.03
0.6176
0.2822
0.01089
0.2806
0.1678
TABLE VII: Quantitative ablation of the two-stage training strategy and Trajectory LoRA. Standard FM trains on independently noised targets; Only warm-up and w/o warm-up isolate adaptation to recorded states and training across additional noise levels. Full combines both stages and improves reconstruction over Standard FM on both backbones, with the best camera accuracy on H3. Removing LoRA gives the poorest reconstruction.
Fig. 9: Qualitative comparison of Trajectory LoRA training strategies. For LongCat (a) and H3 (b), two examples per backbone place the input and warped reference beside No LoRA, Standard FM, and the full method. The full strategy reduces fragmented geometry and residual rendering artifacts, producing more continuous surfaces and finer scene detail than No LoRA and Standard FM. Red boxes and enlarged crops compare corresponding regions.
Coefficient
Value
Description
γk
(0.4,0.4,0.2)
Optical-flow metric weights
λi
0.65
FLF threshold coefficient
ρ
4.0
DSG guidance scale
Table S1: Default guidance coefficients for the Wan2.1 training-free experiments. The flow weights combine displacement, direction, and outlier errors; λi controls the adaptive channel threshold in Eq. ( S31 ), and ρ sets the strength of DSG correction.
Setting
LongCat
H3
Training scenes / pairs
8 / 50
1 / 50
Selected blocks
36 / 48
38 / 50
LoRA rank / α
128 / 128
128 / 128
Optimizer
AdamW
AdamW
Learning rate
2×10−5
2×10−5
Warm-up iterations
200
200
Table S2: Default training settings for Trajectory LoRA. Both backbones first train on recorded guided anchors, then use a mixture of anchors, constructed states, and directly noised targets.
Figure S1: Contribution of IRR, FLF, and DSG to training-free camera control. IRR introduces the target camera motion; adding FLF reduces appearance distortions during reference fusion, and DSG further corrects the guided prediction. The complete method preserves coherent scene content along the requested camera path.
Figure S2: Channel filtering frequencies for training-free control over more than 40 static and 30 dynamic scenes. A higher frequency means that a channel more often retains its original prediction. Stable differences across channels support selecting where to inject the camera guidance according to motion relevance.
Figure S3: Combining the original and trajectory-guided predictions for training-free camera control. Standard CFG introduces structural artifacts, and removing DSG’s adaptive angular weight weakens consistency. Cosine-based projection with adaptive weighting preserves scene structure and fine detail in the guided result.
Backbone
CLIP ↑
ATE ↓
RPE-T ↓
RPE-R ↓
SVD
0.910
0.265
0.316
0.444
Wan2.1
0.948
0.077
0.086
0.221
LongCat-Video
0.949
0.095
0.076
0.230
Table S3: Training-free camera control across three video backbones on the static benchmark of more than 40 scenes. All backbones use the same rendered-reference guidance. LongCat-Video leads on CLIP and RPE-T, while Wan2.1 leads on ATE and RPE-R; both improve all reported metrics over SVD. Bold and underlining mark the best and second-best results.
Figure S4: Training-free camera control across video backbones. Comparisons under a shared SVD backbone show the effect of the guidance method. WorldForge also produces richer scene detail with LongCat-Video, using the same inference-time guidance.
Method
Backbone
Backbone params.
Training data
Training updates
Inference time (s)
UniWorld-View [ 83 ]
Wan2.1-VACE
14B
100K triplets + 100K pairs
20,000
124.5
TrajectoryCrafter [ 5 ]
CogVideoX-Fun
5B
120K multi-view + 60K pairs
15,000
169.0
SymphoMotion [ 84 ]
Wan2.1-I2V
14B
25K videos
—
442.9
Warp-as-History [ 18 ]
Helios
14B
1 scene / 4 clips
1,000
39.3
Ours (LongCat)
LongCat-Video
13.6B
8 scenes / 50 videos
1,200
274.2
Ours (LongCat+acceleration LoRA)
LongCat-Video
13.6B
8 scenes / 50 videos
1,200
67.0
Table S4: Training resources and inference efficiency for camera-controlled generation. Our trajectory adapters learn from 50 videos with 1,200 or 1,500 updates. Combining them with released acceleration LoRAs gives speedups of 4.09× on LongCat and 5.09× on H3. Times are mean seconds per generated video over 200 test cases for each method, estimated for a single A100 from local runs with native video configurations. Parameter counts refer to the backbones. Training updates exclude backbone pretraining; a dash denotes an unreported total update count.
Figure S5: Compatibility of training-free guidance with motion and geometry estimators. (a) FLF reduces rendering artifacts with both Farnebäck and RAFT optical flow. (b) Warped references from VGGT, UniDepth, Mega-SaM, and DepthCrafter support coherent scene layouts and plausible surfaces despite differences in their estimated geometry.
Figure S6: H3 generation with 50-step and eight-step sampling using the same trajectory LoRA. The accelerated results retain scene layout and object geometry in both examples. Enlargements compare corresponding details.
Figure S7: Camera-control limits under erroneous depth and complex scene dynamics. Distorted face geometry and inconsistencies around moving people show how reference errors remain visible in the generated views. Rows compare training-free WorldForge (top) with TrajectoryCrafter (bottom).
Figure S8: FLF in training-free generation under fast motion, non-rigid motion, and heavy occlusion. Comparing w/o FLF with w/ FLF shows that selecting channels for reference fusion reduces distortions in objects, water, and scene boundaries while retaining the requested viewpoint changes.
Figure S9: Training-free generation of an object-centric 360∘ orbit from one input image. The final frame of each clip initializes the next segment, extending camera control beyond the original field of view while retaining recognizable scene content.
Figure S10: Camera and content editing with training-free WorldForge. Rendered references support camera freezing and stabilization while retaining scene content. An edited first frame and region masks propagate appearance changes through the video.
Figure S11: Additional camera-controlled generation on six held-out DL3DV cases beyond the main figure. Our LongCat and H3 results preserve scene layout and local appearance, while several baseline outputs exhibit distorted boundaries, texture loss, or rendering remnants. LongCat uses the main-table model; H3 uses the eight-step accelerated sampling evaluated in Sec. 4.4 . Inputs and warped references appear on the left, and red boxes identify corresponding enlarged regions.
Camera control has been extensively studied in conditioned video generation; however, performing precisely altering the camera trajectories while faithfully preserving the video content remains a challenging task. The mainstream approach to achieving precise camera control is warping a 3D representation according to the target trajectory. However, such methods fail to fully leverage the 3D priors of video diffusion models (VDMs) and often fall into the Inpainting Trap, resulting in subject inconsistency and degraded generation quality. To address this problem, we propose DepthDirector, a video re-rendering framework with precise camera controllability. By leveraging the depth video from explicit 3D representation as camera-control guidance, our method can faithfully reproduce the dynamic scene of an input video under novel camera trajectories. Specifically, we design a View-Content Dual-Stream Condition mechanism that injects both the source video and the warped depth sequence rendered under the target viewpoint into the pretrained video generation model. This geometric guidance signal enables VDMs to comprehend camera movements and leverage their 3D understanding capabilities, thereby facilitating precise camera control and consistent content generation. Next, we introduce a lightweight LoRA-based video diffusion adapter to train our framework, fully preserving the knowledge priors of VDMs. Additionally, we construct a large-scale multi-camera synchronized dataset named MultiCam-WarpData using Unreal Engine 5, containing 8K videos across 1K dynamic scenes. Extensive experiments show that DepthDirector outperforms existing methods in both camera controllability and visual quality. Our code and dataset will be publicly available.
Dong-Yu Chen, Yixin Guo, Shuojin Yang +2
BNRist, Department of Computer Science and Technology, Tsinghua University Beijing 100084, China
Video is a rich and scalable source of 3D/4D visual observations, and camera control is a key capability for video generation models to produce geometrically meaningful content. Existing approaches typically learn a mapping from camera motion to video using additional camera modules and paired data. However, such datasets are often limited in scale, diversity, and scene dynamics, which can bias the model toward a narrow output distribution and compromise the strong prior learned by the base model. These limitations motivate a different perspective on camera control. In this paper, we show that camera control need not be modeled as an implicit mapping problem, but can instead be treated as a form of geometric guidance that induces displacements across frames. Specifically, we reformulate camera control into a set of displacement fields and apply them via differentiable resampling of latent features during denoising. Our simple approach achieves effective camera control with minimal degradation across diverse quality metrics compared to fine-tuned baselines. Since our method is applicable to most video diffusion models without training, it can also serve as a probe to study the camera control capabilities of base models. Using this probe, we identify universal biases shared by representative video models, as well as disparities in their responses to camera control. Finally, we benchmark their performance in multi-view generation, offering insights into their potential for 3D/4D tasks.
Chen Hou, Christian Rupprecht
Visual Geometry Group · Visual Geometry Group University of Oxford
Modern image-and-text-to-video diffusion models can synthesize highly realistic videos by iteratively denoising an initial Gaussian noise tensor conditioned on reference image and text inputs. However, existing approaches still lack precise and unified controllability over both object motion and camera motion within a single generation process. We present UniCaMo, a unified framework that enables simultaneous control of object trajectories and camera viewpoints by directly constructing the input noise of the diffusion model. Specifically, UniCaMo builds a shared 3D-grounded motion-consistent noise space across latent video frames. Sparse 3D point tracks are used to warp the Gaussian noise of the reference frame along desired object trajectories, while a virtual spherical noise representation provides globally consistent noise values for newly revealed scene regions under camera motion. By combining local track-guided noise warping with global sphere-based noise sampling, UniCaMo maintains geometric and temporal consistency under both object movement and viewpoint changes. Because UniCaMo modifies only the input noise, it requires no auxiliary adapters, control branches, or architectural changes to the underlying video diffusion model. With lightweight LoRA fine-tuning on large pretrained video diffusion models, including Wan 2.1 (14B), UniCaMo achieves state-of-the-art results in both video quality and motion controllability on standard controllable video generation benchmarks.
Long Vu, Tan Ngo, Animesh Karnewar +5
Qualcomm AI Research · Trinity College Dublin, Ireland