UniCaMo: Unified Camera and Motion Control for Video Diffusion Models

Long Vu; Tan Ngo; Animesh Karnewar; Amir Habibian; Binh-Son Hua; Hung Bui; Minh Hoai Nguyen; Phong Nguyen

cs.CVOct 31, 2026

UniCaMo: Unified Camera and Motion Control for Video Diffusion Models

Authors: Long Vu, Tan Ngo, Animesh Karnewar, Amir Habibian, Binh-Son Hua, Hung Bui, Minh Hoai Nguyen, Phong Nguyen

Organizations: Qualcomm AI Research · Trinity College Dublin, Ireland

Abstract

We present UniCaMo, an unified framework for joint control of camera and object motion in image-and-text-to-video diffusion models. Given a reference image, text prompt, sparse 3D object trajectories, and a 3D camera trajectory, UniCaMo constructs motion-consistent input noise that directly encodes the desired motion. Our 3D-grounded noise construction combines reference-derived noise propagation with a spherical Gaussian noise representation for newly revealed regions, providing a simple and effective motion prior without requiring reconstruction of unseen scene geometry. UniCaMo requires no architectural modifications or dedicated control modules and can be adapted using lightweight LoRA fine-tuning with low training data volume. Experiments on standard controllable video generation benchmarks demonstrate state-of-the-art motion controllability and high visual quality with minimal inference overhead.

Figures & tables

Figure 1: This teaser demonstrates a man walking and dancing around a woman while the camera pans to the right. UniCaMo jointly controls both object and camera motion through a 3D-grounded noise construction, enabling accurate occlusion handling and consistent appearance in unseen regions across frames, an effect that previous work with 2D cues ( Chu et al., 2025 ) cannot achieve.

Figure 2: The UniCamo pipeline begins with 3D-lifting the input reference image to a 3D point cloud, within which per-frame control signals are defined via an interactive GUI, producing 3D point tracks and camera trajectories. From these, a 3D-grounded scene representation from three noise components: foreground noise derived from 3D point tracks, background noise from depth cues, and 3D spherical noise for persistent out-of-view anchors, which is then merged by a depth-aware noise composition step. The 3D-grounded noise is consumed by a LoRA finetuned video diffusion model to generate a video exhibiting corresponding object and camera motion.

Figure 3: Qualitative results on MoveBench between UniCaMo and recent approaches ( Chu et al., 2025 ; Wang et al., 2025 ; Zhang et al., 2026 ) .

Figure 4: Qualitative results on RE10K between UniCaMo and other baselines ( Zhang et al., 2026 ; Ren et al., 2025 ) . Our method enable more precise camera control than the baselines.

Figure 5: Jointly control over camera and object motions. While other works struggles to generate realistic object dynamics and accurate camera poses, UniCaMo achieves high-quality control.

Figure 6: Qualitative results on ablation study of our 3D-aware structured motion noise.

	Test cases with single-object motion					Test cases with multi-object motions
Methods	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	EPE $\downarrow$	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	EPE $\downarrow$
ImageConductor ( Li et al., 2025b )	34.51	424.11	13.43	0.49	15.66	77.52	764.15	13.92	0.51	9.80
Tora ( Zhang et al., 2025b )	22.57	100.24	15.73	0.55	3.30	53.27	350.00	14.50	0.54	3.50
ATI ( Wang et al., 2025 )	11.27	105.35	15.44	0.58	5.09	30.72	278.91	16.19	0.62	3.17
Go-with-the-Flow (CogX-5B) ( Burgert et al., 2025 )	12.49	216.90	15.80	0.62	3.04	35.72	399.40	16.93	0.60	3.71
Go-with-the-Flow (Wan2.1-14B)	12.31	96.14	17.60	0.64	2.71	31.06	323.15	17.21	0.63	2.94

Table 1: Evaluation results on MoveBench. We report quantitative comparisons on both the single-object and multi-object motion. Best results are shown in bold ; second-best results are underlined .

Methods	FID $\downarrow$	FVD $\downarrow$	RotErr $\downarrow$	TransErr $\downarrow$	CamMC $\downarrow$
MotionCtrl ( Wang et al., 2024b )	23.58	159.52	0.38	7.17	7.23
CamI2V ( Zheng et al., 2024 )	23.38	167.45	0.26	5.15	5.23
CameraCtrl ( He et al., 2024 )	23.80	165.37	0.27	3.25	3.34
Gen3C ( Ren et al., 2025 )	6.87	52.39	0.33	2.14	2.28
SymphoMotion ( Zhang et al., 2026 )	10.33	87.40	1.33	6.31	6.90
UniCaMo (Ours)	5.99	39.93	0.26	1.89	1.98

Table 2: Camera-control evaluation on 300 randomly sampled clips from the RealEstate10K test set. We report rotation error (RotErr), translation error (TransErr), camera motion consistency (CamMC), and FID and FVD for visual quality. Best and second-best are bolded and underlined .

Method	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	EPE $\downarrow$	RotErr $\downarrow$	TransErr $\downarrow$	CamMC $\downarrow$
ATI ( Wang et al., 2025 )	27.83	336.25	14.58	0.54	8.64	18.59	46.41	57.81
Wan-Move ( Chu et al., 2025 )	23.81	294.49	15.28	0.60	5.03	8.84	44.41	48.49
Wan-Move + TAPIP3D tracks ( Chu et al., 2025 )	24.17	288.52	15.16	0.58	4.92	8.12	42.11	46.63
SymphoMotion ( Zhang et al., 2026 )	20.67	219.86	17.95	0.70	3.28	2.34	20.88	21.96
UniCaMo (Ours)	18.19	192.43	18.88	0.72	2.95	1.23	16.27	16.74

Table 3: Evaluation of joint camera and object motion control on 300 clips randomly curated from publicly available sources, each containing simultaneous camera and object motion. Best results are shown in bold ; second-best results are underlined .

Setting	FG noise	BG noise	Sph. noise	DA comp.	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	EPE $\downarrow$	RotErr $\downarrow$	TransErr $\downarrow$	CamMC $\downarrow$
A	✗	✗	✗	✗	21.59	238.41	15.58	0.62	8.23	14.45	43.78	52.86
B	✓	✗	✗	✓	19.38	210.29	17.32	0.66	3.98	7.76	30.45	34.68
C	✓	✓	✗	✓	18.32	191.51	18.15	0.73	3.16	1.51	18.42	19.11
D	✓	✓	✓	✗	18.51	196.23	18.85	0.71	3.35	1.74	18.85	19.56
E	✓	✓	✓	✓	18.19	192.43	18.88	0.72	2.95	1.23	16.27	16.74

Table 4: Ablation study of our noise construction method on 300 clips containing simultaneous camera and object motion. Best in bold ; second-best underlined .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Visualization of our interactive GUI for unified camera and object motion control. Users select objects via polygon masks on the 2D image and edit 3D object trajectories and camera paths using interactive gizmos in the 3D view. Notice how 3D point-tracks allow us to define the exact 3D trajectory of the woman in yellow dress (go behind the man).

(a) Degradation rate $\theta$
Setting	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	EPE $\downarrow$
$\theta=0.0$	26.93	235.3	18.35	0.72	1.88
$\theta=0.2$	26.1	215.2	18.15	0.72	1.92
$\theta=0.5$	26.03	212.78	17.35	0.67	2.12
(b) Number of 3D point tracks
$N=1024$	26.1	215.2	18.15	0.72	1.92

Appendix

Table 5: Analysis of (a) the degradation rate

\theta

for blending the constructed motion-consistent noise with Gaussian noise, (b) the number of 3D point tracks, and (c) the foreground noise patch dimension, evaluated on MoveBench evaluation data with multi-object motions.

(a) Perturbation of the estimated depth
Setting	FID $\downarrow$	FVD $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	EPE $\downarrow$
Depth perturbation $=0\%$ (default)	26.08	215.20	18.15	0.72	1.92
Depth perturbation $=5\%$	26.12	215.21	18.11	0.71	1.94
Depth perturbation $=10\%$	27.14	223.56	17.35	0.68	2.10
Depth perturbation $=30\%$	31.20	283.32	16.22	0.60	2.57
(b) Source of the estimated camera poses and depth

Appendix

Table 6: Robustness of UniCaMo to errors in the estimated 3D signals, evaluated on the multi-object split of MoveBench. (a) perturbs the per-pixel depth by a controlled percentage; (b) substitutes the camera poses and depth of different 3D foundation models. Best result in each block in bold .

Figure 8: Additional qualitative results for camera control on the RealEstate10K dataset.

Figure 9: Additional qualitative results for motion control on the MoveBench dataset.

Figure 10: Additional qualitative results for joint camera and motion control .

Figure 11: Qualitative results for joint control on copmlex trajectory cases

Figure 12: Effect of degradation rate on generated samples. A lower degradation rate ( $\theta=0.0$ ) offers more precise trajectory control but introduces visible artifacts. Conversely, a higher degradation rate ( $\theta=0.5$ ) yields visually plausible videos but reduces 3D trajectory accuracy. We adopt $\theta=0.2$ as the default, which achieves the best trade-off between motion fidelity and visual quality.

Figure 13: Qualitative results of our proposed UniCaMo method using different number of 3D point tracks.

Figure 14: Qualitative results of our proposed UniCaMo method using different configuration of patch size $R$ .

Controllable Video Generation

Controllable video generation steers how videos look, move, and change over time using user-provided instructions and structured controls.

See all 335 papers

Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation
Jul 2, 2026Long Vu, Tan Ngo, Animesh Karnewar +5Camera-Controlled Video Generation Video Diffusion Models
Qualcomm AI Research · Trinity College Dublin, Ireland
Probing into Camera Control of Video Models
May 14, 2026Chen Hou, Christian RupprechtDiffusion Model Guidance Camera-Controlled Video Generation
Visual Geometry Group · Visual Geometry Group University of Oxford
UniMoCa: Unifying Motion and Camera Controls as Visual Proxies for Faithful Human Video Generation
Aug 3, 2026Liming Tan, Ye Chen, Hao Zhang +3Camera-Controlled Video Generation Pose-Conditioned Video Generation
Shanghai Jiao Tong University · Monash University · USC-SJTU Institute of Cultural and Creative Industry

Jul 2, 2026J/K move · Enter open · S save

Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation

Long Vu, Tan Ngo, Animesh Karnewar, Amir Habibian+4

Qualcomm AI Research · Trinity College Dublin, Ireland

Camera-Controlled Video Generation Video Diffusion Models Diffusion Models for Motion Generation Controllable Video Generation Trajectory-Guided Video Generation 3D-Aware Video Generation

Open paper AskPDF arXiv

Modern image-and-text-to-video diffusion models can synthesize highly realistic videos by iteratively denoising an initial Gaussian noise tensor conditioned on reference image and text inputs. However, existing approaches still lack precise and unified controllability over both object motion and camera motion within a single generation process. We present UniCaMo, a unified framework that enables simultaneous control of object trajectories and camera viewpoints by directly constructing the input noise of the diffusion model. Specifically, UniCaMo builds a shared 3D-grounded motion-consistent noise space across latent video frames. Sparse 3D point tracks are used to warp the Gaussian noise of the reference frame along desired object trajectories, while a virtual spherical noise representation provides globally consistent noise values for newly revealed scene regions under camera motion. By combining local track-guided noise warping with global sphere-based noise sampling, UniCaMo maintains geometric and temporal consistency under both object movement and viewpoint changes. Because UniCaMo modifies only the input noise, it requires no auxiliary adapters, control branches, or architectural changes to the underlying video diffusion model. With lightweight LoRA fine-tuning on large pretrained video diffusion models, including Wan 2.1 (14B), UniCaMo achieves state-of-the-art results in both video quality and motion controllability on standard controllable video generation benchmarks.

Abstract

Controllable Video Generation

Explore similar work

Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation

Probing into Camera Control of Video Models

UniMoCa: Unifying Motion and Camera Controls as Visual Proxies for Faithful Human Video Generation