Do View-Consistent Representations Matter for Camera-Controlled Video Generation?

Duolikun Danier; Ge Gao; Chengxi Zeng; Steven McDonagh; Changjian Li; Hakan Bilen; Oisin Mac Aodha

cs.CVNov 24, 2025

Do View-Consistent Representations Matter for Camera-Controlled Video Generation?

Authors: Duolikun Danier, Ge Gao, Chengxi Zeng, Steven McDonagh, Changjian Li, Hakan Bilen, Oisin Mac Aodha

Organizations: University of Edinburgh · University of Bristol

Abstract

Camera-controlled video generation has attracted increasing interest for its applications in simulation, gaming, and world modeling. However, current models often deform rigid objects and scene structure as the viewpoint changes. We investigate whether this failure is related to a measurable internal property: the view consistency of diffusion representations. Across a diverse collection of camera-controlled image-to-video diffusion models, we find that models exhibiting more view-consistent representations generate videos with superior 3D consistency. We complement this correlation-based evidence with a within-model controlled intervention, ViCoDR, which directly supervises the view consistency of internal diffusion representations, during training. Across camera-controlled image-to-video, camera-controlled text-to-video, and multi-view generation tasks, ViCoDR consistently improves output 3D consistency. Finally, we evaluate the loss used for these interventions as an auxiliary training objective by retraining a camera-controlled I2V model using either ViCoDR or existing methods for supervising diffusion representations under the same training setup. We show that the ViCoDR loss improves output 3D consistency more reliably than the alternatives and remains effective across design variants, while exhibiting camera-control trade-offs and requiring additional training computation. Together, these results establish view consistency of internal diffusion representations as an important property for analyzing and improving camera-controlled video generation. Project page: https://danier97.github.io/ViCoDR.

Figures & tables

Figure 1: Internal view consistency is strongly associated with output 3D consistency. MEt3R and geometric-correspondence results are shown for seven camera-controlled I2V models spanning U-Net and DiT architectures on RE10K [ 115 ] and DL3DV [ 51 ] .

Figure 2: Overview of ViCoDR. During video diffusion training, ViCoDR additionally supervises internal diffusion representations $(h^{a},h^{b})$ extracted from frame pairs $(x^{a},x^{b})$ with a 3D correspondence loss ( Eq. 5 ), so as to learn view-consistent representations (here PCA feature maps are visualized).

RE10K
Model	PSNR ( $\uparrow$ )	FVD ( $\downarrow$ )	RErr ( $\downarrow$ )	TErr ( $\downarrow$ )	Succ % ( $\uparrow$ )	MEt3R ( $\downarrow$ )	RPE ( $\downarrow$ )
CameraCtrl	16.89	113	0.80	2.31	86.3	0.066	0.385
+ ViCoDR	17.04	115	0.61	1.88	92.3	0.059	0.334
DaS	17.23	108	0.52	1.65	93.5	0.056	0.310
+ ViCoDR	17.24	109	0.51	1.62	93.8	0.053	0.295
RealCamI2V	17.36	104	0.47	1.50	94.4	0.052	0.286

Table 1: ViCoDR interventions for camera-controlled I2V. We compare image/video quality , camera-control accuracy , and output 3D consistency under matched training. Each base model is followed by its matched +ViCoDR run. DaS and RealCamI2V already use explicit 3D-aware conditioning.

Model	FVD ( $\downarrow$ )	RErr ( $\downarrow$ )	TErr ( $\downarrow$ )	Succ % ( $\uparrow$ )	MEt3R ( $\downarrow$ )	RPE ( $\downarrow$ )
MotionCtrl	780	1.119	4.676	89.1	0.075	0.306
+ ViCoDR	559	1.217	5.288	96.4	0.059	0.255
UCPE	520	0.569	1.257	95.0	0.055	0.300
+ ViCoDR	539	0.509	1.139	95.8	0.049	0.266

Table 2: ViCoDR interventions for camera-controlled T2V on RE10K.

Model	CD ( $\downarrow$ )	Depth ( $\downarrow$ )	cPSNR ( $\uparrow$ )	cSSIM ( $\uparrow$ )	cLPIPS ( $\downarrow$ )
ViFiGen	2.98	11.66	28.91	0.92	0.04
+ ViCoDR	2.97	11.59	29.03	0.94	0.05

Table 3: ViCoDR intervention for multi-view generation on OmniObject3D.

Panel of Figure 3 — Figure 3: Camera-controlled I2V comparison on CameraCtrl. Training with ViCoDR reduces geometric deformation while respecting the conditioning camera trajectory. Frame indices and camera poses are shown in each example.

Figure 4: Camera-controlled T2V comparison on MotionCtrl. Arrows highlight 3D-consistent and 3D-inconsistent regions. See the Appendix for additional examples.

Figure 5: Multi-view generation comparison on ViFiGen. Training with ViCoDR maintains consistent geometry.

	Model	PSNR ( $\uparrow$ )	FVD ( $\downarrow$ )	RErr ( $\downarrow$ )	TErr ( $\downarrow$ )	Succ % ( $\uparrow$ )	MEt3R ( $\downarrow$ )	RPE ( $\downarrow$ )
RE10K	CameraCtrl	1 6.888	113	0.795	2.313	86.3	0 .066	0.385
	+ Track4Gen	16.211	132	0.852	2.861	88.7	0.072	0.398
	+ GF	16.845	117	0 .721	2 .234	9 0.0	0.067	0.381
	+ JOG3R	16.103	120	0.913	2.702	86.7	0 .066	0 .375
	+ ViCoDR	17.043	1 15	0.605	1.883	92.3	0.059	0.334
DL3DV	CameraCtrl	1 3.318	113	2 .119	4.619	63.4	0.120	0.662

Table 4: Alternative auxiliary objectives on CameraCtrl. Improvements from GF, JOG3R, and Track4Gen are mixed across metrics and datasets; ViCoDR is the only evaluated objective that improves both consistency metrics on both datasets.

Model	FVD ( $\downarrow$ )	RErr ( $\downarrow$ )	TErr ( $\downarrow$ )	Succ % ( $\uparrow$ )	MEt3R ( $\downarrow$ )	RPE ( $\downarrow$ )
CameraCtrl	113	0.795	2.313	86.3	0.066	0.385
+ ViCoDR (simple)	118	0.640	1.950	91.0	0.061	0.346
+ ViCoDR (w/o edge)	118	0.618	1.966	92.0	0.061	0.350
+ ViCoDR (DUSt3R)	117	0.627	2.012	90.3	0.060	0.341
+ ViCoDR (full)	115	0.605	1.883	92.3	0.059	0.334

Table 5: Analysis of ViCoDR on CameraCtrl using RE10K. “+ViCoDR (full)” is the default setup using VGGT pseudo-labels.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

	RE10K		DL3DV
Model	MEt3R ( $\downarrow$ )	Geo. Corr. ( $\uparrow$ )	MEt3R ( $\downarrow$ )	Geo. Corr. ( $\uparrow$ )
CameraCtrl †	0.060	65.2	0.113	43.8
CameraCtrl	0.066	57.7	0.120	36.7
CameraCtrl + ViCoDR	0.059	91.0	0.110	75.0

Appendix

Table A1 : Output 3D consistency and geometric correspondence for CameraCtrl trained with and without ViCoDR.

\dagger

denotes the pretrained checkpoint released by [ 29 ] ; the other two models use matched training.

Figure A1 : User study results. Preference ratios are shown for four pairwise comparisons. CameraCtrl+ViCoDR is preferred substantially more often than each baseline for maintaining shape and structural consistency.

Model	FVD ( $\downarrow$ )	RErr ( $\downarrow$ )	TErr ( $\downarrow$ )	Succ % ( $\uparrow$ )	MEt3R ( $\downarrow$ )	RPE ( $\downarrow$ )
MotionCtrl	780 (18)	1.119 (0.057)	4.676 (0.238)	89.1 (1.7)	0.075 (0.001)	0.306 (0.008)
+ ViCoDR	559 (14)	1.217 (0.024)	5.288 (0.049)	96.4 (1.1)	0.059 (0.001)	0.255 (0.002)
UCPE	520	0.569	1.257	95.0	0.055	0.300
+ ViCoDR	539	0.509	1.139	95.8	0.049	0.266

Appendix

Table A2 : Full results of the ViCoDR intervention on camera-controlled T2V generation. Mean results over three seeds are reported for both models; standard deviations are additionally reported for MotionCtrl.

Model	mPSNR ( $\uparrow$ )	mSSIM ( $\uparrow$ )	mLPIPS ( $\downarrow$ )	FID ( $\downarrow$ )
ViFiGen	19.08	0.77	0.19	27.87
+ ViCoDR	19.78	0.81	0.13	20.35

Appendix

Table A3 : Additional results on multi-view generation. On OmniObject3D, ViCoDR outperforms ViFiGen across all four ground-truth-referenced appearance metrics.

Model	PSNR ( $\uparrow$ )	SSIM ( $\uparrow$ )	LPIPS ( $\downarrow$ )	FVD ( $\downarrow$ )	RErr ( $\downarrow$ )	TErr ( $\downarrow$ )	Succ % ( $\uparrow$ )	MEt3R ( $\downarrow$ )	RPE ( $\downarrow$ )
	RE10K
CameraCtrl	1 6.888	0 .695	0 .273	113	0.795	2.313	86.3	0 .066	0.385
+ Track4Gen	16.211	0.666	0.300	132	0.852	2.861	88.7	0.072	0.398
+ GF	16.845	0.693	0.278	117	0 .721	2 .234	9 0.0	0.067	0.381
+ JOG3R	16.103	0.670	0.296	120	0.913	2.702	86.7	0 .066	0 .375
+ ViCoDR	17.043	0.699	0.266	1 15	0.605	1.883	92.3	0.059	0.334

Appendix

Table A4 : Full comparison of alternative auxiliary objectives on CameraCtrl. This table complements the compact main-paper comparison by additionally reporting SSIM and LPIPS.

Figure A2 : Failure case 1: very large view changes. Given the input frame , ViCoDR fails to improve the 3D consistency of CameraCtrl in the highlighted region across view one and view two , which is exposed to large view changes. ViCoDR produces clearer edges.

Figure A3 : Failure case 2: clearer but distorted edges. Given the input frame , compared to CameraCtrl, ViCoDR produces clearer edges across view one and view two . However, this can appear less natural compared to the smeared edges in the CameraCtrl output.

Figure A4 : Impact of

\lambda

on RE10K performance. Increasing

\lambda

further reduces MEt3R but increases RErr and TErr, revealing a trade-off between output 3D consistency and camera-control accuracy; the default ViCoDR setting nevertheless improves both over CameraCtrl.

Figure A5 : Visual comparison of removing edge sampling . Over-sampling keypoints for $\mathcal{L}_{\text{3DC}}$ on edges during training leads to fewer artifacts on lines and corners, as can be seen from the MEt3R reprojection error map (middle column).

Model	Version / Checkpoint	External Models
MVGenMaster [ 12 ]	https://github.com/ewrfcas/MVGenMaster	DepthPro [ 8 ]
MotionCtrl [ 90 ]	https://github.com/TencentARC/MotionCtrl/tree/svd (CMCM only)	N/A
CameraCtrl [ 29 ]	https://github.com/hehao13/CameraCtrl/tree/svd	N/A
GWTF [ 11 ]	https://github.com/Eyeline-Labs/Go-with-the-Flow	MoGe [ 87 ] , RAFT [ 80 ]
DaS [ 27 ]	https://github.com/IGL-HKUST/DiffusionAsShader	MoGe [ 87 ]
GEN3C [ 66 ]	https://github.com/nv-tlabs/GEN3C	MoGe [ 87 ]

Appendix

Table A5 : Details of the pre-trained VDMs analyzed for 3D consistency. The last column specifies the depth/optical flow models used during inference of each model.

	RE10K		DL3DV
Model	RErr ( $\downarrow$ )	TErr ( $\downarrow$ )	RErr ( $\downarrow$ )	TErr ( $\downarrow$ )
CameraCtrl	0.381	1.260	0.725	1.403
+ Track4Gen	0.498	1.945	1.120	1.878
+ GF	0.385	1.322	0.811	1.634
+ JOG3R	0.538	1.786	1.080	1.761
+ ViCoDR	0.302	1.247	0.773	1.367

Appendix

Table A6 : RErr and TErr results of ViCoDR on image-to-video generation. Results are reported only on successfully reconstructed videos common to all methods.

Model	RErr ( $\downarrow$ )	TErr ( $\downarrow$ )
MotionCtrl	0.718 (0.017)	3.509 (0.121)
+ ViCoDR	0.965 (0.039)	4.917 (0.064)

Appendix

Table A7 : RErr and TErr results of ViCoDR on text-to-video generation. Results are reported only on successfully reconstructed videos common to all methods. Mean and standard deviation over three seeds are reported on RE10K.

Figure A6 : Additional results on multi-view generation. The red circles highlight regions of improved 3D consistency.

Panel of Figure 24 — Figure A7 : Additional qualitative results on camera controlled image-to-video generation. Given the input frame , ViCoDR better maintains 3D consistency across view one and view two compared to the baselines.

Panel of Figure 25 — Figure A8 : Additional qualitative results on camera controlled image-to-video generation. Given the input frame , ViCoDR better maintains 3D consistency across view one and view two compared to the baselines.

Panel of Figure 26 — Figure A9 : Qualitative results of applying ViCoDR to text-to-video generation. The input text prompt is shown above the images. Arrows highlight 3D consistent/inconsistent regions.

Camera-Controlled Video Generation

Camera-controlled video generation creates videos that follow specified viewpoints and camera paths while preserving coherent scenes and motion.

See all 142 papers

Probing into Camera Control of Video Models
May 14, 2026Chen Hou, Christian RupprechtDiffusion Model Guidance Camera-Controlled Video Generation
Visual Geometry Group · Visual Geometry Group University of Oxford
Beyond Inpainting: Unleash 3D Understanding for Stable Camera-Controlled Video Generation
Jan 15, 2026Dong-Yu Chen, Yixin Guo, Shuojin Yang +2Camera-Controlled Video Generation Video Diffusion Models
BNRist, Department of Computer Science and Technology, Tsinghua University Beijing 100084, China
Track2View: 4D-Consistent Camera-Controlled Video Generation via Paired 3D Point Tracks
Jun 14, 2026Feng Qiao, Zhaochong An, Zhexiao Xiong +2Multi-View Consistency Camera-Controlled Video Generation
Washington University in St. Louis · University of Copenhagen

May 14, 2026J/K move · Enter open · S save

Probing into Camera Control of Video Models

Chen Hou, Christian Rupprecht

Visual Geometry Group · Visual Geometry Group University of Oxford

Diffusion Model Guidance Camera-Controlled Video Generation Video Diffusion Models Multi-View Video Generation

Open paper AskPDF arXiv

Video is a rich and scalable source of 3D/4D visual observations, and camera control is a key capability for video generation models to produce geometrically meaningful content. Existing approaches typically learn a mapping from camera motion to video using additional camera modules and paired data. However, such datasets are often limited in scale, diversity, and scene dynamics, which can bias the model toward a narrow output distribution and compromise the strong prior learned by the base model. These limitations motivate a different perspective on camera control. In this paper, we show that camera control need not be modeled as an implicit mapping problem, but can instead be treated as a form of geometric guidance that induces displacements across frames. Specifically, we reformulate camera control into a set of displacement fields and apply them via differentiable resampling of latent features during denoising. Our simple approach achieves effective camera control with minimal degradation across diverse quality metrics compared to fine-tuned baselines. Since our method is applicable to most video diffusion models without training, it can also serve as a probe to study the camera control capabilities of base models. Using this probe, we identify universal biases shared by representative video models, as well as disparities in their responses to camera control. Finally, we benchmark their performance in multi-view generation, offering insights into their potential for 3D/4D tasks.

Abstract

Camera-Controlled Video Generation

Explore similar work

Probing into Camera Control of Video Models

Beyond Inpainting: Unleash 3D Understanding for Stable Camera-Controlled Video Generation

Track2View: 4D-Consistent Camera-Controlled Video Generation via Paired 3D Point Tracks