ConfCtrl: Enabling Precise Camera Control in Video Diffusion via Confidence-Aware Interpolation
Organizations: University of Freiburg · Huawei Heisenberg Research Center (Munich) · Technical University of Munich · Ludwig Maximilian University of Munich
Abstract
We address the challenge of novel view synthesis from only two input images under large viewpoint changes. Existing regression-based methods lack the capacity to reconstruct unseen regions, while camera-guided diffusion models often deviate from intended trajectories due to noisy point cloud projections or insufficient conditioning from camera poses. To address these issues, we propose ConfCtrl, a confidence-aware video interpolation framework that enables diffusion models to follow prescribed camera poses while completing unseen regions. ConfCtrl initializes the diffusion process by combining a confidence-weighted projected point cloud latent with noise as the conditioning input. It then applies a Kalman-inspired predict-update mechanism, treating the projected point cloud as a noisy measurement and using learned residual corrections to balance pose-driven predictions with noisy geometric observations. This allows the model to rely on reliable projections while down-weighting uncertain regions, yielding stable, geometry-aware generation. Experiments on multiple datasets show that ConfCtrl produces geometrically consistent and visually plausible novel views, effectively reconstructing occluded regions under large viewpoint changes.
Figures & tables
| Method | CO3D-Hydrant CO3D-Teddybear | CO3D-Hydrant DL3DV | DL3DV GraspNet | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | LPIPS | PSNR | SSIM | |||||||
| NopoSplat [ 68 ] | 0.656 | 14.13 | 0.293 | 1.125 | 0.949 | 0.687 | 10.01 | 0.172 | 1.099 | 0.878 | 0.659 | 10.87 | 0.242 | 0.222 | 0.157 |
| DepthSplat [ 66 ] | 0.675 | 12.48 | 0.290 | 1.218 | 1.032 | 0.659 | 10.65 | 0.201 | 1.079 | 0.854 | 0.621 | 10.02 | 0.189 | 0.202 | 0.143 |
| MvSplat [ 8 ] | 0.724 | 7.74 | 0.119 | 1.445 | 1.214 | 0.715 | 10.26 | 0.202 | 1.120 | 0.876 | 0.670 | 10.48 | 0.257 | 0.357 | 0.255 |
| PixelSplat [ 5 ] | 0.641 | 13.93 | 0.402 | 0.691 | 0.550 | 0.643 | 12.82 | 0.293 | 0.701 | 0.529 | 0.576 | 14.42 | 0.405 | 0.219 | 0.156 |
| CameraCtrl [ 18 ] | 0.460 | 15.24 | 0.392 | 0.560 | 0.433 | 0.408 | 14.21 | 0.311 | 0.392 | 0.309 | 0.490 | 11.77 | 0.369 | 0.220 | 0.156 |
| LPIPS | PSNR | SSIM | |||
| (a) w/o noise init | 0.347 | 15.44 | 0.326 | 0.174 | 0.128 |
| (b) w/o conf | 0.366 | 14.75 | 0.296 | 0.164 | 0.117 |
| (c) w/o update | 0.386 | 14.65 | 0.305 | 0.194 | 0.139 |
| (d) w/o both | 0.362 | 14.85 | 0.304 | 0.213 | 0.155 |
| (e) w/o | 0.339 | 15.49 | 0.322 | 0.152 | 0.110 |
| (f) camera pose only | 0.370 | 14.90 | 0.312 | 0.192 | 0.141 |
Camera-Controlled Video Generation
Camera-controlled video generation creates videos that follow specified viewpoints and camera paths while preserving coherent scenes and motion.
See all 141 papers