Multi-camera dynamic Augmented Reality (AR) applications require a camera pose estimation to leverage individual information from each camera in one common system. This can be achieved by combining contextual information, such as markers or objects, across multiple views. While commonly cameras are calibrated in an initial step or updated through the constant use of markers, another option is to leverage information already present in the scene, like known objects. Another downside of marker-based tracking is that markers have to be tracked inside the field-of-view (FoV) of the cameras. To overcome these limitations, we propose a constant dynamic camera pose estimation leveraging spatiotemporal FoV overlaps of known objects on the fly. To achieve that, we enhance the state-of-the-art object pose estimator to update our spatiotemporal scene graph, enabling a relation even among non-overlapping FoV cameras. To evaluate our approach, we introduce a multi-camera, multi-object pose estimation dataset with temporal FoV overlap, including static and dynamic cameras. Furthermore, in FoV overlapping scenarios, we outperform the state-of-the-art on the widely used YCB-V and T-LESS dataset in camera pose accuracy. Our performance on both previous and our proposed datasets validates the effectiveness of our marker-less approach for AR applications. The code and dataset are available on https://github.com/roth-hex-lab/IEEE-VR-2026-MultiCam.
Figures & tables
Table 1: Dataset Overview. Comparison of existing instance-level 6D pose datasets with our proposed dataset in terms of camera setup, field-of-view (FOV) overlap, and object complexity.
Figure 1: Architecture of our proposed graph-based pose estimation. We generate object candidates from a set of captured RGB images using our 6D pose estimation module. Based on object candidates from different camera views, we select the matching candidates and calculate camera poses. Finally, we implement object and camera optimization using our novel spatiotemporal scene graph. We build our spatiotemporal scene graph using object information from different spaces and times. The icons represent camera nodes, i.e., hmd , camera C1 , and camera Cn . The circles represent objects. If there are overlaps between two cameras, we calculate camera poses based on the hmd SLAM coordinate system.
Figure 2: Objects included in our Femoral Nailing dataset.
Approach
One-stage
Two-stage
Method
PoseCNN
SegDriven
YOLOPose
GDR-Net
[ 60 ]
[ 17 ]
( Ours )
[ 55 ]
P.E.
1
1
1
1
N
002_master_chef_can
3.6
33.0
70.2
51.7
41.5
003_cracker_box
25.1
44.6
85.4
45.1
83.2
004_sugar_box
40.3
75.6
73.8
83.9
91.5
Table 2: Single-view object pose evaluation on YCB-V dataset [ 60 ] with ADD(-S)-0.1d. P.E. means whether the method is trained with 1 pose estimator for the whole dataset or 1 per object ( N objects in total). ( ∗ ) denotes symmetric objects and ( − ) denotes unavailable results.
Figure 3: Multi-view visualization of object poses in YCB-V dataset: Our proposed approach, MultiCam, (bottom) is able to detect and optimize object poses in a more accurate way compared to the state-of-the-art CosyPose (center) [ 24 ] in multi-view setups. The ground truth is displayed on top. Differences are notable on the clamp right next to the can, and on the other clamp, a drift in different views becomes apparent.
Figure 4: Multi-view visualization from our Femoral Nailing dataset in near distance (top two rows) and far distance (bottom two rows): We compared YOLOPose + CosyPose [ 24 ] (second left), CNOS [ 32 ] + Megapose [ 25 ] + CosyPose [ 24 ] (second right), and our approach (MultiCam) (right) with the ground truth (left). Our approach has smaller visualization errors than the other approaches. MultiCam also integrates more objects using spatiotemporal information, leveraging shared and non-shared fov information. Calibration markers presented in the scene are used to compare with our proposed marker-less approach.
Table 3: Quantitative multi-view 6D object pose results on YCB-V dataset [ 60 ] . The best results per category (known vs. unknown camera poses) are printed in bold .
Table 4: Evaluation of camera pose estimation error before and after object-level bundle adjustment on the YCB-V dataset [ 60 ] : We report pose rotation error (in mm) and translational error(in degrees) for the camera poses before and after bundle adjustment. Our proposed bundle adjustment helps improve both object and camera accuracy.
Table 5: Effect of the 3D loss in YCB-V dataset [ 60 ] . The proposed 3D loss outperforms the 2D key point loss. The larger the distance between the object and the camera is, the more obvious the advantage of the 3D loss.
Methods
4 views
8 views
etrans↓
erot↓
etrans↓
erot↓
CosyPose [ 24 ]
44.38
4.19
65.93
8.11
ARToolKitPlus marker [ 52 ]
64.27
14.01
61.97
13.48
MultiCam (ours)
38.22
3.25
38.96
3.33
Table 6: Evaluation of camera pose estimation error on the T-LESS dataset [ 15 ] . We report pose rotation error (in mm) and translational error(in degrees) for the camera poses with state-of-the-art calibration and pose estimation methods.
Methods
Object-level
Calibration board
Conditions
YOLOpose + CosyPose [ 24 ]
CNOS [ 32 ] + Megapose [ 25 ] + CosyPose [ 24 ]
MultiCam (ours)
Charuco marker
etrans↓
erot↓
runtime
etrans↓
erot↓
runtime
etrans↓
erot↓
runtime
etrans↓
erot↓
runtime
Near
64.54
9.81
135.05
47.17
6.84
84831.93
45.54
6.48
43.37
34.98
3.40
21.72
Far
94.69
8.90
194.70
70.47
6.91
79163.96
52.79
5.53
47.65
81.68
6.76
18.56
Mean
79.62
9.36
164.9
58.8
6.88
81997.95
49.17
6.01
45.51
58.33
5.08
20.14
Table 7: Camera pose estimation/Calibration results on our Femoral Nailing dataset: The translation error is in millimeters (mm) ( eave_trans denoted as etrans , ↓ ), the rotation error ( eave_rot denoted as erot , ↓ ) is in degrees and the average runtime is in ms. The best results among all methods are labeled in bold.
Methods
YOLOpose + CosyPose [ 24 ]
MultiCam (ours)
ADD(S)AUC↑
etrans↓
erot↓
ADD(S)AUC↑
etrans↓
erot↓
Aiming arm
0.00
158.62
20.53
30.90
85.57
19.00
Bar ⋆
0.00
244.59
-
37.51
109.23
-
Gear ⋆
33.33
156.18
-
47.65
127.31
-
Insertion handle
0.00
215.67
13.01
56.29
46.62
9.13
Nail
0.00
261.23
140.67
46.59
75.46
83.61
Table 8: Multiview object pose evaluation on Femoral Nailing ADD(-S)-AUC, translation error in mm and rotation error in degrees with near and far conditions. Symmetric objects are denoted by ⋆ . Rotational errors are only evaluated for unsymmetrical objects.
Figure 5: Drift error correction with MultiCam: The figure shows the increasing sensor drift (orange) which can be reduced when on-the-fly camera pose estimation is performed (blue).
We address the challenging problem of dense dynamic scene reconstruction and camera pose estimation from multiple freely moving cameras -- a setting that arises naturally when multiple observers capture a shared event. Prior approaches either handle only single-camera input or require rigidly mounted, pre-calibrated camera rigs, limiting their practical applicability. We propose a two-stage optimization framework that decouples the task into robust camera tracking and dense depth refinement. In the first stage, we extend single-camera visual SLAM to the multi-camera setting by constructing a spatiotemporal connection graph that exploits both intra-camera temporal continuity and inter-camera spatial overlap, enabling consistent scale and robust tracking. To ensure robustness under limited overlap, we introduce a wide-baseline initialization strategy using feed-forward reconstruction models. In the second stage, we refine depth and camera poses by optimizing dense inter- and intra-camera consistency using wide-baseline optical flow. Additionally, we introduce MultiCamRobolab, a new real-world dataset with ground-truth poses from a motion capture system. Finally, we demonstrate that our method significantly outperforms state-of-the-art feed-forward models on both synthetic and real-world benchmarks, while requiring less memory.
Shuo Sun, Unal Artan, Malcolm Mielle +2
Örebro University, Sweden · Schindler EPFL Lab, Lausanne, Switzerland · Technical University of Munich, Germany
This paper proposes a fast and online method for jointly performing 3D multi-object tracking and pose estimation using multiple monocular cameras. Our algorithm requires only 2D bounding box and pose detections, eliminating the need for costly 3D training data or computationally expensive deep learning models. Our solution is an efficient implementation of a Bayes-optimal multi-object tracking filter, enhancing computational efficiency while maintaining accuracy. We demonstrate that our algorithm is significantly faster than state-of-the-art methods without compromising accuracy, using only publicly available pre-trained 2D detection models. We also illustrate the robust performance of our algorithm in scenarios where multiple cameras are intermittently disconnected or reconnected during operation.
Linh Van Ma, Tran Thien Dat Nguyen, Juhua Hu +2
School of Electrical Engineering and Computer Science at GIST, Gwangju, Korea · School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Australia · School of Engineering and Technology, University of Washington, USA
Camera pose estimation from multi-view images remains a challenge in computer vision. Traditional methods often address this problem using Structure-from-Motion (SfM) with bundle adjustment. However, camera poses estimated from sparse views are inherently ambiguous due to insufficient geometric constraints. Recent work leverages probabilistic models, such as diffusion models, to generate multiple camera pose hypotheses and therefore capture this uncertainty better. Most of these methods represent camera rotations using unit quaternions, but treat them as unconstrained 4D vectors during the generative processes, thereby ignoring the unit-norm constraint of quaternions. Unconstrained quaternions create non-smooth and suboptimal generation trajectories. To this end, we propose QuacamFM, a quaternion-constrained flow matching framework for camera pose estimation that preserves unit quaternion representations throughout the entire flow trajectory. We design the optimal transport of the quaternion flows using smooth spherical linear interpolation. Experiments on CO3Dv2 demonstrate our method's advantage in camera pose accuracy over diffusion-based methods and classical SfM approaches. We further show that our quaternion-constrained formulation outperforms the naive application of standard flow matching to 4D quaternion vectors on sparse-view camera pose estimation. Finally, it is observed that QuacamFM generalizes well across datasets and in-the-wild examples.