QuadSync: Quadrifocal Tensor Synchronization via Tucker Decomposition
Authors: Daniel Miao, Gilad Lerman, Joe Kileel
Organizations: School of Mathematics, University of Minnesota · Department of Mathematics and Oden Institute for Computational Engineering and Sciences, University of Texas at Austin
In structure from motion, quadrifocal tensors capture more information than their pairwise counterparts (essential matrices), yet they have often been thought of as impractical and only of theoretical interest. In this work, we challenge such beliefs by providing a new framework to recover n cameras from the corresponding collection of quadrifocal tensors. We form the block quadrifocal tensor and show that it admits a Tucker decomposition whose factor matrices are the stacked camera matrices, and which thus has a multilinear rank of (4,~4,~4,~4) independent of n. We develop the first synchronization algorithm for quadrifocal tensors, using Tucker decomposition, alternating direction method of multipliers, and iteratively reweighted least squares. We further establish relationships between the block quadrifocal, trifocal, and bifocal tensors, and introduce an algorithm that jointly synchronizes these three entities. Numerical experiments demonstrate the effectiveness of our methods on modern datasets, indicating the potential and importance of using higher-order information in synchronization.
Figures & tables
Figure 1: Mean location error for ETH3D datasets
Figure 2: Mean location error for EPFL datasets
Figure 3: QuadSync retrieved camera poses on near-collinear views from plant_scene_1 dataset from ETH3D SLAM.
Figure 4: Randomized updates in QuadSync tested on ETH3D ‘relief’ dataset.
Figure 5: Ground truth location of 10 cameras. ×
perc. (%)
noise(%)
etˉ
et^
erˉ
er^
100
0.00
0.00
0.00
0.00
0.00
100
1
0.04
0.04
0.23
0.18
100
5
0.24
0.22
2.67
2.52
80
0
0.00
0.00
0.00
0.00
80
1
0.04
0.03
0.37
0.37
80
5
0.36
0.36
3.07
2.68
Table 1: Results for synthetic experiments with collinear cameras. etˉ : mean location error. et^ : median location error. erˉ : mean rotation error. er^ : median rotation error.
Figure 6: Retrieved poses (colored, where each cluster has a distinct color) vs. ground truth poses (black) for CastleP30 with handpicked clusters.
size
noise
etˉ
et^
erˉ
er^
time(s)
Full (30 cams)
0
0
0
0
0
1666.3
Cluster1 (10 cams)
0
0
0
0
0
27.3
Cluster2 (15 cams)
0
0
0
0
0
92.6
Cluster3 (15 cams)
0
0
0
0
0
97.7
Aligned (30 cams)
0
0
0
0
0
150.2
Full (30 cams)
1
0.11
0.10
0.08
0.09
1944.8
Table 2: Results for synthetic distributed synchronization. etˉ : mean location error. et^ : median location error. erˉ : mean rotation error. er^ : median rotation error. Noise is added as percentage onto the cameras. In the noisy cases, only the quadrifocal tensors were synchronized.
Dataset
Completion %
Joint Opt. (s)
TrifocalSync (s)
courtyard (18/38)
33.10
111.2
137.9
electro (12/39)
78.70
30.6
45.13
facade (46/76)
61.98
4226.2
1633.0
kicker (20/30)
31.84
157.3
174.3
meadow (6/14)
42.13
11.7
9.1
office (13/21)
39.92
34.2
53.3
Table 3: Completion rates and runtimes of different methods
We present SyncFix, a framework that enforces cross-view consistency during the diffusion-based refinement of reconstructed scenes. SyncFix formulates refinement as a joint latent bridge matching problem, synchronizing distorted and clean representations across multiple views to fix the semantic and geometric inconsistencies. This means SyncFix learns a joint conditional over multiple views to enforce consistency throughout the denoising trajectory. Our training is done only on image pairs, but it generalizes naturally to an arbitrary number of views during inference. Moreover, reconstruction quality improves with additional views, with diminishing returns at higher view counts. Qualitative and quantitative results demonstrate that SyncFix consistently generates high-quality reconstructions and surpasses current state-of-the-art baselines, even in the absence of clean reference images. SyncFix achieves even higher fidelity when sparse references are available.
Pairwise translation directions are a key input to camera location estimation in global structure-from-motion. Existing estimators usually process each image pair independently, producing directions that may be locally plausible but inconsistent with the other relative directions in the viewing graph. To jointly estimate the direction, we propose TriDE, which exploits camera-triangle consistency as an efficient higher-order verification signal. Instead of solving a costly global nonlinear optimization problem that is sensitive to initialization, TriDE refines unreliable pairwise directions through message passing between directions and their incident weighted triangles. This information propagation strategy enables us to establish a strong phase-transition bound for exact recovery under a realistic random corruption model. Experiments on real image graphs show that TriDE improves direction accuracy by a large margin and yields better downstream camera locations, providing a practical link between local pairwise estimation and global camera pose geometry.
Francisco Chen, Yiran Wang, Yunpeng Shi
Department of Mathematics University of California, Davis
Camera pose estimation from multi-view images remains a challenge in computer vision. Traditional methods often address this problem using Structure-from-Motion (SfM) with bundle adjustment. However, camera poses estimated from sparse views are inherently ambiguous due to insufficient geometric constraints. Recent work leverages probabilistic models, such as diffusion models, to generate multiple camera pose hypotheses and therefore capture this uncertainty better. Most of these methods represent camera rotations using unit quaternions, but treat them as unconstrained 4D vectors during the generative processes, thereby ignoring the unit-norm constraint of quaternions. Unconstrained quaternions create non-smooth and suboptimal generation trajectories. To this end, we propose QuacamFM, a quaternion-constrained flow matching framework for camera pose estimation that preserves unit quaternion representations throughout the entire flow trajectory. We design the optimal transport of the quaternion flows using smooth spherical linear interpolation. Experiments on CO3Dv2 demonstrate our method's advantage in camera pose accuracy over diffusion-based methods and classical SfM approaches. We further show that our quaternion-constrained formulation outperforms the naive application of standard flow matching to 4D quaternion vectors on sparse-view camera pose estimation. Finally, it is observed that QuacamFM generalizes well across datasets and in-the-wild examples.