QuadSync: Quadrifocal Tensor Synchronization via Tucker Decomposition
Authors: Daniel Miao, Gilad Lerman, Joe Kileel
Organizations: School of Mathematics, University of Minnesota · Department of Mathematics and Oden Institute for Computational Engineering and Sciences, University of Texas at Austin
In structure from motion, quadrifocal tensors capture more information than their pairwise counterparts (essential matrices), yet they have often been thought of as impractical and only of theoretical interest. In this work, we challenge such beliefs by providing a new framework to recover n cameras from the corresponding collection of quadrifocal tensors. We form the block quadrifocal tensor and show that it admits a Tucker decomposition whose factor matrices are the stacked camera matrices, and which thus has a multilinear rank of (4,~4,~4,~4) independent of n. We develop the first synchronization algorithm for quadrifocal tensors, using Tucker decomposition, alternating direction method of multipliers, and iteratively reweighted least squares. We further establish relationships between the block quadrifocal, trifocal, and bifocal tensors, and introduce an algorithm that jointly synchronizes these three entities. Numerical experiments demonstrate the effectiveness of our methods on modern datasets, indicating the potential and importance of using higher-order information in synchronization.
Figures & tables
Figure 1: Mean location error for ETH3D datasets
Figure 2: Mean location error for EPFL datasets
Figure 3: QuadSync retrieved camera poses on near-collinear views from plant_scene_1 dataset from ETH3D SLAM.
Figure 4: Randomized updates in QuadSync tested on ETH3D ‘relief’ dataset.
Figure 5: Ground truth location of 10 cameras. ×
perc. (%)
noise(%)
etˉ
et^
erˉ
er^
100
0.00
0.00
0.00
0.00
0.00
100
1
0.04
0.04
0.23
0.18
100
5
0.24
0.22
2.67
2.52
80
0
0.00
0.00
0.00
0.00
80
1
0.04
0.03
0.37
0.37
80
5
0.36
0.36
3.07
2.68
Table 1: Results for synthetic experiments with collinear cameras. etˉ : mean location error. et^ : median location error. erˉ : mean rotation error. er^ : median rotation error.
Figure 6: Retrieved poses (colored, where each cluster has a distinct color) vs. ground truth poses (black) for CastleP30 with handpicked clusters.
size
noise
etˉ
et^
erˉ
er^
time(s)
Full (30 cams)
0
0
0
0
0
1666.3
Cluster1 (10 cams)
0
0
0
0
0
27.3
Cluster2 (15 cams)
0
0
0
0
0
92.6
Cluster3 (15 cams)
0
0
0
0
0
97.7
Aligned (30 cams)
0
0
0
0
0
150.2
Full (30 cams)
1
0.11
0.10
0.08
0.09
1944.8
Table 2: Results for synthetic distributed synchronization. etˉ : mean location error. et^ : median location error. erˉ : mean rotation error. er^ : median rotation error. Noise is added as percentage onto the cameras. In the noisy cases, only the quadrifocal tensors were synchronized.
Dataset
Completion %
Joint Opt. (s)
TrifocalSync (s)
courtyard (18/38)
33.10
111.2
137.9
electro (12/39)
78.70
30.6
45.13
facade (46/76)
61.98
4226.2
1633.0
kicker (20/30)
31.84
157.3
174.3
meadow (6/14)
42.13
11.7
9.1
office (13/21)
39.92
34.2
53.3
Table 3: Completion rates and runtimes of different methods