Perspective-1-Ellipsoid: Formulation, Analysis and Solutions of the Camera Pose Estimation Problem from One Ellipse-Ellipsoid Correspondence
Authors: Vincent Gaudillière, Gilles Simon, Marie-Odile Berger
Organizations: Université de Lorraine, CNRS, Inria, LORIA, F-54000, Nancy, France. · SnT - Interdisciplinary Centre for Security, Reliability and Trust, University of Luxembourg, 29 Avenue John F. Kennedy, L-1855, Luxembourg, Luxembourg.
In computer vision, camera pose estimation from correspondences between 3D geometric entities and their projections into the image has been a widely investigated problem. Although most state-of-the-art methods exploit low-level primitives such as points or lines, the emergence of very effective CNN-based object detectors in the recent years has paved the way to the use of higher-level features carrying semantically meaningful information. Pioneering works in that direction have shown that modelling 3D objects by ellipsoids and 2D detections by ellipses offers a convenient manner to link 2D and 3D data. However, the mathematical formalism most often used in the related litterature does not enable to easily distinguish ellipsoids and ellipses from other quadrics and conics, leading to a loss of specificity potentially detrimental in some developments. Moreover, the linearization process of the projection equation creates an over-representation of the camera parameters, also possibly causing an efficiency loss. In this paper, we therefore introduce an ellipsoid-specific theoretical framework and demonstrate its beneficial properties in the context of pose estimation. More precisely, we first show that the proposed formalism enables to reduce the pose estimation problem to a position or orientation-only estimation problem in which the remaining unknowns can be derived in closed-form. Then, we demonstrate that it can be further reduced to a 1 Degree-of-Freedom (1DoF) problem and provide the analytical derivations of the pose as a function of that unique scalar unknown. We illustrate our theoretical considerations by visual examples and include a discussion on the practical aspects. Finally, we release this paper along with the corresponding source code in order to contribute towards more efficient resolutions of ellipsoid-related pose estimation problems.
Figures & tables
Figure 1: The Perspective-1-Ellipsoid (P1E) problem aims at estimating the locus of solutions (c) to the camera pose estimation problem from one ellipse-ellipsoid correspondence (a,b).
Figure 2: Illustrating the image plane, camera center, ellipsoid and projected ellipse.
Figure 3: Camera relocalization based on the decoupling between orientation and position, here applied to images from the RGB-D TUM dataset ( Gaudillière et al., 2019b ) . The first row shows the detected boxes, the inscribed ellipsoids (in yellow) and the outlines of the reprojected ellipsoids (in green), with the automatically generated labels. The blue ellipses correspond to the reprojected ellipsoids when the QuadricSLAM residual error is minimized ( Nicholson et al., 2019 ) . The second row shows the reprojected ellipsoids (in green the inliers, in white the outliers and undetected ellipsoids) when using the method in ( Gaudillière et al., 2019b ) .
Ellipsoid type
Triaxial
Spheroid
Sphere
Projection cone type
Non-circular
✓
✓
×
(Algo. 5 , 6 )
(Algo. 6 )
Circular
×
✓
✓
(Algo. 7 )
(Algo. 8 )
Table 1: Possible co-occurences of ellipsoids and projection cones according to their types. ✓ indicates that ellipsoid and cone of the corresponding types may occur simultaneously. × indicates that they cannot.
Figure 4: Loci of the centers (black) and principal axes endpoints (red, green, blue) of the ellipsoids solutions. The camera center E appears along with the image plane (grey) and ellipse (orange). A video is available 2 2 footnotemark: 2 .
Figure 5: The two spheroids solutions with a non-circular backprojection cone.
Figure 6: Triaxial ellipsoid: locus of cone vertices i.e. camera centers.
Figure 7: Spheroid: locus of cone vertices i.e. camera centers.
Figure 8: Illustrating the eight backprojection cones tangent to the triaxial ellipsoid for a given σ value.
Figure 9: Illustrating the two cameras compatible with each backprojection cone tangent to the triaxial ellipsoid.
λB′,2<0
λB′,2>0
λA,single<λA,double
#1
#2
λA,single>λA,double
#3
#4
Table 2: Configurations of the problem depending on A and B′ eigenvalues.
Figure 10: The spheroid solution with a circular backprojection cone.
Figure 11: Examples of camera loci (i.e. solutions for position) corresponding to six different ellipse-ellipsoid pairs. Corresponding 2D ellipse (top), 3D ellipsoid (bottom) and 3D camera locus (bottom) use the same color. The groundtruth camera position E is at the unique intersection of the six loci. The image is extracted from the T-LESS dataset ( Sturm et al., 2012 ) .
This paper presents a new algorithm to estimate absolute camera pose given an axis of the camera's rotation matrix. Current algorithms solve the problem via algebraic solutions on limited input domains. This paper shows that the problem can be solved efficiently by finding the intersection points of a hyperbola and the unit circle. The solution can flexibly accommodate combinations of point and line features in minimal and overconstrained configurations. In addition, the two special cases of planar and minimal configurations are identified to yield simpler closed-form solutions. Extensive experiments validate the approach.
Estimating a camera pose from 3D-2D correspondences - Perspective-n-Point - is one of the most fundamental problems in computer vision. Existing methods either solve a minimal problem or replace the objective with an easier one. Surprisingly, no scalable certifiable solver for the actual Maximum Likelihood Estimation (MLE) exists. We propose PnP*, the first practical solver that guarantees efficient computation and returns a certifiable globally optimal solution. We prove that the underlying convex relaxation is exact under moderate noise, which to our knowledge is the first stability guarantee for any relaxation for the MLE of PnP. Furthermore, we reduce the relaxation by exploiting chordal sparsity pattern and quotient ring reduction such that the relaxation is tractable and scalable. The stability and scalability results are verified numerically, as PnP* returns a provably correct camera pose at over 10 Hz.
Bang-Shien Chen, Caitrin E. Eaton, David M. Rosen
Northeastern University, Boston, MA 02115, USA. · Draper, Cambridge, MA 02139, USA.
Solving non-linear least-squares problem for pose estimation (rotation and translation) is often a time consuming yet fundamental problem in several real-time computer vision applications. With an adequate rotation parametrization, the optimization problem can be reduced to the solution of a~system of polynomial equations and solved in closed form. Recent advances in efficient closed form solvers utilizing resultant matrices have shown a promising research direction to decrease the computation time while preserving the estimation accuracy. In this paper, we propose a new class of resultant-based solvers that exploit Sylvester forms to further reduce the complexity of the resolution. We demonstrate that our proposed methods are numerically as accurate as the state-of-the-art solvers, and outperform them in terms of computational time. We show that this approach can be applied for pose estimation in two different types of problems: estimating a pose from 3D to 3D correspondences, and estimating a pose from 3D points to 2D points correspondences.
Jana Vráblíková, Ezio Malis, Laurent Busé
AROMATH · Centre Inria d’Universit´e Cˆote d’Azur, France · ACENTAURI