Recently, DeMenthon and Davis [4,5] proposed a method for determining the pose of a 3-D object with respect to a camera from 3-D to 2-D point correspondences. The method consists of iteratively improving the pose computed with a weak perspective camera model to converge, at the limit, to a pose estimation computed with a perspective camera model. In this paper we give an algebraic derivation of DeMenthon and Davis' method and we show that it belongs to a larger class of methods where the perspective camera model is approximated either at zero order (weak perspective) or first order (paraperspective). We describe in detail an iterative paraperspective pose computation method for both non coplanar and coplanar object points. We analyse the convergence of these methods and we conclude that the iterative paraperspective method (proposed in this paper) has better convergence properties than the iterative weak perspective method. We introduce a simple way of taking into account the orthogonality constraint associated with the rotation matrix. We analyse the sensitivity to camera calibration errors and we define the optimal experimental setup with respect to imprecise camera calibration. We compare the results obtained with this method and with a non-linear optimization method.
Figures & tables
Figure 1: This figure shows the general setup. One of the object’s point is selected as the origin of the object frame. Therefore, the pose parameters are the coordinates of this object point ( P0 ) in the camera frame and the orientation of the object frame with respect to the camera frame.
Figure 2: This figure shows p0 , pi – the perspective projections of two object points, P0 and Pi , and piw , pip – the weak and paraperspective projections of Pi . The quality of the paraperspective projection depends on the angle between the line of projection of Pi and the line of projection of P0 . Similarly, the quality of the weak perspective projection depends on the angle between the line of projection of Pi and the optical axis.
Figure 3: A coplanar configuration of object points.
Figure 4: A planar object and a paraperspective camera model produces two orientations. However, these two object orientations have distinct perspective projections – pi and pi′ .
Figure 5: A binary tree search for selecting the best object pose when the object points are coplanar.
Figure 6: Error in orientation as a function of depth in the presence of image gaussian noise. The solid line corresponds to the iterative paraperspective algorithm, the dotted line corresponds to the iterative paraperspective algorithm combined with the orthogonal constraint, and the dashed line corresponds to a non linear algorithm.
Figure 7: Error in position as a function of depth in the presence of image gaussian noise. The iterative weak and paraperspective algorithms have the same behaviour (solid line) while the non linear algorithm (dashed line) performs slightly better.
Figure 8: Error in orientation as a function of depth in the presence of camera uniform noise. The behaviour of the iterative paraperspective algorithm (solid line) is compared with a non linear algorithm (dashed line).
Figure 9: Error in orientation as a function of depth in the presence of camera and image uniform noise. The iterative paraperspective algorithm (solid line) has a behaviour that differs from the non-linear algorithm (dashed line).
Figure 10: Speed of convergence as a function of depth, the offset is equal to 23o (triangles: paraperspective, squares: weak perspective).
Figure 11: Speed of convergence as a function of depth, the offset is equal to 30o (triangles: paraperspective, squares: weak perspective).
Figure 12: Rate of convergence as a function of depth, the offset is equal to 30o (triangles: paraperspective, squares: weak perspective).
Figure 13: Rate of convergence as a function of depth, the offset is equal to 35o (triangles: paraperspective, squares: weak perspective).
Figure 14: An example of applying the iterative paraperspective algorithm to a non coplanar set of point correspondences.
Figure 15: An example of applying the iterative paraperspective algorithm to a set of 4 coplanar point correspondences.
Figure 16: The four image dots correspond to a coplanar set of object points. Because the object is farther away from the optical axis, the iterative paraperspective method performs better in this case than the iterative weak perspective method.
Method
Number of iterations
CPU time (ms)
It. weak persp.
10
4.3
It. parapersp.
5
2.9
Non linear
91
65.3
Table 1: A comparison of the performance of the three algorithms. The computer being used is a Sun Sparc10/51.
Figure 17: It takes 1.47 miliseconds on a Sun-Sparc10/51 to compute the pose in this case with the iterative paraperspective algorithm.
Iterative paraperspective
Non linear
Number of points
N. of iterations
CPU time (ms)
N. of iterations
CPU time (ms)
4
5
0.87
114
87.5
5
5
0.95
67
58.7
6
6
1.02
66
68.0
7
5
1.06
66
72.5
8
6
1.12
58
70.5
Table 2: A comparison of the iterative paraperspective and non linear methods as a function of the number of points. One may notice the large number of iterations required by the non linear method for 4 point correspondences.
In this paper we present a method for optimally estimating the rotation and translation between a camera and a 3-D object from point and/or line correspondences. First we devise an error function and second we show how to minimize this error function. The quadratic nature of this function is made possible by representing rotation and translation with a dual number quaternion. We provide a detailed account of the computational aspects of a trust-region optimization method. This method compares favourably with Newton's method which has extensively been used to solve the problem at hand, with Faugeras-Toscani linear method [6] for calibrating a camera, and with the Levenberg-Marquardt non-linear optimization method. Finally we present some experimental results which demonstrate the robustness of our method with respect to image noise and matching errors.
Thai Quynh Phong, Radu Horaud, Adnan Yassine +1
Inria Rhˆone-Alpes 46, avenue F´elix Viallet 38031 Grenoble FRANCE · D´epartement de Math´ematiques Universit´e de Nancy I BP 239 54506 Vandoeuvre-les-Nancy FRANCE · D´epartement G´enie Math´ematique INSA – Rouen BP 08 Place Emile Blondel 76131 Mont-Saint-Aignan FRANCE
Estimating a camera pose from 3D-2D correspondences - Perspective-n-Point - is one of the most fundamental problems in computer vision. Existing methods either solve a minimal problem or replace the objective with an easier one. Surprisingly, no scalable certifiable solver for the actual Maximum Likelihood Estimation (MLE) exists. We propose PnP*, the first practical solver that guarantees efficient computation and returns a certifiable globally optimal solution. We prove that the underlying convex relaxation is exact under moderate noise, which to our knowledge is the first stability guarantee for any relaxation for the MLE of PnP. Furthermore, we reduce the relaxation by exploiting chordal sparsity pattern and quotient ring reduction such that the relaxation is tractable and scalable. The stability and scalability results are verified numerically, as PnP* returns a provably correct camera pose at over 10 Hz.
Bang-Shien Chen, Caitrin E. Eaton, David M. Rosen
Northeastern University, Boston, MA 02115, USA. · Draper, Cambridge, MA 02139, USA.
The blind perspective-n-point (PnP) problem is to estimate the pose of a calibrated camera from 2D and 3D point sets with unknown correspondences. We show that, for complete, noiseless observations, fourteen permutation-invariant image moments up to order four identify a fixed camera pose for almost every 3D point set containing at least twenty points. Our learning-free method, MomentPnP, matches these moments between the observed 2D points and the projected 3D points to initialize a camera pose without correspondences and refines the pose and correspondence weights using expectation--maximization. Across the test sets of three benchmarks, our method reduces median rotation error by 5--170× relative to the best learned baseline, both with and without outliers. We recover accurate poses even with as many outliers as inliers. On dense ETH3D scans, pose accuracy improves as the point sets grow, even as individual correspondences become harder to resolve.
Subhransu S. Bhattacharjee, Dylan Campbell, Rahul Shome
School of Computing, The Australian National University, Canberra