Object Pose from 2-D to 3-D Point and Line Correspondences

Thai Quynh Phong; Radu Horaud; Adnan Yassine; Pham Dinh Tao

cs.CVOct 14, 2026

Object Pose from 2-D to 3-D Point and Line Correspondences

Authors: Thai Quynh Phong, Radu Horaud, Adnan Yassine, Pham Dinh Tao

Organizations: Inria Rhˆone-Alpes 46, avenue F´elix Viallet 38031 Grenoble FRANCE · D´epartement de Math´ematiques Universit´e de Nancy I BP 239 54506 Vandoeuvre-les-Nancy FRANCE · D´epartement G´enie Math´ematique INSA – Rouen BP 08 Place Emile Blondel 76131 Mont-Saint-Aignan FRANCE

Abstract

In this paper we present a method for optimally estimating the rotation and translation between a camera and a 3-D object from point and/or line correspondences. First we devise an error function and second we show how to minimize this error function. The quadratic nature of this function is made possible by representing rotation and translation with a dual number quaternion. We provide a detailed account of the computational aspects of a trust-region optimization method. This method compares favourably with Newton's method which has extensively been used to solve the problem at hand, with Faugeras-Toscani linear method [6] for calibrating a camera, and with the Levenberg-Marquardt non-linear optimization method. Finally we present some experimental results which demonstrate the robustness of our method with respect to image noise and matching errors.

Figures & tables

Figure 1: The object line, its projection onto the image, and the center of projection

F

are coplanar and this plane is shown in grey.

\vec{n}

is the unit vector normal to this plane.

Figure 2: Configurations of 3, 4, 5, or 6 points in general position that can be replaced by 3 lines.

Figure 3: An example of minimization when several minima are present. Newton’s method may lead to an increase of the value of the function

f(x)

Figure 4: This figure shows an example of minimizing the function

f(x)

with the trust region method which always ensures that at each iteration the value of the function decreases.

Number	Transformation error			Image error			Intrinsic parameters
of points	axis	angle	translation	largest	smallest	average	$u_{0}$	$v_{0}$	$\alpha_{u}$	$\alpha_{v}$
10	0.0170	0.0032	2.4277	0.9754	0.0008	0.2150	243.1	224.3	-1677.8	1183.2
20	0.0064	0.0143	3.2806	1.0983	0.0002	0.1912	269.0	233.2	-1663.5	1173.5
40	0.0016	0.0047	0.8157	1.0935	0.0018	0.1577	235.2	238.3	-1715.7	1209.6
60	0.0009	0.0015	0.2563	0.7255	0.0007	0.1508	239.9	242.9	-1734.2	1223.2
80	0.0024	0.0038	0.6620	0.9277	0.0001	0.1500	235.9	245.5	-1741.8	1229.0

Table 1: This table shows the performance of the linear method of Faugeras and Toscani as a function of the number of point correspondences. The solution obtained with 460 points (the last row) is taken to be the ground-truth. While rotation seems to be relatively stable, translation degrades gently as the number of points decreases. Notice however the large variations of the intrinsic parameters.

Noise max.	Transformation error			Image error			Intrinsic parameters
amplitude	axis	angle	translation	largest	smallest	average	$u_{0}$	$v_{0}$	$\alpha_{u}$	$\alpha_{v}$
0.00	0.0023	0.0006	0.3333	0.5997	0.0008	0.1461	244.0	236.5	-1726.2	1217.6
0.01	0.0042	0.0035	0.6013	1.1291	0.0002	0.1574	235.2	246.8	-1738.3	1225.9
0.05	0.0014	0.0006	0.0970	0.7807	0.0011	0.1460	245.2	242.1	-1731.9	1221.9
0.10	0.0018	0.0029	0.4556	0.8290	0.0009	0.1463	239.4	238.7	-1721.9	1214.7
0.50	0.0103	0.0046	0.9617	0.7590	0.0016	0.1589	241.8	230.2	-1714.2	1208.9

Table 2: This table shows the performance of the linear method (Faugeras and Toscani) for 100 point correspondences and in the presence of image noise. The first column of this table shows the maximum noise amplitude around the nominal image point locations. Rotation resists quite nicely to noise while translation is not reliable for a maximum noise amplitude larger than half of a pixel.

Number	Transformation error			Image error			Performance
of lines	axis	angle	translation	largest	smallest	average	iterations	CPU time
5	0.0034	0.0005	0.0056	0.9421	0.0106	0.2870	10	0.12
10	0.0047	0.0008	0.0156	0.9049	0.0098	0.2127	7	0.03
20	0.0042	0.0012	0.0037	0.6071	0.0020	0.1804	6	0.02
30	0.0038	0.0004	0.0038	0.7150	0.0117	0.1775	6	0.03
40	0.0036	0.0001	0.0025	0.6534	0.0060	0.1856	7	0.05

Table 3: The performance of the rotation-then-translation implementation of our method with line correspondences. The CPU time is in seconds and was obtained on a Sun/Sparc10.

Noise max.	Transformation error			Image error			Performance
amplitude	axis	angle	translation	largest	smallest	average	iterations	CPU time
0.00	0.0034	0.0005	0.0001	0.5653	0.0121	0.1675	6	0.07
0.01	0.0039	0.0005	0.0007	0.6794	0.0105	0.1729	6	0.07
0.05	0.0038	0.0010	0.0010	0.5823	0.0126	0.1696	6	0.07
0.10	0.0039	0.0003	0.0016	0.7211	0.0025	0.1732	6	0.07
0.50	0.0034	0.0005	0.0028	0.7690	0.0129	0.2016	7	0.07

Table 4: Same as the table before but the quantities are functions of image noise.

Number	Transformation error			Image error			Performance
of lines	axis	angle	translation	largest	smallest	average	iterations	CPU time
5	0.0151	0.0316	1.7323	6.3106	0.1415	2.9870	3	0.05
10	0.0036	0.0018	0.0014	0.6187	0.0063	0.1948	18	0.13
20	0.0040	0.0021	0.0023	0.5790	0.0055	0.1830	30	0.33
30	0.0035	0.0021	0.0032	0.7203	0.0134	0.1933	23	0.35
40	0.0034	0.0007	0.0018	0.6408	0.0132	0.1881	23	0.45

Table 5: Performance of the rotation-and-translation implemetation of the trust region method for line correspondences.

Noise max.	Transformation error			Image error			Performance
amplitude	axis	angle	translation	largest	smallest	average	iterations	CPU time
0.00	0.0033	0.0008	0.0003	0.4916	0.0096	0.1706	11	0.27
0.01	0.0038	0.0000	0.0004	0.6495	0.0045	0.1720	21	0.48
0.05	0.0041	0.0008	0.0028	0.6654	0.0031	0.1768	22	0.55
0.10	0.0037	0.0007	0.0009	0.6408	0.0086	0.1690	21	0.53
0.50	0.0040	0.0030	0.0050	1.0173	0.0128	0.2381	10	0.20

Table 6: Same as the table before but the quantities are functions of image noise.

Number	Transformation error			Image error			Performance
of points	axis	angle	translation	largest	smallest	average	iterations	CPU time
10	0.0032	0.0010	0.0171	0.6706	0.0015	0.2186	24	0.32
20	0.0033	0.0012	0.0001	0.5503	0.0051	0.1780	10	0.18
40	0.0035	0.0000	0.0000	0.5747	0.0189	0.1709	9	0.37
60	0.0035	0.0002	0.0002	0.5220	0.0115	0.1665	9	0.45
80	0.0035	0.0000	0.0002	0.5201	0.0163	0.1657	9	0.55

Table 7: The performance of the rotation-and-translation implementation of the trust region method with point correspondences. This implementation seems to provide the most accurate results and the most stable ones with respect to the number of correspondences.

Noise max.	Transformation error			Image error			Performance
amplitude	axis	angle	translation	largest	smallest	average	iterations	CPU time
0.00	0.0034	0.0004	0.0012	0.4761	0.0121	0.1708	9	0.77
0.01	0.0035	0.0003	0.0008	0.5356	0.0091	0.1678	9	0.72
0.05	0.0035	0.0000	0.0001	0.5414	0.0096	0.1656	9	0.80
0.10	0.0038	0.0004	0.0004	0.5877	0.0105	0.1664	9	0.72
0.50	0.0034	0.0002	0.0001	0.5635	0.0069	0.1823	9	0.80

Table 8: Same as the table before but the quantities are functions of image noise.

Figure 5: Axis of rotation: The error between the computed direction of rotation and the “true” one, as a function of the number of correspondences (left) and as a function of image noise (right). The dashed curves correspond to the Faugeras-Toscani linear method while the full curves correspond to the rotation-then-translation implementation of our method.

Figure 6: Angle of rotation: The error between the computed angle of rotation and the “true” one, measured in radians, as a function of the number of correspondences (left) and as a function of image noise (right). The dashed curves correspond to Faugeras-Toscani while the full curves correspond to the rotation-then-translation implementation of our method.

Figure 7: Vector of translation: The error between the computed translation and the “true” one, as a function of the number of correspondences (left) and as a function of image noise (right). The dashed curves correspond to Faugeras-Toscani while the full curves correspond to the rotation-then-translation implementation of our method. The translation error is measured in centimeters. Notice the rapid degradation of the estimation of translation in the presence of noise with the linear method.

Figure 8: From 4 correct point correspondences, one may define one line mismatched pair.

Panel of Figure 9 — Figure 9: Matching errors: The errors in rotation axis (top), rotation angle (middle), and translation (bottom) as functions of the number of point mismatched pairs. The results were obtained with an initial set of 400 points among which 1, 2, … 10 point pairs were chosen at random and mismatched. The dashed curves correspond to the Faugeras-Toscani method while the full curves correspond to the rotation-then-translation implementation of the non linear method.

Object Pose Estimation

Object pose estimation recovers an object's 3D position and orientation from images, depth, point clouds, or contact sensing.

See all 187 papers

PnP*: A certifiable solver for Perspective-n-Point
Oct 22, 2026Bang-Shien Chen, Caitrin E. Eaton, David M. RosenCamera Pose Estimation Convex Relaxation
Northeastern University, Boston, MA 02115, USA. · Draper, Cambridge, MA 02139, USA.
Online Learning of Correspondences between Images
Aug 13, 2026Michael Felsberg, Fredrik Larsson, Johan Wiklund +2Multi-View Geometry Image Matching
Computer Vision Laboratory, Dept. Electrical Engineering, Linköping University, Sweden · Division of Information Systems, FOI Swedish Defence Research Agency, Sweden · Termisk Systemteknik AB, Sweden
Efficient closed-form approaches for pose estimation using Sylvester forms
Apr 16, 2026Jana Vráblíková, Ezio Malis, Laurent BuséCamera Pose Estimation Point Cloud Registration
AROMATH · Centre Inria d’Universit´e Cˆote d’Azur, France · ACENTAURI

Oct 22, 2026J/K move · Enter open · S save

PnP*: A certifiable solver for Perspective-n-Point

Bang-Shien Chen, Caitrin E. Eaton, David M. Rosen

Northeastern University, Boston, MA 02115, USA. · Draper, Cambridge, MA 02139, USA.

Camera Pose Estimation Convex Relaxation

Open paper AskPDF arXiv

Estimating a camera pose from 3D-2D correspondences - Perspective-n-Point - is one of the most fundamental problems in computer vision. Existing methods either solve a minimal problem or replace the objective with an easier one. Surprisingly, no scalable certifiable solver for the actual Maximum Likelihood Estimation (MLE) exists. We propose PnP*, the first practical solver that guarantees efficient computation and returns a certifiable globally optimal solution. We prove that the underlying convex relaxation is exact under moderate noise, which to our knowledge is the first stability guarantee for any relaxation for the MLE of PnP. Furthermore, we reduce the relaxation by exploiting chordal sparsity pattern and quotient ring reduction such that the relaxation is tractable and scalable. The stability and scalability results are verified numerically, as PnP* returns a provably correct camera pose at over 10 Hz.

Abstract

Object Pose Estimation

Explore similar work

PnP*: A certifiable solver for Perspective-n-Point

Online Learning of Correspondences between Images

Efficient closed-form approaches for pose estimation using Sylvester forms