Singularity Analysis for the Perspective-Four and Five-Line Problems
Authors: Jorge García Fontán, Abhilash Nayak, Sébastien Briot, Mohab Safey El Din
Organizations: Sorbonne Universit´e, LIP6, Equipe PolSys, Paris, France · Centre national de la Recherche Scientifique (CNRS), Laboratoire des Sciences du Num´erique de Nantes (LS2N), UMR CNRS 6004, Nantes, France
This paper deals with image-based visual servoing and pose estimation by observing four and five lines. Our main interest is to determine the relative configurations of the camera and the observed lines that lead to problems in control and stability. Since it is equivalent to finding the singularities of the corresponding Jacobian matrix, we use tools from computational algebraic geometry to seek configurations such that all of its minors vanish simultaneously. By choosing a suitable basis for this matrix, we revisit the problem in the case of three lines to show that one type of the singularities is when the camera lies on the hyperboloid of one sheet uniquely defined by the lines. This result is further exploited to prove that the one-dimensional singularities, if any, in the case of n lines appear when the camera lies on the transversals to the observed lines. Thus, by forcing the transversals to be complex, we can avoid the aforementioned type of singularities in the case of four lines although the algebra shows that there can always be up to 10 inevitable singular locations of the camera for the other type of singularity. For five lines, we find out that there are no singularities in the generic case. The singularities are also characterized for four and five lines with orthogonality and parallelism constraints. Furthermore, a visual servoing library is used to conduct some simulated experiments to substantiate the theoretical results. As expected, we observe problems in control in the vicinity of a singularity as well as increased errors in pose estimation.
Figures & tables
Figure 1 : Observation of a line.
Figure 2 : One of the singularities in P3L is when the camera center C lies on the hyperboloid formed by the three observed lines.
Figure 3 : Four observed lines Li,i=1,2,3,4 in a hyperbolic congruence leading to two singular lines LM and LN .
Figure 4 : Singularities in P4L with orthogonality and parallelism constraints: Four observed lines Li,i=1,2,3,4 and their traversals LM and LN .
Figure 5 : Singularities in P5L with orthogonality and parallelism constraints: Five observed lines Li,i=1,2,3,4,5 and their traversal LM .
Cases
Subcases
Singularity configurations
P3L
Three skew lines
C lies on the hyperboloid of one sheet uniquely defined by the observed lines or on a cubic surface that contains the three lines
P4L
Four lines in a hyperbolic congruence
C lies on two affine lines intersecting the four observed lines and up to 10 real points
Four lines in a parabolic congruence
C lies on an affine line intersecting the four observed lines and up to 10 real points
Four lines in an elliptic congruence
Up to 10 real points
With orthogonality and parallelism constraints
C lies on the two affine lines intersecting the four observed lines
P5L
Five lines in a regular linear line complex
No singularities
Table 1 : Different cases of singularities in P4L and P5L.
Figure 6 : Visual servoing using four image lines, starting from four initial poses (coloured). The desired end pose is in black. The black dashed line is the singularity line LM that intersects the four observed lines.
Figure 7 : Inverse of the condition number κ of the interaction matrix M(4) (left) and norm of the error vector ∣∣s−s∗∣∣ (right).
Figure 8 : Velocity inputs τc for the camera in a stable control scheme (left), and when crossing a singularity (right).
Figure 9 : Distance to the target during the visual servo.
ΔX
ΔY
ΔZ
Note
Desired
0.30
−0.30
0.30
Target end position s∗
Start 1
0.20
0.30
0
Near to singularity.
Start 2
−0.30
0.30
−0.30
Opposed to desired.
Start 3
0
0.30
0.40
Near to singularity.
Start 4
0.10
−0.60
0.10
Away from singularity.
Table 2 : Initial and desired positions relative to a point on the singularity line LM (all units are in meters).
Figure 10 : VS from four image lines: Trajectory described by the camera when controlling it along a prescribed path (thin dotted line). The control becomes unstable along the trajectory that crosses the singularity line LN (black dashed line).
Figure 11 : Camera velocity inputs τc (top) and position error ∣∣r(t)−r∗(t)∣∣ (bottom) along the trajectories. The vertical step in the bottom figure indicates where Trajectory 1 crosses the singularity line LN .
Figure 12 : Visual servoing along a trajectory with the shape of a quadrifolium (red) centered at the singularity point P1 . The true end camera position is drawn in black. A large translation error occurs every time the camera approaches the singularity.
Figure 13 : Velocity inputs τc (top) and translation error ∣∣r−r∗∣∣ during experiments 0 and 1 . The vertical steps indicate where the trajectory in Ex. 1 traverses the singularity point P1 .
Figure 14 : Maximum and median error along the quadrifolium trajectory for all experiments.
Ex.
Coordinates
Note
1
[5.02.03.0]
Away from singularities.
2
[−9.858−2.473−1.841]
Isolated point singularity.
3
[0.7809−2.0243.50]
On singularity line LM .
Table 3 : Point coordinates used for simulations of pose determination (all coordinates given in meters).
Figure 15 : Pose estimation from four image lines along a trajectory with the shape of a quadrifolium centered at different points: a generic point away from singularities (left), an isolated point singularity (center) and a point on a line singularity (right). The top images show the true camera position (red), the estimation from the non-iterative RP n L algorithm (blue), and its refinement by VVS (yellow). In the bottom are displayed the translation error and the absolute error angle ( 70 ). The vertical steps indicate the points where the camera passes through the singularity. Far away from any singularities both methods have a near-zero error; only the yellow plot is visible in the left image because the three trajectories overlap. In a large area around a singularity, RP n L becomes very sensitive to noise in the data, while VVS is quite effective in refining the result from RP n L except when very near the singularity. In the near proximity of the line singularity, both methods output an abhorrent estimation, with the errors tending to infinity.
Figure 16 : Visual servoing from five lines starting from different positions (coloured) towards a desired pose (black). The line LM (dashed line) that intersects all five lines is a singularity of the interaction matrix.
Figure 17 : Camera velocity inputs τc in a stable situation (left), and when crossing a singularity (right).
Figure 18 : Inverse of the condition number κ of the interaction matrix M(4) (left) and norm of the error vector ∣∣s−s∗∣∣ (right).
Figure 19 : Pose computation from RP n L using five image lines along a quadrifolium trajectory centered at a point on the line singularity LM , and its refinement from VVS. In the proximity of the singularity the error in the estimation grows unbounded.
Figure 20 : Translation (top) and rotation (bottom) errors in pose estimation from five lines along a quadrifolium trajectory. The vertical steps indicate where the camera crosses the singularity.
This paper presents a new algorithm to estimate absolute camera pose given an axis of the camera's rotation matrix. Current algorithms solve the problem via algebraic solutions on limited input domains. This paper shows that the problem can be solved efficiently by finding the intersection points of a hyperbola and the unit circle. The solution can flexibly accommodate combinations of point and line features in minimal and overconstrained configurations. In addition, the two special cases of planar and minimal configurations are identified to yield simpler closed-form solutions. Extensive experiments validate the approach.
We present VGM-VS, a visual servoing method built on a pretrained feed-forward visual geometry model. Given the current view and a reference image captured at the target configuration, we estimate the relative camera pose with a visual geometry model and apply it iteratively as the pose increment of a closed-loop pose-based visual servoing (PBVS) scheme. The geometry-aware representation acquired from large-scale pretraining keeps this estimate reliable when the target is occluded, weakly textured, or covers only a small part of the image. However, the scale ambiguity inherent to these models leaves the predicted translation defined up to an unknown scale, while the pose increment must be metric for robot control. We close this gap with a scene-specific metric adaptation: the robot autonomously records image--pose pairs along a predefined motion starting from the target pose, and we fine-tune the camera head on these data, jointly learning the hand--eye transform and thus removing the need for a dedicated calibration process. We evaluate our method on three real-world assembly tasks with demanding tolerances: USB-C cable picking, cable insertion, and RAM insertion. Running in real time at 30Hz, VGM-VS converges to submillimeter terminal accuracy on the cable tasks, and reaches success rates of 90--100% when the target is moved during servoing. It converges in all trials under initial displacements of up to 30cm from the reference pose and with 50% of the target object occluded, outperforming the compared visual servoing baselines.
Yimin Pan, Sen Wang, You Zhou +4
Agile Robots SE, Munich, Germany · Technical University of Munich
Solving non-linear least-squares problem for pose estimation (rotation and translation) is often a time consuming yet fundamental problem in several real-time computer vision applications. With an adequate rotation parametrization, the optimization problem can be reduced to the solution of a~system of polynomial equations and solved in closed form. Recent advances in efficient closed form solvers utilizing resultant matrices have shown a promising research direction to decrease the computation time while preserving the estimation accuracy. In this paper, we propose a new class of resultant-based solvers that exploit Sylvester forms to further reduce the complexity of the resolution. We demonstrate that our proposed methods are numerically as accurate as the state-of-the-art solvers, and outperform them in terms of computational time. We show that this approach can be applied for pose estimation in two different types of problems: estimating a pose from 3D to 3D correspondences, and estimating a pose from 3D points to 2D points correspondences.
Jana Vráblíková, Ezio Malis, Laurent Busé
AROMATH · Centre Inria d’Universit´e Cˆote d’Azur, France · ACENTAURI