Authors: Nicolas Andreff, Bernard Espiau, Radu Horaud
Organizations: Institut Français de Mécanique Avancée, BP 265, 63175 Aubière Cedex, France. · INRIA Rhône-Alpes and GRAVIR-IMAG, 655, av. de l’Europe, 38330 Montbonnot Saint Martin, France.
In this work, we present a fundamentally new approach to visual servoing using lines. It is based on a theoretical and geometrical study of the main line representations which allowed us to define a new representation, the so-called binormalized Plucker coordinates. They are particularly well suited to visual servoing. Indeed, they allow the definition of a proper image line alignment concept. The control law which realizes such an alignment has moreover several properties: partial decoupling between rotation and translation, analytical inversion of the motion equations and global asymptotic stability conditions. This control law was validated both in simulation and experimentally in the specific case of an orthogonal trihedron.
Figures & tables
Figure 1: Detail of a part to be welded.
Figure 2: Geometric interpretation of the normalized Plücker coordinates of a line.
Figure 3: The binormalized Plücker coordinates have a sequential interpretation: from image measurements to the estimation of 3 d components.
Figure 4: Geometrical objects generated by lines with fixed components of their binormalized Plücker coordinates
Figure 5: Two different choices for the definition interval of θ
Figure 6: The action of the translation control V=−μhu×(h−h∗) : in the case where u=u∗ (left) and in the general case (right).
Figure 7: A laser pointer is rigidly linked to the camera
Figure 8: Sequential activation of the control in rotation and then in translation: Rotation (left) and translation (right) control.
Figure 9: Sequential activation of the control in rotation and then in translation: Errors in orientation (solid line), on the image line projections (dotted line) and on the laser spot position (dashed line).
Figure 10: Sequential activation of the control in rotation and then in translation: Trajectory of the trihedron in the image (left) and in the 3 d space (right).
Figure 11: Simultaneous activation of the control in rotation and then in translation: Rotation (left) and translation (right) control.
Figure 12: Simultaneous activation of the control in rotation and then in translation: Errors in orientation (solid line), on the image line projections (dotted line) and on the laser spot position (dashed line).
Figure 13: Simultaneous activation of the control in rotation and then in translation: Trajectory of the trihedron in the image (left) and in the 3 d space (right).
Figure 14: Pure translation: Desired (left) and initial (right) positions of the robot.
Figure 15: Pure translation: coefficients of the vectors hi (left) and ui (right) along the control.
Figure 16: Pure translation: Rotation (left) and translation (right) control.
Figure 17: Pure translation: Errors (left) and image trajectory of the trihedron center (right).
Figure 18: Pure rotation: Rotation (left) and translation (right) control.
Figure 19: Pure rotation: Errors (left) and image trajectory of the trihedron center (right).
This paper deals with image-based visual servoing and pose estimation by observing four and five lines. Our main interest is to determine the relative configurations of the camera and the observed lines that lead to problems in control and stability. Since it is equivalent to finding the singularities of the corresponding Jacobian matrix, we use tools from computational algebraic geometry to seek configurations such that all of its minors vanish simultaneously. By choosing a suitable basis for this matrix, we revisit the problem in the case of three lines to show that one type of the singularities is when the camera lies on the hyperboloid of one sheet uniquely defined by the lines. This result is further exploited to prove that the one-dimensional singularities, if any, in the case of n lines appear when the camera lies on the transversals to the observed lines. Thus, by forcing the transversals to be complex, we can avoid the aforementioned type of singularities in the case of four lines although the algebra shows that there can always be up to 10 inevitable singular locations of the camera for the other type of singularity. For five lines, we find out that there are no singularities in the generic case. The singularities are also characterized for four and five lines with orthogonality and parallelism constraints. Furthermore, a visual servoing library is used to conduct some simulated experiments to substantiate the theoretical results. As expected, we observe problems in control in the vicinity of a singularity as well as increased errors in pose estimation.
Jorge García Fontán, Abhilash Nayak, Sébastien Briot +1
Sorbonne Universit´e, LIP6, Equipe PolSys, Paris, France · Centre national de la Recherche Scientifique (CNRS), Laboratoire des Sciences du Num´erique de Nantes (LS2N), UMR CNRS 6004, Nantes, France
We present VGM-VS, a visual servoing method built on a pretrained feed-forward visual geometry model. Given the current view and a reference image captured at the target configuration, we estimate the relative camera pose with a visual geometry model and apply it iteratively as the pose increment of a closed-loop pose-based visual servoing (PBVS) scheme. The geometry-aware representation acquired from large-scale pretraining keeps this estimate reliable when the target is occluded, weakly textured, or covers only a small part of the image. However, the scale ambiguity inherent to these models leaves the predicted translation defined up to an unknown scale, while the pose increment must be metric for robot control. We close this gap with a scene-specific metric adaptation: the robot autonomously records image--pose pairs along a predefined motion starting from the target pose, and we fine-tune the camera head on these data, jointly learning the hand--eye transform and thus removing the need for a dedicated calibration process. We evaluate our method on three real-world assembly tasks with demanding tolerances: USB-C cable picking, cable insertion, and RAM insertion. Running in real time at 30Hz, VGM-VS converges to submillimeter terminal accuracy on the cable tasks, and reaches success rates of 90--100% when the target is moved during servoing. It converges in all trials under initial displacements of up to 30cm from the reference pose and with 50% of the target object occluded, outperforming the compared visual servoing baselines.
Yimin Pan, Sen Wang, You Zhou +4
All authors are with Agile Robots SE, Munich, Germany. · Technical University of Munich
We introduce IKKA (Inversion Classification via Critical Anomalies), a topologically motivated weighting framework for robust visual servoing under distribution shift. Unlike conventional outlier handling, IKKA treats maverick points as structurally informative observations: points where small perturbations can induce qualitatively different control responses or class assignments. The method combines local extremality, boundary transversality, and multi-scale persistence into a single anomaly weight, W(x) = E(x) x T(x) x M(x), which modulates control updates near ambiguous decision regions. We instantiate IKKA in a CPU-only embedded visual-servoing pipeline on Raspberry Pi 4 and evaluate it across 230 reproducible runs under nominal and stress conditions. In stress scenarios involving dim illumination and transient occlusion, IKKA reduces the 95th-percentile lateral error by 24% relative to a hybrid baseline (0.124 to 0.094) while increasing throughput from 20.0 to 24.8 Hz. Non-parametric analysis confirms a large effect size (Cliff's delta = 0.79).