TacLoc: Global Tactile Localization on Objects from a Registration Perspective

Zirui Zhang; Boyang Zhang; Fumin Zhang; Huan Yin

cs.ROMar 11, 2026

TacLoc: Global Tactile Localization on Objects from a Registration Perspective

Authors: Zirui Zhang, Boyang Zhang, Fumin Zhang, Huan Yin

Abstract

Pose estimation is essential for robotic manipulation, particularly when visual perception is occluded during gripper-object interactions. Existing tactile-based methods generally rely on tactile simulation or pre-trained models, which limits their generalizability and efficiency. In this study, we propose TacLoc, a novel tactile localization framework that formulates the problem as a one-shot point cloud registration task. TacLoc introduces a graph-theoretic partial-to-full registration method, leveraging dense point clouds and surface normals from tactile sensing for efficient and accurate pose estimation. Without requiring rendered data or pre-trained models, TacLoc achieves improved performance through normal-guided graph pruning and a hypothesis-and-verification pipeline. TacLoc is evaluated extensively on the YCB dataset. We further demonstrate TacLoc on real-world objects across two different visual-tactile sensors.

Figures & tables

Fig. 1: Real-world demonstration using TacLoc. A GelSight Mini sensor touches a textureless 3D-printed object (left), producing a tactile image that is converted into a dense point cloud and registered to a prior CAD model (right). The zoomed view shows the tactile point cloud (blue) aligned on the CAD model (red).

Fig. 2: Overview of the TacLoc pipeline for one-shot global tactile localization. The process starts with feature extraction to establish initial correspondences. A compatibility graph is constructed based on distance and normal consistency, and maximal cliques are identified for pose hypothesis generation. Finally, a hypothesis-and-verification approach is applied to refine the pose estimation, achieving partial-to-full tactile localization. It is worth noting that a CAD model is indispensable as a prior map and end-effector poses are required to build submaps during sliding touch.

Fig. 3: For each frame captured by the tactile sensor, we first convert it into a point cloud with normals. Then we construct a submap representing either a sliding touch or multi-time retouch by integrating end-effector poses. This submap is then processed using our front-end preprocessing pipeline. Best viewed zoomed in and in color.

Panel of Figure 4 — Fig. 4: Visualization of candidate extraction pipeline with a minimal example (best viewed in color). (a) Three points in the source point cloud (blue) and three points in the target point cloud (orange), connected by six initial correspondences (gray lines). (b) The correspondences form an undirected graph $G$ , with nodes $V=\{\xi_{1,1},\xi_{1,2},\xi_{2,1},\xi_{2,2},\xi_{2,3},\xi_{3,3}\}$ and edges $E=\{\{\xi_{1,1},\xi_{2,2}\},\{\xi_{1,1},\xi_{3,3}\},\{\xi_{1,2},\xi_{2,1}\},\{\xi_{2,2},\xi_{3,3}\}$ , constructed via pairwise consistency checks. (c) The first maximal clique $C_{1}=\{\xi_{1,1},\xi_{2,2},\xi_{3,3}\}$ . (d) The second maximal clique $C_{2}=\{\xi_{1,2},\xi_{2,1}\}$ .

Module	Parameters	Value
Feature Extraction	Voxel size	$1\text{mm}$
Feature Extraction	Radii	$5\text{mm}$
Corr. Pruning	Distance threshold/bound $\delta_{d}$	$6\text{mm}$
Corr. Pruning	Angular threshold/bound $\delta_{\alpha}$	$30^{\circ}$
Pose Estimation	Number of pose candidates $K$	$300$
Pose Estimation	Weight balance $\alpha$	$1$

TABLE I: Key Parameters of TacLoc for YCB-Reg

Panel of Figure 6 — Fig. 5: Selected samples from the YCB-Reg benchmark. From left to right are ten objects from the YCB dataset [ 32 ] . Gray semi-transparent objects represent the target models, while green point clouds correspond to tactile-based point clouds using the pre-processing approach in Section IV-A . Zoomed views of these sliding samples are also presented at the bottom.

Front-end	Back-end	RE ( $\mathrm{\SIUnitSymbolDegree}$ )	TE ( $\mathrm{mm}$ )	Time ( $\sec$ )
FPFH	RANSAC	128.62	99.94	1.66
FPFH	TEASER++	19.89	8.46	13.04
FPFH	3DMAC	19.07	9.54	2.06
FPFH	TacLoc	0.94	0.69	1.40
SpinNet	RANSAC	118.63	55.89	$0.05^{\star}$
SpinNet	TacLoc	130.37	28.50	$0.92^{\star}$

TABLE II: Performance Comparison on Tactile Registration

Panel of Figure 8 — Fig. 6: Recall rate evaluated on the YCB-Reg using the FPFH descriptor for different back-end approaches under varying criterion of successful registration. The results demonstrate that the proposed TacLoc consistently achieves better performance across all tested conditions.

Panel of Figure 9 — Fig. 7: Failure case studies on YCB-Reg benchmark. In each subfigure, the left shows the ground truth while the right displays the misalignment. TacLoc fails in (a) due to the extremely low inlier ratio; the failure in (b) is caused by repetitive patterns in symmetric objects.

Panel of Figure 10 — Fig. 8: Parameter analysis on two YCB-Reg objects. A longer sliding length enhances the performance of tactile localization, aligning with human intuition. Additionally, the accuracy of the end-effector pose is critical: a more precise end-effector pose results in a more consistent submap, thereby improving the performance of TacLoc.

Panel of Figure 11 — Fig. 9: Experimental results demonstrate the efficacy of normal-guided pruning. (a) Graph density, number of cliques, and computation time decrease as $\delta_{\alpha}$ varies; (b) The number of edges and computation time exhibit an approximately polynomial relationship. When $\delta_{\alpha}$ is $30^{\circ}$ or $180^{\circ}$ , the average number of edges and computation times are $7,473.82$ edges with $0.20\text{s}$ and $15,476.75$ edges with $2.87\text{s}$ , respectively.

Panel of Figure 12 — Fig. 10: Rotation error and time cost with varying thresholds $\delta_{d}$ and $\delta_{\alpha}$ .

Fig. 11: Per-stage computational profile. FPFH-based initial data association and transformation verification are the most time-intensive stages.

Panel of Figure 14 — Fig. 12: (a) TacLoc is deployed on two real-world sensors beyond the DIGIT used in YCB simulation, despite these sensors having different characteristics. The right side shows the visual-tactile images obtained from the sensors. (b) Evaluation on five real-world objects using the GelSight Mini. We perform sliding touches on the objects 50 times, and the proposed TacLoc achieves a success rate of 33/50. Note that the red object is used for the single-touch demonstration shown in Figure 1 . The right side displays the successful cases, where the yellow points represent the submaps.

Object Pose Estimation

Object pose estimation recovers an object's 3D position and orientation from images, depth, point clouds, or contact sensing.

See all 187 papers

You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact
Jun 27, 2026Pengfei Ye, Yuxiang Ma, Haonan Chen +5Object Pose Estimation Robotic Manipulation
MIT CSAIL · Harvard University · University of Cambridge
VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds
Jul 17, 2026Zhiyuan Wu, Zhuo Chen, Shan LuoVision-Based Tactile Sensing Tactile Sensing
Department of Engineering, King’s College London, Strand, London, WC2R 2LS, United Kingdom
InvariantCloud: A Globally Invariant, Uniquely Indexed Point Cloud Framework for Robust 6-DoF Tactile Pose Tracking
May 24, 2026Pengfei Ye, Yuxiang Ma, Yi Zhou +3Vision-Based Tactile Sensing Object Pose Estimation
Department of Mechanical and Aerospace Engineering, The Hong Kong University of Science and Technology · Department of Mechanical Engineering, Massachusetts Institute of Technology · Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong

Jun 27, 2026J/K move · Enter open · S save

You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact

Pengfei Ye, Yuxiang Ma, Haonan Chen, Guangming Wang+4

MIT CSAIL · Harvard University · University of Cambridge

Object Pose Estimation Robotic Manipulation Tactile Sensing for Robotic Manipulation

Open paper AskPDF arXiv

Accurate 6-DoF object pose estimation is fundamental to robotic manipulation, yet vision-based methods often fail under occlusion, poor lighting, and reflective or transparent surfaces. We present YOTO, a tactile-only pose estimation system that recovers the full 6-DoF object pose from a single pair of simultaneous contacts, without requiring contact history. YOTO represents each tactile contact as a local 3D point cloud and localizes it on the object surface through a coarse-to-fine network. The two localized contacts, together with the calibrated sensor poses, are then fed to a closed-form normal-aware SVD solver that recovers the full 6-DoF object pose in one step. To reduce real-data requirements, the localization network is pretrained on virtual tactile patches sampled from the object model and fine-tuned with a small number of real contacts. We further show that YOTO can operate on object models reconstructed from consumer-grade mobile scans, and quantify the gap relative to CAD-based models. Experiments on four geometrically diverse objects demonstrate accurate tactile contact localization and pose estimation, outperforming vision-based and geometric baselines, especially when visual perception is unreliable. Code, trained models, and the real GelSight dataset will be released upon publication.

Abstract

Object Pose Estimation

Explore similar work

You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact

VTLoc: Learning-based Tactile Contact Localization in Visual Point Clouds

InvariantCloud: A Globally Invariant, Uniquely Indexed Point Cloud Framework for Robust 6-DoF Tactile Pose Tracking