TapTacTouch: Shape Completion from Sparse Tactile Contacts with Geometric Priors
Organizations: School of Computing, iRASC Lab., Gachon University · Department of Electronic Engineering, Namseoul University
Abstract
3D object reconstruction is essential for robotic perception and manipulation, yet conventional approaches predominantly rely on visual observations that can be limited by occlusion or unavailable in visually constrained environments. In this work, we propose a vision-free, training-free, and category-agnostic framework for reconstructing 3D object meshes from sparse tactile contacts without relying on pretrained shape models. A 6-DoF robotic arm equipped with a 20-DoF multi-finger robotic hand collects tactile observations using 3x3 pressure sensor arrays mounted on the finger pads. The acquired tactile measurements are transformed into 3D contact constraints through robot kinematics and integrated into a multi-scale Gaussian Process Implicit Surface (GPIS). To address sparse and incomplete surface observations, the proposed method incorporates contact confidence weighting, weak pseudo-surface constraints, and an observation-driven geometric prior to preserve local surface geometry and provide structural guidance in unobserved regions. Real-world experiments on six objects achieve a Chamfer Distance (CD) of 1.610, an Earth Mover's Distance (EMD) of 7.738, and an F-Score of 95.00, with reconstruction completed in an average of 49.99 s per object. These results demonstrate that sparse tactile sensing can provide sufficient geometric information for continuous 3D object reconstruction without visual or semantic information.
Figures & tables
| Object | CD | EMD | F-Score | Reconstruction time(s) |
|---|---|---|---|---|
| Object 1 | 1.589 | 7.993 | 96.98 | 61.67 |
| Object 2 | 1.500 | 7.144 | 95.82 | 13.01 |
| Object 3 | 1.602 | 6.817 | 95.71 | 34.10 |
| Object 4 | 1.166 | 7.405 | 99.14 | 47.94 |
| Object 5 | 1.204 | 7.038 | 97.86 | 127.60 |
| Object 6 | 2.602 | 10.033 | 84.49 | 15.64 |
| Method | CD | EMD | F-Score |
|---|---|---|---|
| Standard GPIS (Single-scale) | 3.448 | 11.060 | 75.580 |
| Multi-scale GPIS | 1.244 | 6.491 | 97.504 |
| Multi-scale GPIS + PCG | 1.248 | 6.491 | 97.504 |
| Multi-scale GPIS + CSP | 1.229 | 6.818 | 97.497 |
| Ours: Multi-scale GPIS + CSP + PCG | 1.236 | 6.637 | 97.509 |
| Method | CD | EMD | F-Score |
|---|---|---|---|
| Standard GPIS (Single-scale) | 4.076 | 13.019 | 71.253 |
| Multi-scale GPIS | 2.133 | 8.968 | 89.705 |
| Multi-scale GPIS + PCG | 2.102 | 8.731 | 90.124 |
| Multi-scale GPIS + CSP | 2.003 | 8.573 | 92.148 |
| Ours: Multi-scale GPIS + CSP + PCG | 1.992 | 8.862 | 92.605 |
| Method | Object1 | Object2 | Object3 | Object4 | Object5 | Object6 | Mean |
|---|---|---|---|---|---|---|---|
| Single-scale GPIS | 4.456 | 3.277 | 4.324 | 3.362 | 2.915 | 4.261 | 3.766 |
| Multi-scale GPIS | 1.673 | 1.828 | 1.791 | 1.193 | 1.229 | 2.521 | 1.706 |
| Multi-scale GPIS+PCG | 1.674 | 1.766 | 1.758 | 1.205 | 1.213 | 2.533 | 1.691 |
| Multi-scale GPIS+CSP | 1.569 | 1.597 | 1.684 | 1.150 | 1.214 | 2.594 | 1.635 |
| Ours | 1.590 | 1.552 | 1.680 | 1.166 | 1.204 | 2.602 | 1.632 |
3D Reconstruction
3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.
See all 1,025 papers