3D object reconstruction is essential for robotic perception and manipulation, yet conventional approaches predominantly rely on visual observations that can be limited by occlusion or unavailable in visually constrained environments. In this work, we propose a vision-free, training-free, and category-agnostic framework for reconstructing 3D object meshes from sparse tactile contacts without relying on pretrained shape models. A 6-DoF robotic arm equipped with a 20-DoF multi-finger robotic hand collects tactile observations using 3x3 pressure sensor arrays mounted on the finger pads. The acquired tactile measurements are transformed into 3D contact constraints through robot kinematics and integrated into a multi-scale Gaussian Process Implicit Surface (GPIS). To address sparse and incomplete surface observations, the proposed method incorporates contact confidence weighting, weak pseudo-surface constraints, and an observation-driven geometric prior to preserve local surface geometry and provide structural guidance in unobserved regions. Real-world experiments on six objects achieve a Chamfer Distance (CD) of 1.610, an Earth Mover's Distance (EMD) of 7.738, and an F-Score of 95.00, with reconstruction completed in an average of 49.99 s per object. These results demonstrate that sparse tactile sensing can provide sufficient geometric information for continuous 3D object reconstruction without visual or semantic information.
Figures & tables
Fig. 1: Overview of the proposed tactile-only 3D reconstruction framework. Tactile contact points are aggregated into a 3D point cloud and reconstructed into a 3D mesh using Gaussian Process Implicit Surfaces (GPIS).
Fig. 2: Overview of the proposed tactile reconstruction pipeline. (a) An robotic arm equipped with an robotic hand collects tactile contacts and kinematic information, which are mapped into world-frame contacts and FK-derived normals. (b) Pseudo-Contact Generation (PCG) augments the observations, and the Cross-Section Prior (CSP) defines the GP mean. Multi-scale Gaussian regression estimates a residual field that adjusts this mean to the tactile constraints. Marching Cubes extracts the reconstructed mesh from the zero-level set of the resulting implicit field.
Fig. 3: Hardware setup of the proposed system, showing the ROBOTIS OMY robotic arm, ROBOTIS HX5-D20-MRT robotic hand, and finger pad tactile sensing regions.
Fig. 4: Input contacts, reconstructed meshes produced by the proposed method (Ours), and ground-truth (GT) reference meshes for Objects 1–6. Objects 1–3 are shown on the left and Objects 4–6 on the right. Within each object, the input, reconstruction, and GT are displayed from the same viewpoint and at the same scale.
Object
CD ↓
EMD ↓
F-Score ↑
Reconstruction time(s)
Object 1
1.589
7.993
96.98
61.67
Object 2
1.500
7.144
95.82
13.01
Object 3
1.602
6.817
95.71
34.10
Object 4
1.166
7.405
99.14
47.94
Object 5
1.204
7.038
97.86
127.60
Object 6
2.602
10.033
84.49
15.64
TABLE I: Reconstruction performance on six objects.
Method
CD ↓
EMD ↓
F-Score ↑
Standard GPIS (Single-scale)
3.448
11.060
75.580
Multi-scale GPIS
1.244
6.491
97.504
Multi-scale GPIS + PCG
1.248
6.491
97.504
Multi-scale GPIS + CSP
1.229
6.818
97.497
Ours: Multi-scale GPIS + CSP + PCG
1.236
6.637
97.509
TABLE II: Quantitative comparison in the observed region.
Method
CD ↓
EMD ↓
F-Score ↑
Standard GPIS (Single-scale)
4.076
13.019
71.253
Multi-scale GPIS
2.133
8.968
89.705
Multi-scale GPIS + PCG
2.102
8.731
90.124
Multi-scale GPIS + CSP
2.003
8.573
92.148
Ours: Multi-scale GPIS + CSP + PCG
1.992
8.862
92.605
TABLE III: Quantitative comparison in the unobserved region.
Method
Object1
Object2
Object3
Object4
Object5
Object6
Mean
Single-scale GPIS
4.456
3.277
4.324
3.362
2.915
4.261
3.766
Multi-scale GPIS
1.673
1.828
1.791
1.193
1.229
2.521
1.706
Multi-scale GPIS+PCG
1.674
1.766
1.758
1.205
1.213
2.533
1.691
Multi-scale GPIS+CSP
1.569
1.597
1.684
1.150
1.214
2.594
1.635
Ours
1.590
1.552
1.680
1.166
1.204
2.602
1.632
TABLE IV: Overall Chamfer distance (mm, ↓ ) after Fill on six objects.
Fig. 5: PCG and CSP construction for the object 3 object: contact-pair interpolation, layer-wise hulls, and the expanded common cross-section.