We present GelSphere, a spherical vision-based tactile sensor designed for real-time continuous surface scanning. Unlike traditional vision-based tactile sensors that can only sense locally and are damaged when slid across surfaces, and cylindrical tactile sensors that can only roll along a fixed direction, our design enables omnidirectional rolling on surfaces. We accomplish this through our novel sensing system design, which has steel balls inside the sensor, forming a bearing layer between the gel and the rigid housing that allows rolling motion in all axes. The sensor streams tactile images through Wi-Fi, with online large-surface reconstruction capabilities. We present quantitative results for both reconstruction accuracy and image fusion performance. The results show that our sensor maintains geometric fidelity and high reconstruction accuracy even under multi-directional rolling, enabling uninterrupted surface scanning.
Figures & tables
Fig. 1: Our novel vision-based tactile sensor, GelSphere, achieves omnidirectional continuous scanning of large surfaces through a unique mechanical design. It can be easily human-operated for online surface reconstruction or potentially used in robotic platforms for automation. We use tactile SLAM methods to stitch the contact patches together, forming a continuous surface model.
Fig. 2: Optical and mechanical design of GelSphere. (A) The simulated tactile image is shown alongside a real sensor image under a spherical indentation, presenting a close visual similarities between the optical simulation and the fabricated sensor. The RGB LED ring consists of 18 LEDs (6 of each color), and three color groups are spaced by approximately 120∘ , which provides directional illumination from every direction. (B) Cross-sectional view of the GelSphere sensor. The internal optical module (camera, LED right, and battery) is mounted on a base that is magnetically suspended, which helps maintain a consistent camera orientation while the outer shell rolls. A magnetic switch enables power on/off control without opening the housing. The steel ball bearing layer ensures low-friction contact between the sensing gel assembly and the housing, enabling smooth rolling during scanning. The sensor is self-contained, with onboard power and wireless image streaming.
Fig. 3: GelSphere sensor design. (A) Fabricated GelSphere components. The sensing gel pad is removable to allow battery replacement and maintenance. The cap is fastened to the housing to support the magnetic suspension structure and retain the steel balls in the bearing race. (B) Assembled prototype. The sensor is able to roll omnidirectionally when fully assembled.
Fig. 4: Fabrication Process for making the gel pad and the shell. The silicone mold (Mold Star 15) is created using the gel pad replica, and XP-565 silicone elastomer is added to the cured silicone mold. The off-the-shelf plastic shell is added to the top mold and pressed carefully with the bottom part of the mold, ensuring no air is caught. The cured elastomer is air-sprayed with the desired coating material to finish the gel pad.
(A) Single-frame (Hex Indenter)
(B) Large-surface (Feather)
Coating
Dot Product
MSE
MAE
SSIM
PSNR [dB]
Dist. MSE [ mm2 ]
Dist. MAE [ mm ]
Angle MSE [ ∘2 ]
Angle MAE [ ∘ ]
Specular
0.965
0.023
0.111
0.950
22.62
1.6169
1.070
3.7
1.3
Matte
0.939
0.041
0.156
0.930
20.08
2.1813
1.251
12.8
2.7
TABLE I: Comparison of reconstruction performance under specular (bronze) and matte coatings for single-frame and large-surface reconstruction. Higher Dot Product, SSIM, and PSNR, and lower MSE/MAE indicate better alignment with ground truth.
Fig. 5: GelSphere’s single frame sensing performance. (A) A 2D normal-estimation accuracy plot was generated from the indentation of a hexagonal pyramid at multiple locations. The RGB direction labels in the dot-product maps are for visual reference. The three LED groups are positioned 120∘ apart around the ring. Left: specular, Right: matte. (B) A key and an M8 screw were pressed into both specular and matte sensing gel, showcasing the reconstruction performance.
Fig. 6: Large surface reconstruction of an SLA-printed feather using specular and matte-coated GelSphere sensors. (A) The printed ground-truth and the reconstructed meshes from specular and matte coatings. (B) Surface normal maps are used for error analysis. 24 correspondence points are manually selected on the ground truth and each reconstructed normal map. These points are used to compare pair-wise distances and local normal directions between corresponding points. The specular coating yields lower error than the matte coating, as shown in Table I .
Fig. 7: Contact normal force estimation using GelSphere. (A) We use linear guides to constrain motion to the z-axis in the force application setup. The ground truth force is measured using a force-torque sensor (NRS-6050-D50, Nordbo Robotics), while indenters of different radii are manually pushed onto the sensor surface. (B) The size of the contact area increases as the force increases. The specular surface has a much higher visual sensitivity compared to the matte coating. (C) The specular-coated sensor estimate forces with MAE of 0.44 N while the Matte coating has almost double the amount, resulting from a lower signal-to-noise ratio compared to specular.