Tactile sensing is central to robotic manipulation, among which slip detection stands out as a quintessential and critical task. However, existing slip datasets are predominantly limited to binary classification, lacking fine-grained directional perception. To address this limitation, we propose a visuo-tactile sensor featuring customized uniform RGB illumination, alongside a unified perception framework. At the hardware level, the sensor achieves high-precision, sub-millimeter depth reconstruction. Based on this capability, we collect a multi-task visuo-tactile dataset encompassing 15 objects, synchronously generating depth information for each data sample. Algorithmically, we design a dual-head TimeSformer network to process dynamic spatiotemporal slip. On unseen objects, this network achieves robust accuracies of 95.5% and 91.5% for 3-class contact state prediction and fine-grained 8-class slip direction classification, respectively. Furthermore, static tactile-based object class recognition utilizing a ResNet-50 backbone yields an outstanding accuracy of 98.8% across 15 categories. The proposed hardware-software framework provides high-fidelity feedback and a powerful multi-modal perception baseline for complex robotic manipulation.
Figures & tables
Fig. 1: (a) Overall structural diagram of the proposed sensor.(b) Dataset collection using the visuotactile sensor for slip status detection.(c) Actual tactile images collected by pressing 15 common objects. These images constitute a slip detection dataset of 7,925 samples and a high-precision material classification dataset of approximately 19,336 images. (The figure shows some of the collected objects.)
Reference
Sensor Type
Objects
Classes
Slip State Granularity
Dataset Scale
Meier et al. [ 18 ]
Piezoresistive array
–
3
Stable, Translational Slip, Rotation
Time-series data
Zhang et al. [ 13 ]
GelSight (Marker-based)
Multiple
2
Slip, Non-slip (Binary)
Video sequences
James et al. [ 19 ]
TacTip (Biomimetic)
–
2
Slip, Non-slip (Binary)
40 sequences
Zapata-Impata et al. [ 20 ]
BioTac SP
11
7
Contact, 4 Translation, 2 Rotation
300 frames
STNet [ 21 ]
Visuo-tactile
–
2
Slip, Non-slip (Binary)
Not specified
Ours
Marker-less VTS
15
10
Static, Pressing, 8-directional slip
7925 sequences
TABLE I: COMPARISON OF EXISTING TACTILE SLIP DATASETS
Fig. 2: (a) Exploded view of the sensor components. ① Optically clear silicone coated with a white silicone pigment layer. ② Supports the elastomer. ③ Transparent acrylic plate. ④ Main housing. ⑤ Translucent frosted diffuser. ⑥ PCB for RGB illumination. ⑦ Miniature camera. ⑧ Base plate. (b) The overall dimensions of the sensor are 37 mm × 34 mm × 34 mm. (c) The internal optical path design, which effectively prevents specular highlights generated by the acrylic plate. (d) Actual image captured by the visuo-tactile sensor. Decomposing the image into individual RGB channels demonstrates a uniform intensity distribution and consistent brightness across all channels.
Fig. 3: Fabrication process of the elastic contact module. The optically clear silicone (Smooth-On Solaris) was first mixed and poured into a customized 3D-printed mold, followed by curing. A custom matte reflection coating, blended from silicone base and white pigment, was then uniformly airbrushed onto the elastomer surface. The final assembled contact module features a compact and easily replaceable design.
Fig. 4: Empirical photometric stereo calibration using a reference sphere. (a) Geometric model of the spherical indentation. The side view illustrates the physical radius R and the projected radial distance r , while the top view demonstrates the mapping relationship from the contact center (xc,yc) and RGB intensities to the surface gradients. (b) The physical calibration setup, where a standard reference sphere with a diameter of 6 mm is pressed into the sensor using a precision three-axis linear stage. (c) The corresponding RGB image captured by the internal miniature camera during the spherical indentation.
Fig. 5: The 15 representative daily objects, along with their corresponding high-resolution tactile images and reconstructed depth maps. Benefiting from the uniform illumination, micro-textures and geometric contours are presented with exceptional clarity without specular highlight interference.
Fig. 6: Visualization of the real-world grasping process and the corresponding tactile data. The figure displays a sequence of continuous raw RGB tactile images captured during dynamic interaction, alongside the corresponding temporal difference maps between consecutive frames.
Fig. 7: The overall architecture of the proposed spatiotemporal slip detection network based on a dual-head TimeSformer. The input sequence is processed through patch embedding and divided space-time attention blocks to extract dynamic features. A global token is then utilized by dual FC heads to predict the contact state and slip direction, optimized by a masked joint loss.
Fig. 8: The visuo-tactile perception pipeline for 3D depth reconstruction. The sub-millimeter precision (RMSE = 0.072 mm) relies on the high-quality gradient decoupling and the masked Poisson integration.
Fig. 9: Confusion matrices for dynamic spatiotemporal slip detection on unseen objects. (a) Results for the 3-class contact state prediction (Static, Press, and Slip). (b) Results for the fine-grained 8-class slip direction classification.
Fig. 10: Confusion matrices for dynamic and static visuo-tactile perception tasks, respectively showing the 3-class contact state prediction on unseen objects, the 8-class slip direction classification on unseen objects, and the 15-class static tactile-based object class recognition.
Perception Task
Test Condition (Modality)
Accuracy
Static Recognition (15-class)
RGB (Baseline)
98.80%
Depth-only
96.06%
Contact State (3-class)
RGB (Baseline)
99.68%
RGB (Unseen Objects)
95.50%
Depth-only
94.51%
Slip Direction (8-class)
RGB (Baseline)
96.95%
TABLE II: Summary of Visuo-Tactile Perception Accuracies
Slip detection is fundamental to dexterous manipulation, yet existing systems often lack precise characterization of detection latency and cross-platform generalization. We present SlipSense, a multimodal tactile slip-detection framework built on TacV5, a compact sensor integrating a 32×32 piezoresistive array operating at 240 Hz and a 3-axis MEMS accelerometer operating at 8 kHz. The piezoresistive array captures spatial pressure distributions, while the accelerometer captures friction-induced vibrations, providing complementary slip cues. The framework performs modality-specific encoding, intra-sensor fusion, and cross-modal attention with causal temporal prediction at 240 Hz. Experiments on a dataset of 1.4 million frames spanning 37 objects demonstrate the complementarity of the two modalities. SlipSense achieves 96.7% Macro F1 with a false-positive rate below 1.6%, detecting 76% of slip events within 23.1 ms. When trained solely on UMI data, SlipSense generalizes zero-shot to a Tesollo dexterous hand, transferring across unseen objects, distinct sensor units, and robotic platforms without retraining.
Tong Jian, Aditya Thurvas Senthil Kumar, Xinyi Li +7
Stable and reliable grasp is critical to robotic manipulations especially for fragile and glazed objects, where the grasp force requires precise control as too large force possibly damages the objects while small force leads to slip and fall-off. Although it is assumed the objects to manipulate is grasped firmly in advance, slip detection and timely prevention are necessary for a robot in unstructured and universal environments. In this work, we addressed this issue by utilizing multimodal tactile feedback from a five-fingered bio-inspired hand. Motivated by human hands, the tactile sensing elements were distributed and embedded into the soft skin of robotic hand, forming 24 tactile channels in total. Different from the threshold method that was widely employed in most existing works, we converted the slip detection problem to contact status recognition in combination with binning technique first and then detected the slip onset time according to the recognition results. After the 24-channel tactile signals passed through discrete wavelet transform, 17 features were extracted from different time and frequency bands. With the optimal 120 features employed for status recognition, the test accuracy reached 96.39% across three different sliding speeds and six kinds of materials. When applied to four new unseen materials, a high accuracy of 91.95% was still achieved, which further validated the generalization of our proposed method. Finally, the performance of slip detection is verified based on the trained model of contact status recognition.
Chengxiao He, Wenhui Yang, Hongliang Zhao +3
School of Mechanical Engineering, Southeast University, Nanjing 211189, China · School of Electronic Science & Engineering, Southeast University, Nanjing 211189, China
Reliable in-hand manipulation requires accurate real-time estimation of slip between a gripper and a grasped object. Existing tactile sensing approaches based on vision, capacitance, or force-torque measurements face fundamental trade-offs in form factor, durability, and their ability to jointly estimate slip direction and magnitude. We present A-SLIP, a multi-channel acoustic sensing system integrated into a parallel-jaw gripper for estimating continuous slip in the grasp plane. The A-SLIP sensor consists of piezoelectric microphones positioned behind a textured silicone contact pad to capture structured contact-induced vibrations. The A-SLIP model processes synchronized multi-channel audio as log-mel spectrograms using a lightweight convolutional network, jointly predicting the presence, direction, and magnitude of slip. Across experiments with robot- and externally induced slip conditions, the fine-tuned four-microphone configuration achieves a mean absolute directional error of 14.1 degrees, outperforms baselines by up to 12 percent in detection accuracy, and reduces directional error by 32 percent. Compared with single-microphone configurations, the multi-channel design reduces directional error by 64 percent and magnitude error by 68 percent, underscoring the importance of spatial acoustic sensing in resolving slip direction ambiguity. We further evaluate A-SLIP in closed-loop reactive control and find that it enables reliable, low-cost, real-time estimation of in-hand slip. Project videos and additional details are available at https://a-slip.github.io.
Uksang Yoo, Yuemin Mao, Jean Oh +1
Robotics Institute, Carnegie Mellon University, USA