Organizations: Department of Electrical, Electronic, Telecommunications Engineering, and Naval Architecture (DITEN), University of Genoa, 16145 Genoa, Italy · Department of Computer and Communication Engineering, Lebanese International University (LIU), Beirut, Lebanon
Tactile sensing provides information about object properties that can improve robotic grasping and manipulation. However, accurate object recognition during the early stages of grasping remains challenging, particularly under strict real-time constraints. This study presents a flexible, distributed piezoelectric tactile sensing system for early-stage object recognition. The system was experimentally evaluated using six everyday objects and three recognition approaches: feature-based classification using support vector machines (SVMs), raw-signal classification using 1D convolutional neural networks (1D-CNNs) and convolutional recurrent neural networks (C-RNNs), and spatiotemporal heatmap classification using a 2D-CNN. The effects of sensor count and temporal window size on recognition accuracy and real-time feasibility were systematically investigated. Increasing the number of sensing elements improved recognition performance across all approaches. The best-performing method, based on spatiotemporal heatmaps and a 2D-CNN, achieved F1-scores of 0.9796, 0.9846, and 0.9903 using 16, 24, and 32 sensors, respectively. The optimal configuration, comprising 32 sensors and a 100-ms temporal window (50 samples), was deployed on an embedded edge platform, achieving an end-to-end recognition latency of approximately 101 ms from contact onset and an energy consumption of 63.8 microjoules per inference. These results demonstrate the potential of distributed piezoelectric tactile sensing for accurate, energy-efficient, and real-time object recognition during robotic grasping.
Figures & tables
References
Sensor type
Mechanical structure
Number of sensors
Spatial resolution
Application
Hardware deployment
Response time
[ 4 ]
Piezoresistive
Flexible
64
2.5 mm
Object recognition
✗
✗
[ 5 ]
Piezoresistive
–
1
27.6 taxels/cm 2
Material classification
✗
103 ms (GPU)
[ 6 ]
Piezoresistive
Rigid
256
3.5 mm
Object recognition
✗
✗
[ 7 ]
Capacitive
Flexible
–
–
Object recognition
✗
✗
[ 8 ]
Capacitive
Flexible
64
2 mm
Material recognition
✗
✗
[ 9 ]
Piezoelectric
Flexible
1
–
Object surface recognition
✗
✗
TABLE I: Overview of most relevant existing work on tactile-based object recognition.
Fig. 1: (a) Top: Single sensor structure and Bottom: layout of the sensing array. (b) Development procedure of the sensing caps (c), System block diagram (d), Sensing system mounted on the Tiago gripper.
Sensitivity
Linearity
SINAD
1.5–31.96 pC/N
2.3% nonlinearity error
56 dB
TABLE II: Sensor characterization metrics for the proposed sensing system.
Fig. 2: Channel-to-channel consistency of three sensors. The charge response is shown as a function of the applied force for sensors S5, S11, and S1.
Fig. 3: (a) Response of a single sensor to a sine stimulus at variable frequencies (1 Hz - 100 Hz). (b) Response of a single sensor to a square stimulus at variable forces (0.12 N - 6 N).
Fig. 4: (a) Set of daily-use objects, (b) TIAGo robot manipulating a PB, and (c) the corresponding sensor response.
Algorithm 1 Energy Computation
Fig. 6: Block diagram of object recognition approaches: End-to-end (EEA), Feature-Based Approach (FBA), and Heatmap Approach (HMA).
Sensor Group
Approach
Nc=16
Nc=24
Nc=32
EEA
Dg3W
Dg4W
DW
FBA
D~g3W
D~g4W
D~W
HMA
Hg3W
Hg4W
HW
TABLE III: Summary of the datasets
Algorithm 2 HeatMap Construction
Fig. 7: Illustration of the heatmap constructed from the tactile signals with the Dataset H50 .
Algorithm
Hyperparameter
Values
CNN
F (Filters)
{8,16,32,64}
K (Kernel Size)
{2,3,4}
SVM
C (Penalty)
{0.01,0.1,1,10,100}
γ (Kernel Coefficient)
{0.01,0.1,1,10,100}
C-RNN
F (Filters)
{8,16,32,64}
K (Kernel Size)
{2,3,4}
TABLE IV: Summary of Hyperparameter Configurations for Each Algorithm
Fig. 8: Radar plots presenting the performance metrics of the implemented algorithms across different window size ( W ) and sensor groups ( NC ).
Approach
Dataset
F1-score
EEA
Dg350
0.8824
Dg450
0.9102
D50
0.9532
FBA
D~g350
0.9445
D~g450
0.9604
D~50
0.9661
TABLE V: Table presenting the highest F1-score values achieved with W =50, by the three approaches including C-RNN, SVM, and 2D-CNN across the three datasets.
Fig. 9: Performance of the implemented models in each of the three approaches for the three sensor groups using the smallest window size ( W=50 ).
Dataset
H50
Inference time (ms)
0.8817
Flash Size (kB)
7.65
kFLOPs
11.3
Power (mW)
72.3
Energy (µJ)
63.8
TABLE VI: Performance Metrics of the Best Performing Model (2D-CNN)
We present FlexiTac, a low-cost, open-source, and scalable piezoresistive tactile sensing solution designed for robotic end-effectors. FlexiTac is a practical "plug-in" module consisting of (i) thin, flexible tactile sensor pads that provide dense tactile signals and (ii) a compact multi-channel readout board that streams synchronized measurements for real-time control and large-scale data collection. FlexiTac pads adopt a sealed three-layer laminate stack (FPC-Velostat-FPC) with electrode patterns directly integrated into flexible printed circuits, substantially improving fabrication throughput and repeatability while maintaining mechanical compliance for deployment on both rigid and soft grippers. The readout electronics use widely available, low-cost components and stream tactile signals to a host computer at 100 Hz via serial communication. Across multiple configurations, including fingertip pads and larger tactile mats, FlexiTac can be mounted on diverse platforms without major mechanical redesign. We further show that FlexiTac supports modern tactile learning pipelines, including 3D visuo-tactile fusion for contact-aware decision making, cross-embodiment skill transfer, and real-to-sim-to-real fine-tuning with GPU-parallel tactile simulation. Our project page is available at https://flexitac.github.io/.
Rapid deployment of new tactile sensors is essential for scalable robotic manipulation, especially in multi-fingered hands equipped with vision-based tactile sensors. However, current methods for inferring contact properties rely heavily on convolutional neural networks (CNNs), which, while effective on known sensors, require large, sensor-specific datasets. Furthermore, they require retraining for each new sensor due to differences in lens properties, illumination, and sensor wear. Here we introduce TacViT, a novel tactile perception model based on Vision Transformers, designed to generalize on new sensor data. TacViT leverages global self-attention mechanisms to extract robust features from tactile images, enabling accurate contact property inference even on previously unseen sensors. This capability significantly reduces the need for data collection and retraining, accelerating the deployment of new sensors. We evaluate TacViT on sensors for a five-fingered robot hand and demonstrate its superior generalization performance compared to CNNs. Our results highlight TacViTs potential to make tactile sensing more scalable and practical for real-world robotic applications.
Christopher J. Ford, Kaichen Shi, Laura Butcher +2
School of Engineering Mathematics and Technology and Bristol Robotics Laboratory, University of Bristol, UK
Multi-finger hands contact objects at different times, so a global close command can overload early-contact fingers while others are still moving. Dense tactile arrays address this problem but add hardware, calibration, bandwidth, and computation. We present an edge-native pipeline that uses only built-in joint-angle and pre-use-calibrated actuator-load feedback to recognize the grasped object and predict an independent Stop event for each finger. Four samples and their first differences form a 4-by-20 contact-evolution window. At its core is a heterogeneous dual-expert architecture, termed DualExpert, that assigns complementary tasks to a Temporal Convolutional Network (TCN) and a Finger Graph Filter Network (FGFN). The TCN models short-term contact evolution and supplies all five Stop probabilities, while the FGFN encodes finger topology and contributes only to object classification. Fixed 0.7/0.3 class-probability fusion reaches 93.78% trajectory-level classification accuracy on 225 held-out trials, improving the standalone TCN and FGFN by 4.00 and 2.22 percentage points, respectively. The TCN Stop path attains 95.25% F1. The dataset contains 1,500 automatically collected trajectories from five object instances. In Flash deployment, mixed accelerator execution retains 93.78% classification accuracy and 94.84% Stop F1 with 2.890ms mean inference time and 9.172ms mean sample-to-stop latency. Across 50 physical trials, the system achieves 96.00% recognition and 98.00% grasp success while the complete embedded board draws approximately 0.80~W.
Jiajian Zheng, Bin Zhang, Wen Fan +1
School of Microelectronics Shanghai University Shanghai, 201800, China · Department of Bioengineering Imperial College London London SW7 2AZ, United Kingdom