Organizations: Department of Electrical, Electronic, Telecommunications Engineering, and Naval Architecture (DITEN), University of Genoa, 16145 Genoa, Italy · Department of Computer and Communication Engineering, Lebanese International University (LIU), Beirut, Lebanon
Tactile sensing provides information about object properties that can improve robotic grasping and manipulation. However, accurate object recognition during the early stages of grasping remains challenging, particularly under strict real-time constraints. This study presents a flexible, distributed piezoelectric tactile sensing system for early-stage object recognition. The system was experimentally evaluated using six everyday objects and three recognition approaches: feature-based classification using support vector machines (SVMs), raw-signal classification using 1D convolutional neural networks (1D-CNNs) and convolutional recurrent neural networks (C-RNNs), and spatiotemporal heatmap classification using a 2D-CNN. The effects of sensor count and temporal window size on recognition accuracy and real-time feasibility were systematically investigated. Increasing the number of sensing elements improved recognition performance across all approaches. The best-performing method, based on spatiotemporal heatmaps and a 2D-CNN, achieved F1-scores of 0.9796, 0.9846, and 0.9903 using 16, 24, and 32 sensors, respectively. The optimal configuration, comprising 32 sensors and a 100-ms temporal window (50 samples), was deployed on an embedded edge platform, achieving an end-to-end recognition latency of approximately 101 ms from contact onset and an energy consumption of 63.8 microjoules per inference. These results demonstrate the potential of distributed piezoelectric tactile sensing for accurate, energy-efficient, and real-time object recognition during robotic grasping.
Figures & tables
References
Sensor type
Mechanical structure
Number of sensors
Spatial resolution
Application
Hardware deployment
Response time
[ 4 ]
Piezoresistive
Flexible
64
2.5 mm
Object recognition
✗
✗
[ 5 ]
Piezoresistive
–
1
27.6 taxels/cm 2
Material classification
✗
103 ms (GPU)
[ 6 ]
Piezoresistive
Rigid
256
3.5 mm
Object recognition
✗
✗
[ 7 ]
Capacitive
Flexible
–
–
Object recognition
✗
✗
[ 8 ]
Capacitive
Flexible
64
2 mm
Material recognition
✗
✗
[ 9 ]
Piezoelectric
Flexible
1
–
Object surface recognition
✗
✗
TABLE I: Overview of most relevant existing work on tactile-based object recognition.
Fig. 1: (a) Top: Single sensor structure and Bottom: layout of the sensing array. (b) Development procedure of the sensing caps (c), System block diagram (d), Sensing system mounted on the Tiago gripper.
Sensitivity
Linearity
SINAD
1.5–31.96 pC/N
2.3% nonlinearity error
56 dB
TABLE II: Sensor characterization metrics for the proposed sensing system.
Fig. 2: Channel-to-channel consistency of three sensors. The charge response is shown as a function of the applied force for sensors S5, S11, and S1.
Fig. 3: (a) Response of a single sensor to a sine stimulus at variable frequencies (1 Hz - 100 Hz). (b) Response of a single sensor to a square stimulus at variable forces (0.12 N - 6 N).
Fig. 4: (a) Set of daily-use objects, (b) TIAGo robot manipulating a PB, and (c) the corresponding sensor response.
Algorithm 1 Energy Computation
Fig. 6: Block diagram of object recognition approaches: End-to-end (EEA), Feature-Based Approach (FBA), and Heatmap Approach (HMA).
Sensor Group
Approach
Nc=16
Nc=24
Nc=32
EEA
Dg3W
Dg4W
DW
FBA
D~g3W
D~g4W
D~W
HMA
Hg3W
Hg4W
HW
TABLE III: Summary of the datasets
Algorithm 2 HeatMap Construction
Fig. 7: Illustration of the heatmap constructed from the tactile signals with the Dataset H50 .
Algorithm
Hyperparameter
Values
CNN
F (Filters)
{8,16,32,64}
K (Kernel Size)
{2,3,4}
SVM
C (Penalty)
{0.01,0.1,1,10,100}
γ (Kernel Coefficient)
{0.01,0.1,1,10,100}
C-RNN
F (Filters)
{8,16,32,64}
K (Kernel Size)
{2,3,4}
TABLE IV: Summary of Hyperparameter Configurations for Each Algorithm
Fig. 8: Radar plots presenting the performance metrics of the implemented algorithms across different window size ( W ) and sensor groups ( NC ).
Approach
Dataset
F1-score
EEA
Dg350
0.8824
Dg450
0.9102
D50
0.9532
FBA
D~g350
0.9445
D~g450
0.9604
D~50
0.9661
TABLE V: Table presenting the highest F1-score values achieved with W =50, by the three approaches including C-RNN, SVM, and 2D-CNN across the three datasets.
Fig. 9: Performance of the implemented models in each of the three approaches for the three sensor groups using the smallest window size ( W=50 ).
Dataset
H50
Inference time (ms)
0.8817
Flash Size (kB)
7.65
kFLOPs
11.3
Power (mW)
72.3
Energy (µJ)
63.8
TABLE VI: Performance Metrics of the Best Performing Model (2D-CNN)
School of Microelectronics Shanghai University Shanghai, 201800, China · Department of Bioengineering Imperial College London London SW7 2AZ, United Kingdom