Tactile Recognition of Both Shapes and Materials with Automatic Feature Optimization-Enabled Meta Learning

Hongliang Zhao; Wenhui Yang; Yang Chen; Zhuorui Wang; Baiheng Liu; Longhui Qin

cs.ROMar 9, 2026

Tactile Recognition of Both Shapes and Materials with Automatic Feature Optimization-Enabled Meta Learning

Authors: Hongliang Zhao, Wenhui Yang, Yang Chen, Zhuorui Wang, Baiheng Liu, Longhui Qin

Organizations: School of Mechanical Engineering, Southeast University, Nanjing 211189, China. · State Key Laboratory of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing, 10081, China · State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin, 150001, China.

Abstract

Tactile perception is indispensable for robots to implement various manipulations dexterously, especially in contact-rich scenarios. However, alongside the development of deep learning techniques, it meanwhile suffers from training data scarcity and a time-consuming learning process in practical applications since the collection of a large amount of tactile data is costly and sometimes even impossible. Hence, we propose an automatic feature optimization-enabled prototypical network to realize meta-learning, i.e., AFOP-ML framework. As a ``learn to learn" network, it not only adapts to new unseen classes rapidly with few-shot, but also learns how to determine the optimal feature space automatically. Based on the four-channel signals acquired from a tactile finger, both shapes and materials are recognized. On a 36-category benchmark, it outperforms several existing approaches by attaining an accuracy of 96.08% in 5-way-1-shot scenario, where only 1 example is available for training. It still remains 88.7% in the extreme 36-way-1-shot case. The generalization ability is further validated through three groups of experiment involving unseen shapes, materials and force/speed perturbations. More insights are additionally provided by this work for the interpretation of recognition tasks and improved design of tactile sensors.

Figures & tables

Fig. 1: Conceptual overview of the proposed AFOP-ML framework. Multi-channel tactile signals produced by a tactile finger are firstly leveraged to learn how to optimize the feature space automatically and predict correct shape and material category as a source task. Then, these knowledge is transferred to the AFOP-ML model to recognize new shapes and materials under normal or changed experimental conditions as a target task with few-shot data available. Features that are fed into the prototypical network are automatically determined for different recognition tasks.

Fig. 2: Tactile sensor and 36 categories to recognize. (a) Bio-inspired tactile finger mainly consists of rigid phalanx, soft skin, PDMS support, fingerprint, fixture and two types of SEs: PVDFs and SGs. Two PVDFs and two SGs constitute Channel 1 to 4. (b) The 36 categories to recognize. Three materials: Resin, Wood and Aluminum. There are 12 shapes for each material.

Fig. 3: The framework of proposed AFOP-ML algorithm. (a) Offline feature selection. Four synchronous channels are converted to a 386-dimensionality feature pool per trial. (b) Episode-time adaptation. For each

N

-way-

K

-shot task, support sets are mapped to Top-

D

and averaged into class prototypes

W

. Prototypes and queries are

\ell_{2}

-normalized; cosine similarities are temperature-scaled (

\alpha

) and shifted by class biases

b

, then fed to Softmax. Cross-entropy with entropy regularization updates only

(W,b)

on the support set; queries are forward-only. (c) Features in frequency domain (192-D). (d) Features in time domain (194-D).

Fig. 4: Automatic determination of the optimal dimensionality of feature space,

D

. 5‑way-5‑shot accuracy vs. the number of selected features (episodic mean ± 95% confidence intervals (CI) across splits).

		1-Shot Acc (%)					3-Shot Acc (%)			5-Shot Acc (%)			Time Cost (5-way-5-shot)
		5 Way	10 Way	12 Way	28 Way	36 Way	5 Way	12 Way	36 Way	5 Way	12 Way	36 Way	Pretrain	Adapt/episode (ms)
Meta-learning (episodic)	AFOP-ML (Ours)	96.08	94.69	94.00	90.32	88.74	98.69	97.23	93.65	98.76	97.63	94.56	$\sim$ 2 s	391.04
	AFO-MLP-ML [ 21 ]	93.47	90.09	87.82	75.79	69.64	97.99	95.56	85.78	98.82	96.25	86.22	$\sim$ 2 s	10241.72
	Direct-Prot-ML	92.05	88.79	86.18	80.92	78.47	96.82	93.56	87.89	97.22	95.20	89.87	$\sim$ 2 s	545.65
	MAML [ 22 ]	93.28	86.70	84.38	73.99	70.77	94.88	88.61	63.89	95.37	89.32	68.06	$\sim$ 20 min	72.87
	CWT-ResNet-ML [ 16 ]	95.67	92.47	91.13	86.18	84.24	98.95	97.59	93.52	99.27	98.55	96.15	$\sim$ 8 min	1428.30

TABLE I: CLOSED-SET ACCURACY AND EFFICIENCY COMPARISON

Fig. 5: Generalization performance in 1-shot task. (a) Cross‑shape, (b) Cross‑material, and (c) Force–speed perturbations. Shaded bands show the variations within 95% CI across splits (no split for perturbations).

Fig. 6: Adaptive determination of feature dimensionality ( $D$ ) and SE contributions in each groups of task. Median number of selected features $D$ (bars, left vertical axis) and the proportion of PVDF-SG-based features among the selected set (line, right vertical axis) across four settings: Shape, Closed-set, Force&Speed, and Material.

Fig. 7: t‑SNE of the closed‑set features ( $D=8$ , 5-way-5-shot); marker encodes shape (12 class‑specific markers) and color encodes material. Shape clusters are compact and materials are well mixed within each shape cluster. Quantitative indicators: 1‑NN=0.982 (nearest‑neighbour shape accuracy), mix‑sil=0.840 (0–1 material‑mixing score; the higher is better), DGI=4.769 (ratio of cross‑material to same‑material neighbors; $>1$ indicates strong mixing). t-SNE is used only for qualitative visualization; overlaps in 2D do not necessarily imply misclassification in the original Top- $D$ space, where queries are classified by cosine similarities to class prototypes.

Explore similar work

How to Train your Tactile Model: Tactile Perception with Multi-fingered Robot Hands
Apr 1, 2026Christopher J. Ford, Kaichen Shi, Laura Butcher +2Tactile Sensing Tactile Sensing for Robotic Manipulation
School of Engineering Mathematics and Technology and Bristol Robotics Laboratory, University of Bristol, UK
An Interpretable Deep Learning Framework for Material Perception and Classification from Multisensory Tactile Data
Aug 22, 2026Li Zou, Dave Hogendoorn, Yasemin VardarTactile Sensing Multimodal Classification
Department of Cognitive Robotics, Faculty of Mechanical Engineering, Delft University of Technology, Delft, The Netherlands.
TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception
Jun 24, 2026Lan Wei, Gurmeher Khurana, Sirine Bhouri +6Vision-Based Tactile Sensing Few-Shot Learning
Imperial College London, UK. · Queen Mary University of London, UK · King’s College London, UK

Apr 1, 2026J/K move · Enter open · S save

How to Train your Tactile Model: Tactile Perception with Multi-fingered Robot Hands

Christopher J. Ford, Kaichen Shi, Laura Butcher, Nathan F. Lepora+1

School of Engineering Mathematics and Technology and Bristol Robotics Laboratory, University of Bristol, UK

Tactile Sensing Tactile Sensing for Robotic Manipulation Vision-Based Tactile Sensing Vision Transformer

Open paper AskPDF arXiv

Rapid deployment of new tactile sensors is essential for scalable robotic manipulation, especially in multi-fingered hands equipped with vision-based tactile sensors. However, current methods for inferring contact properties rely heavily on convolutional neural networks (CNNs), which, while effective on known sensors, require large, sensor-specific datasets. Furthermore, they require retraining for each new sensor due to differences in lens properties, illumination, and sensor wear. Here we introduce TacViT, a novel tactile perception model based on Vision Transformers, designed to generalize on new sensor data. TacViT leverages global self-attention mechanisms to extract robust features from tactile images, enabling accurate contact property inference even on previously unseen sensors. This capability significantly reduces the need for data collection and retraining, accelerating the deployment of new sensors. We evaluate TacViT on sensors for a five-fingered robot hand and demonstrate its superior generalization performance compared to CNNs. Our results highlight TacViTs potential to make tactile sensing more scalable and practical for real-world robotic applications.

Abstract

Explore similar work

How to Train your Tactile Model: Tactile Perception with Multi-fingered Robot Hands

An Interpretable Deep Learning Framework for Material Perception and Classification from Multisensory Tactile Data

TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception