Sound of Touch: Active Acoustic Tactile Sensing via String Vibrations

Xili Yi; Ying Xing; Zachary Manchester; Nima Fazeli

cs.ROFeb 18, 2026

Sound of Touch: Active Acoustic Tactile Sensing via String Vibrations

Authors: Xili Yi, Ying Xing, Zachary Manchester, Nima Fazeli

Organizations: University of Michigan · MIT

Abstract

Distributed tactile sensing remains difficult to scale over large areas: dense sensor arrays increase wiring, cost, and fragility, while many alternatives provide limited coverage or miss fast interaction dynamics. We present Sound of Touch, an active acoustic tactile-sensing methodology that uses vibrating tensioned strings as sensing elements. The string is continuously excited electromagnetically, and a small number of pickups (contact microphones) observe spectral changes induced by contact. From short-duration audio signals, our system estimates contact location and normal force, and detects slip. To guide design and interpret the sensing mechanism, we derive a physics-based string-vibration simulator that predicts how contact position and force shift vibration modes. Experiments demonstrate millimeter-scale localization, reliable force estimation, and real-time slip detection. Our contributions are: (i) a lightweight, scalable string-based tactile sensing hardware concept for instrumenting extended robot surfaces; (ii) a physics-grounded simulation and analysis tool for contact-induced spectral shifts; and (iii) a real-time inference pipeline that maps vibration measurements to contact state.

Figures & tables

Fig. 2 : (A) A contact at position $x$ with force $F$ alters the string’s effective vibrating lengths and tension, inducing characteristic shifts in modal frequencies. (B) Dual EBow-like electromagnetic drivers provide continuous excitation, while two pickup microphones capture complementary vibration responses that enable separation of contact location and force.

Fig. 3 : Resonant-frequency modulation by contact. (A, D) Theoretical trends showing asymmetric frequency shifts for position and monotonic shifts for force. (B, E) Simulated spectra reproducing these modal shifts. (C, F) Real-world measurements exhibiting complex harmonic structures and overtones.

Fig. 4 : Learning architecture. Short audio windows are transformed into spectral features and encoded by a frozen Audio Spectrogram Transformer (AST). The resulting embeddings are shared by lightweight task-specific heads for contact location, normal force, contact detection, and slip detection.

Fig. 5 : Hardware design of the string-based tactile sensor. The system integrates an aluminum frame supporting a steel guitar string, dual electromagnetic EBows for continuous excitation, and contact microphones for high-frequency vibration acquisition.

Fig. 6 : System-level audio acquisition pipeline. Audio signals from two contact microphones are independently amplified, digitized by a multi-channel audio interface, and streamed to a desktop computer for real-time processing.

Fig. 7 : Experiment setup. (A) The sensor is mounted on a KUKA LBR Med R820 robot with a WSG-50 gripper. A table-mounted ATI Gamma F/T sensor provides ground-truth force and geometry labels during data collection. (B) We collect training data using 3D printed object, and evaluate on allen key, metal tube, and wood sick.

Fig. 8 : t-SNE visualization of audio feature spaces. We compare (A) Simulated AST features, (B) Real-world AST features, and (C) Real-world FFT features across four tactile tasks. Simulated data (A) shows a highly organized structure, indicating that tactile relationships are easily learnable in ideal conditions. Real-world embeddings (B, C) are more complex due to physical phenomena like overtones and damping. AST features (B) significantly outperform FFT (C) in slip detection because the transformer architecture captures critical temporal dynamics that static spectral descriptors miss. Contact detection remains robustly separated across all modalities.

Object	MAE (mm)	$\leq$ 5 mm (%)	Pearson $r$	Condition
Plastic	2.7	87.8	0.994	Clean
	3.1	83.3	0.994	Noise
Wood	8.8	51.4	0.863	Clean
	8.6	55.7	0.857	Noise
Metal tube	5.4	70.0	0.952	Clean
	8.0	57.1	0.893	Noise

TABLE I : Contact location estimation robustness under external acoustic noise. Performance is reported using mean absolute error (MAE), percentage of predictions within 5 mm, and Pearson correlation coefficient for clean and noisy conditions. All results use AST features with a 0.1 s audio window.

Object	MAE (N)	$\leq$ 0.2 N (%)	Pearson $r$	Condition
Plastic	0.141	74.4	0.909	Clean
	0.131	75.6	0.928	Noise
Wood	0.111	84.3	0.903	Clean
	0.115	82.9	0.927	Noise
Metal tube	0.172	70.0	0.908	Clean
	0.173	65.7	0.906	Noise

TABLE II : Contact force estimation robustness under external acoustic noise. Performance is evaluated using mean absolute error (MAE), percentage of predictions within 0.2 N, and Pearson correlation coefficient for clean and noisy conditions. All results use AST features with a 0.1 s audio window.

Fig. 9 : Simulation convergence example. Two-segment string simulation for a contact at $x/L=0.35$ . (A,B) Time-domain velocity signals with envelopes indicating convergence to steady oscillation. (C,D) Spectrograms showing stable dominant frequency components. (E,F) FFT spectra computed over transient (10–50 ms) and steady-state (300–350 ms) windows.

Fig. 10 : Multiple steady-state vibration modes in real audio (no contact). Two-channel spectrograms recorded from the same hardware in nominally identical no-contact conditions. (A)–(E) show five distinct steady-state spectral patterns that can arise due to different excitation histories (e.g., after contacts at different locations/forces followed by disengagement). This multi-stability motivates robust, data-driven feature extraction rather than deterministic peak-tracking.

Tactile Sensing

Tactile sensing measures contact forces, deformation, texture, and motion to help robots perceive and act through physical interaction.

See all 138 papers

Scalable Dynamic Tactile Sensing Enabled by Passive and Flexible Acoustic Waveguides
Jun 11, 2026Guimin Long, Changhong Linghu, Chuanping Liu +2Tactile Sensing Wearable Sensing
Department of Mechanical Engineering, City University of Hong Kong, Hong Kong, China
Metallic Ultrasound Waveguides as a Distributed Tactile Sensing Platform for Contact Localization, Force Estimation, and Material Class Discrimination
Jul 2, 2026Alexandros Rosakis, Alessio Tamborini, Basile Fakhoury +2Tactile Sensing Robotic Tactile Sensing
Division of Engineering and Applied Science, Caltech, Pasadena, CA, USA
VecTile: A Low-Cost Flexible Tactile Array for 3-DoF Force Sensing
Oct 22, 2026Jian Hou, Xin Zhou, Adam J. SpiersTactile Sensing Robotic Tactile Sensing
All authors are with the Manipulation and Touch Lab, Dept. Electrical and Electronic Engineering, Imperial College London.

Jun 11, 2026J/K move · Enter open · S save

Scalable Dynamic Tactile Sensing Enabled by Passive and Flexible Acoustic Waveguides

Guimin Long, Changhong Linghu, Chuanping Liu, Ke Xu+1

Department of Mechanical Engineering, City University of Hong Kong, Hong Kong, China

Tactile Sensing Wearable Sensing

Open paper AskPDF arXiv

Artificial dynamic tactile sensing requires sensitivity, robustness, and compliance, yet existing technologies face trade-offs when scaling to large-area arrays, compounded by wiring complexity and cost. Here, we report a passive distributed paradigm using deep sub-wavelength acoustic waveguides that decouples performance from structural flexibility. Elastic-membrane-capped Helmholtz resonators interconnected by spring-reinforced microtubes form an enclosed network with invariant acoustic transmission under macroscopic bending. By sparsely embedding microphones, the system achieves real-time localization (4 mm highest spatial resolution; >99% accuracy in a 4 microphones 64-node sensing array) and waveform reconstruction of low-frequency signals (<100 Hz). Fast Continuous Wavelet Transform and a lightweight neural network enable inference within 5.5 ms. We demonstrate conformable prototypes-fingertip arrays, a tactile glove, and large-area skins-detecting stimuli from single-hair contact to 5-mg particle impacts, arterial pulse waves, feather touches, and finger contact. This establishes a scalable, flexible, low-cost paradigm for next-generation human-machine interfaces.

Abstract

Tactile Sensing

Explore similar work

Scalable Dynamic Tactile Sensing Enabled by Passive and Flexible Acoustic Waveguides

Metallic Ultrasound Waveguides as a Distributed Tactile Sensing Platform for Contact Localization, Force Estimation, and Material Class Discrimination

VecTile: A Low-Cost Flexible Tactile Array for 3-DoF Force Sensing