Organizations: School of Engineering Mathematics and Technology, University of Bristol, UK. · Graduate School of Data Science, Kyungpook National University, Daegu, South Korea. · Department of Electrical and Information Engineering, Seoul National University of Science and Technology, Seoul, Republic of Korea.
Accurate modelling of soft tactile sensors remains difficult because finite element methods (FEM) are computa- tionally expensive, while real-time simulators often lack fidelity for hyperelastic deformation. Large physical datasets are also impractical to collect because repeated testing can damage fragile sensors. We present a data-efficient framework that combines limited real-world validation, high-fidelity FEM, and graph neural networks (GNNs) for accurate real-time prediction of two-dimensional tactile deformation in a planar soft tactile structure. The key idea is to decouple validation from training: 82 camera-tracked contact configurations are used only to tune and validate an ANSYS FEM model, which then generates 768 synthetic contact scenarios for GNN training. We evaluate both forward prediction from marker deformation to contact localisation and inverse inference from contact configuration to marker deformation. The validated FEM achieves sub-0.5 mm error in more than 90% of validation cases, while the learned GNN surrogate reaches a mean absolute error of 0.32 mm with millisecond-scale inference, compared with minute-scale FEM simulation. Models trained entirely on synthetic FEM data retain sub-millimetre accuracy on real measurements from the 2D tactile structure, demonstrating strong sim-to-real transfer within the tested planar setting. We further integrate the learned model into a real-time visualisation pipeline, enabling rapid deformation prediction for virtual prototyping and simulator-based design exploration without extensive physical testing.
Figures & tables
Fig. 1 : (1) Experimental setup of the tactile sensing system with a compliant TPU pin structure constrained between rigid supports and loaded to an indentation depth of 14 mm. Marker M0 denotes the indenter marker, while markers M1–M7 denote the pin markers, identifiable as white circular markers. (2) Corresponding ANSYS simulation showing the total deformation under identical loading conditions. (3) The trained Graph Neural Network facilitates real-time inference of structural deformation based on the spatial coordinates of the indenter. (4) Visualisation of the structural deformation in a real-time robotic simulator.
Fig. 2 : Overview of the proposed framework for tactile deformation modelling and real-time visualisation. (1) Experimental setup of the tactile sensing system, consisting of a compliant TPU structure constrained between rigid supports and indented at different spatial locations (y,z). Marker M0 denotes the indenter marker, while markers M1–M7 denote the tracked pin markers. (2) Finite element model (ANSYS) validation against experimental measurements to verify deformation accuracy. (3) Generation of a synthetic dataset through 368 structured and 400 random parametric FEM simulations by varying indenter positions, extracting the y and z coordinates of the seven markers for each case. (4) Training of Graph Neural Network (GNN) models using the FEM-generated dataset to learn the mapping between indenter position and marker displacements. (5) Real-time visualisation of predicted structural deformation in MuJoCo based on GNN inference.
Range (mm)
Step
Y (mm)
[-39, -30]
(-30, 30)
[30, 39]
3
Z (mm)
[2, 6]
[2, 14]
[2, 6]
2
TABLE I: Indenter position ranges and step sizes for data collection. The origin of the coordinate system is defined at the centre of the skin surface.
Parameter
Value
Convolutional hidden layers
[64, 64, 128, 256]
Concat hidden layers
1024
FC hidden layers
[512, 256]
Hidden-layer activation
LeakyReLU
Dropout coefficient
0.2
Noise magnitude
0.5
TABLE II : GNN hyperparameter configurations.
FEM Real Val.
GNN Synthetic Val.
GNN Real Val.
Y
90.6% ≤ 0.5 mm
MAE 0.43 mm
MAE 0.80 mm
Z
93.9% ≤ 0.5 mm
MAE 0.10 mm
MAE 0.41 mm
TABLE III : Summary of main quantitative results.
Error range (%)
(Y+Z)≤0.5
Error thresholds (mm)
E≤0.3
E≤0.5
0.5<E≤1
E>1
M1
y
98.8
100
0
0
99.4
z
69.5
98.8
1.2
0
M2
y
42.7
67.1
26.8
6.1
79.9
z
68.3
92.7
7.3
0
M3
y
100
100
0
0
92.1
TABLE IV: Validation of ANSYS FEM Against Physical Measurements. Cumulative percentages of samples achieving specified displacement error thresholds in mm for individual markers (M1-M7) and aggregated results (Total). The final column reports joint Y+Z accuracy below 0.5mm threshold. N=82 validation configurations.
Pin Marker Number (#)
M1
M2
M3
M4
M5
M6
M7
Avg.
Y
Syn.
0.0608
0.1780
0.0625
0.1525
0.0611
0.1523
0.0579
0.1036
Real
0.0607
0.4916
0.0684
0.4245
0.0519
0.2006
0.0601
0.1940
Z
Syn.
0.1348
0.1378
0.1387
0.1443
0.1297
0.1456
0.1513
0.1403
Real
0.1743
0.2827
0.4612
0.5458
0.4882
0.2994
0.1732
0.3464
TABLE V : MAE results for the displacement Δpmkr (mm) of each marker along the Y and Z axes.
Fig. 3 : Scatter plots for the forward and inverse models on synthetic and real data. The forward model predicts the absolute position of the indenter marker from the absolute positions of the pin markers. The inverse model predicts the displacements of all seven pin markers from the relative positions between the indenter marker and the pin markers.
Fig. 4 : (a): Several geometric constraints, including the fixed length of the pins, angle bisector and line midpoint, define the control points on the surface of the structure. These control points are used to generate the cubic spline equations that fit the surface. (b): A number of standard cylinders arranged at equal intervals used for the visualization in MuJoCo. The number of cylinders determines the level of visual detail: low (top) and high (bottom).