Organizations: School of Engineering Mathematics and Technology, University of Bristol, UK. · Graduate School of Data Science, Kyungpook National University, Daegu, South Korea. · Department of Electrical and Information Engineering, Seoul National University of Science and Technology, Seoul, Republic of Korea.
Accurate modelling of soft tactile sensors remains difficult because finite element methods (FEM) are computa- tionally expensive, while real-time simulators often lack fidelity for hyperelastic deformation. Large physical datasets are also impractical to collect because repeated testing can damage fragile sensors. We present a data-efficient framework that combines limited real-world validation, high-fidelity FEM, and graph neural networks (GNNs) for accurate real-time prediction of two-dimensional tactile deformation in a planar soft tactile structure. The key idea is to decouple validation from training: 82 camera-tracked contact configurations are used only to tune and validate an ANSYS FEM model, which then generates 768 synthetic contact scenarios for GNN training. We evaluate both forward prediction from marker deformation to contact localisation and inverse inference from contact configuration to marker deformation. The validated FEM achieves sub-0.5 mm error in more than 90% of validation cases, while the learned GNN surrogate reaches a mean absolute error of 0.32 mm with millisecond-scale inference, compared with minute-scale FEM simulation. Models trained entirely on synthetic FEM data retain sub-millimetre accuracy on real measurements from the 2D tactile structure, demonstrating strong sim-to-real transfer within the tested planar setting. We further integrate the learned model into a real-time visualisation pipeline, enabling rapid deformation prediction for virtual prototyping and simulator-based design exploration without extensive physical testing.
Figures & tables
Fig. 1 : (1) Experimental setup of the tactile sensing system with a compliant TPU pin structure constrained between rigid supports and loaded to an indentation depth of 14 mm. Marker M0 denotes the indenter marker, while markers M1–M7 denote the pin markers, identifiable as white circular markers. (2) Corresponding ANSYS simulation showing the total deformation under identical loading conditions. (3) The trained Graph Neural Network facilitates real-time inference of structural deformation based on the spatial coordinates of the indenter. (4) Visualisation of the structural deformation in a real-time robotic simulator.
Fig. 2 : Overview of the proposed framework for tactile deformation modelling and real-time visualisation. (1) Experimental setup of the tactile sensing system, consisting of a compliant TPU structure constrained between rigid supports and indented at different spatial locations (y,z). Marker M0 denotes the indenter marker, while markers M1–M7 denote the tracked pin markers. (2) Finite element model (ANSYS) validation against experimental measurements to verify deformation accuracy. (3) Generation of a synthetic dataset through 368 structured and 400 random parametric FEM simulations by varying indenter positions, extracting the y and z coordinates of the seven markers for each case. (4) Training of Graph Neural Network (GNN) models using the FEM-generated dataset to learn the mapping between indenter position and marker displacements. (5) Real-time visualisation of predicted structural deformation in MuJoCo based on GNN inference.
Range (mm)
Step
Y (mm)
[-39, -30]
(-30, 30)
[30, 39]
3
Z (mm)
[2, 6]
[2, 14]
[2, 6]
2
TABLE I: Indenter position ranges and step sizes for data collection. The origin of the coordinate system is defined at the centre of the skin surface.
Parameter
Value
Convolutional hidden layers
[64, 64, 128, 256]
Concat hidden layers
1024
FC hidden layers
[512, 256]
Hidden-layer activation
LeakyReLU
Dropout coefficient
0.2
Noise magnitude
0.5
TABLE II : GNN hyperparameter configurations.
FEM Real Val.
GNN Synthetic Val.
GNN Real Val.
Y
90.6% ≤ 0.5 mm
MAE 0.43 mm
MAE 0.80 mm
Z
93.9% ≤ 0.5 mm
MAE 0.10 mm
MAE 0.41 mm
TABLE III : Summary of main quantitative results.
Error range (%)
(Y+Z)≤0.5
Error thresholds (mm)
E≤0.3
E≤0.5
0.5<E≤1
E>1
M1
y
98.8
100
0
0
99.4
z
69.5
98.8
1.2
0
M2
y
42.7
67.1
26.8
6.1
79.9
z
68.3
92.7
7.3
0
M3
y
100
100
0
0
92.1
TABLE IV: Validation of ANSYS FEM Against Physical Measurements. Cumulative percentages of samples achieving specified displacement error thresholds in mm for individual markers (M1-M7) and aggregated results (Total). The final column reports joint Y+Z accuracy below 0.5mm threshold. N=82 validation configurations.
Pin Marker Number (#)
M1
M2
M3
M4
M5
M6
M7
Avg.
Y
Syn.
0.0608
0.1780
0.0625
0.1525
0.0611
0.1523
0.0579
0.1036
Real
0.0607
0.4916
0.0684
0.4245
0.0519
0.2006
0.0601
0.1940
Z
Syn.
0.1348
0.1378
0.1387
0.1443
0.1297
0.1456
0.1513
0.1403
Real
0.1743
0.2827
0.4612
0.5458
0.4882
0.2994
0.1732
0.3464
TABLE V : MAE results for the displacement Δpmkr (mm) of each marker along the Y and Z axes.
Fig. 3 : Scatter plots for the forward and inverse models on synthetic and real data. The forward model predicts the absolute position of the indenter marker from the absolute positions of the pin markers. The inverse model predicts the displacements of all seven pin markers from the relative positions between the indenter marker and the pin markers.
Fig. 4 : (a): Several geometric constraints, including the fixed length of the pins, angle bisector and line midpoint, define the control points on the surface of the structure. These control points are used to generate the cubic spline equations that fit the surface. (b): A number of standard cylinders arranged at equal intervals used for the visualization in MuJoCo. The number of cylinders determines the level of visual detail: low (top) and high (bottom).
Accurate soft tissue simulation is essential for surgical training, pre-operative planning, and haptic feedback systems. While learning-based surrogate models trained on data using the finite element method (FEM) offer a promising path to real-time inference, their reliability depends on well-calibrated constitutive models. Existing approaches neither provide systematic guidance on model selection across stiffness levels, nor generalize across different tissue stiffnesses or geometries. We perform a comprehensive calibration of hyperelastic constitutive models in the SOFA Framework using gravity-loaded silicone beams with different stiffnesses. Using calibrated simulations as training data, we use a softness conditioned equivariant graph neural network, enabling deformation and force prediction across multiple tissue types and unseen geometries. Our model achieves sub-millimeter mean deformation accuracy at 0.010s inference time, while showing that force prediction quality is directly tied to upstream calibration consistency.
Madina Kojanazarova, Sidaty El Hadramy, Philippe C. Cattin
University of Basel, Department of Biomedical Engineering, Allschwil, Switzerland
Tactile perception is key to dexterous manipulation, yet simulating high-resolution elastomer deformation remains computationally prohibitive. Finite element methods (FEM) deliver high fidelity but demand costly remeshing, while Material Point Methods (MPM) suffer from heavy particle-memory tradeoffs. We propose a {reduced-order neural simulation framework} that couples coarse-grained MPM dynamics with an implicit neural decoder to reconstruct sub-particle tactile details from compact latent states. The framework learns a continuous deformation manifold from paired high- and low-resolution simulations, enabling physically consistent, differentiable inference. Compared to the TacIPC, our method achieves over 65% faster simulation and {40% lower memory usage}, while maintaining better geometric fidelity. In tactile rendering and 3D surface reconstruction, our methods further improve accuracy by 25% and produce realistic depth images and surface mesh within a faster inference speed. These results demonstrate that the proposed reduced-order neural model enables high-detail, physically grounded tactile simulation with substantial efficiency gains for robotic interaction and optimization.
Yuhu Guo, Zhikai Shen, Jiasheng Qu +4
The University of Manchester, UK · The Chinese University of Hong Kong, Hong Kong SAR, China · University of Leeds, UK +1
Vision-based tactile sensors require high-fidelity simulation for reinforcement learning, yet existing approaches struggle to provide accurate mechanical stress fields within GPU-accelerated robotics platforms. We present TaCauchy, an extensible Finite Element Method (FEM) framework that integrates rigorous physics-based force computation into Isaac Sim. Built on the Unified Incremental Potential Contact (UIPC) solver, TaCauchy directly computes Cauchy stress tensors from hyperelastic constitutive laws and projects them onto contact surfaces to obtain traction forces and pressure distributions, providing mechanical ground truth from first principles rather than empirical estimation. Our framework features automatic mesh generation with geometry-aware adaptive refinement and a modular sensor interface enabling rapid integration of diverse sensors (GelSight Mini, DIGIT, 9DTact) with minimal configuration. Performance benchmarks demonstrate 33.40 FPS for single environments and 555 FPS aggregate throughput across 60 parallel environments, with stress extraction overhead under 1 ms. Physical validation experiments show strong agreement between simulated and real tactile responses across force ranges from 1.2556 N to 4.7332 N, achieving SSIM above 0.93, confirming the framework's capability to provide accurate, physically-grounded force supervision for downstream robotic manipulation tasks.
Hengfei Zhao, Yifan Xie, Junhao Gong +6
Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China · Huawei Inc., Shenzhen, China