Model-Free Co-Optimization of Manufacturable Sensor Layouts and Deformation Proprioception
Authors: Yingjun Tian, Guoxin Fang, Aoran Lyu, Xilong Wang, Zikang Shi, Yuhu Guo, Weiming Wang, Charlie C. L. Wang
Organizations: Department of Mechanical and Aerospace Engineering, The University of Manchester, United Kingdom · Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong
Flexible sensors are increasingly employed in soft robotics and wearable devices to provide proprioception of freeform deformations.Although supervised learning can train shape predictors from sensor signals, prediction accuracy strongly depends on sensor layout, which is typically determined heuristically or through trial-and-error. This work introduces a model-free, data-driven computational pipeline that jointly optimizes the number, length, and placement of flexible length-measurement sensors together with the parameters of a shape prediction network for large freeform deformations. Unlike model-based approaches, the proposed method relies solely on datasets of deformed shapes, without requiring physical simulation models, and is therefore broadly applicable to diverse robotic sensing tasks. The pipeline incorporates differentiable loss functions that account for both prediction accuracy and manufacturability constraints. By co-optimizing sensor layouts and network parameters, the method significantly improves deformation prediction accuracy over unoptimized layouts while ensuring practical feasibility. The effectiveness and generality of the approach are validated through numerical and physical experiments on multiple soft robotic and wearable systems.
Figures & tables
Fig. 1: For a pneumatic actuated deformable mannequin (a), the body shapes of different individuals, as shown in (b), can be realized and controlled with the help of stretchable sensors. When the sensor layout is determined by the intuitive design of experts in the garment industry – i.e., heuristic consideration as the dimensions of girth measurement as shown in (c), large errors are observed in the shape predicted by these sensors (10 sensors with a total length of 1,204 mm). After applying the optimization approach presented in this paper, the error of shape prediction can be effectively reduced when using the same number of sensors but different layout as shown in (d) – the total length reduced to 609 mm. The color map for shape approximation error on the models with maximal error are shown on the right of (c) and (d). The statistical comparisons of the shape prediction errors by using different sensor layouts can be found in (e) and (f) for the maximal errors and the average errors on 3,000 different shapes.
Fig. 2: The deformable freeform surface is parameterized by fitting a B-spline surface, defining a u,v -domain used as the design space for sensor layout optimization. Given the start point (us,vs) and the end point (ue,ve) , each sensor is represented as a line subdivided into (K−1) segments, which are mapped onto the deformed 3D surface to compute its current length under deformation.
Fig. 3: Overview of our computational pipeline for co-optimizing (i) the sensor layout, parameterized by sensor locations L in the u,v -domain and the occupancy variable b (see (a) for the sensor layout module), and (ii) the shape prediction neural network Np , parameterized by its coefficients θp . Co-optimization is performed using a learning routine based on backpropagation over a training dataset of models with varying shapes Θ={Sc} . The optimization is guided by loss functions that account for both shape prediction error and constraints related to design and manufacturability (see (b) for the shape estimation module). In the context of wearable applications, the input shapes are selected from regions-of-interest (RoI) on the human body.
Fig. 4: Possible intersection scenarios between Sensor s1 (AB) and Sensor s2 (CD), represented as straight line segments, include: (a, b) one sensor lies entirely on the same side of the other – no intersection occurs; and (c) the endpoints of each sensor lie on opposite sides of the other, indicating a potential intersection.
Fig. 5: Illustration of computing the minimum distance between two sensors: (a) each sensor is first uniformly subdivided into K points, denoted as {(uη,vη)} and {(uξ,vξ)} , with η and ξ as the respective indices, and (b) the minimum Euclidean distance between the two sensors is obtained by exhaustively computing the pairwise distances between all points from the two sets mapped onto the surface.
Component
What is computed
Time Complexity
Notes / Dependencies
Sensor length
Compute sensor length using K sample points on each sensor (Eqs. ( 2 )-( 4 )).
O(K)
O(1) complexity for each point due to B-spline’s local support formulation.
Reconstruction loss Lrecon
MSE evaluation over m×n control points of B-spline surface (Eq. ( 8 )).
O(mn)
Linear to the total number of control points.
Overlap avoidance Loverlap
Pairwise intersection check among active sensors (Eqs. ( 9 )-( 10 )).
O(M2)
Pairwise checks in the (u,v) domain for Ma active sensors.
Inter-sensor distance LminSpace
Soft-min distance between sensor pairs by sample points (Eqs. ( 11 )-( 13 )).
O(M2K2)
For each sensor pair, compute distances between K2 pairs of samples.
Length control LminL , LtotalL
Total length regularization and minimal length penalty (Eq. ( 14 )-( 15 )).
O(MK)
Similar to the complexity for evaluating the lengths of K sensors.
Shape prediction by Np(⋅)
Forward computation of NN that maps sensor signals to the freeform shape as B-spline surface’s control points.
O((M+mn)H+LH2)
O(MH) for the input layer, O(LH2) for the hidden layers, and O(mnH) for the output layer predicting m×n control pnts.
TABLE I: Complexity analysis of the proposed co-optimization pipeline. †
Fig. 6: Two hardware setups used to validate the effectiveness of our co-optimization pipeline: (a) a soft manipulator consisting of two segments with a total of six pneumatic actuation chambers, and (b) a wearable device positioned over the shoulder region for sensing shoulder motion.
Fig. 7: Sensor and skin fabrication: (a) a sensor layout represented as a collection of curves, (b) offsetting curves to create sensor tracks, (c) connecting curves to form channels for liquid metal filling, (d) mold for sensor fabrication, (e) liquid metal injection and electrode wiring, (f) mold for skin fabrication, (g) mold with installation slots for placing sensors according to the optimized layout, and (h) final fabricated skin with integrated sensors.
Fig. 8: Distribution of the B-spline surface fitting errors for the dataset of deformable mannequin. While the mean error (0.116 mm) and the maximum error (0.208 mm) are indicated by the arrows, the distribution has a very small standard deviation (0.006 mm).
Fig. 9: Optimization process of sensor layout on the soft manipulator based on the dataset with deformation tracked by the markers, where the convergence curves of the reconstruction loss (in red ), the overlap avoidance loss (in blue ), and the number of sensors (in purple ) are given. The initial and the optimized sensor layouts, corresponding to the starting and converged states of the optimization, are highlighted.
Fig. 10: Results of sensor layout optimization with 6 sensors, compared to unoptimized layouts with 20 and 6 sensors – i.e., with only the shape prediction network Np trained: (a) sensor layouts and color maps showing the approximation errors on the worst cases, (b) distributions of maximum errors over 400 test samples, and (c) distributions of average errors.
Fig. 11: Physical experiments on soft manipulation comparing proprioception errors with optimized and unoptimized 6-sensor layouts: (a) sensor layouts and (b) distributions of maximum and average shape prediction errors.
Fig. 12: The co-optimization taken on a shoulder deformation sensing wearable, where the convergence curves of the reconstruction loss (in red ), the overlap avoidance loss (in blue ), and the number of sensors (in purple ) are displayed.
Fig. 13: Results of sensor layout optimization with 4 sensors, compared to unoptimized layouts with 20 and 4 sensors – i.e., with only the shape prediction network Np trained: (a) sensor layouts and (b) distributions of average and maximum errors over 400 test samples.
Fig. 14: Comparison of deformation sensing during shoulder motion using an unoptimized sensor layout vs. an optimized sensor layout – both have 4 sensors. Shape prediction errors were evaluated over 1,000 poses, with ground-truth deformations captured by MoCap.
Fig. 15: Progressive results of sensor layout optimization starting from a random layout of 20 sensors, converging to a final layout of 10 short, spaced, and non-intersecting sensors. The right panel shows the convergence curves of the reconstruction loss ( red ), overlap avoidance loss ( blue ), and the number of sensors ( purple ).
Fig. 16: Results of sensor optimization on the deformable mannequin in its UV-domain (a) from a random initial layout with 20 sensors and (b) from an expert-designed initial layout with 10 sensors. The distributions of corresponding shape prediction errors are given in (c).
Layout
Sensor Num.
Total Length †
Max Shape Err. ‡
Trial ID
Before
After
Before
After
Before
After
1
20
10
3018.58
610.09
2.44
1.07
2
20
10
3460.06
666.04
3.10
1.11
3
20
10
3911.56
674.36
5.33
1.28
4
20
11
3910.27
718.60
3.21
1.43
5
20
10
3149.73
624.34
2.82
0.96
TABLE II: Study of Sensor Layout Initialization
Fig. 17: Ablation studies validating the necessity of each loss term. (a) Sensor layouts obtained with different combinations of loss terms. Convergence curves for these combinations are shown for (b) number of sensors, (c) length of the shortest sensor, (d) number of intersections, and (e) the maximum value of LminSpace across all sensor pairs.
Fig. 18: Ablation study of LminSpace and Loverlap : (a) an initial random layout with 20 sensors, (b) the result with only Lrecon and LminSpace where intersection cannot be completely removed due to topological obstacles, (c) the result with only Lrecon and Loverlap shows no intersection, and (d) the result by using both Loverlap and LminSpace together with Lrecon .
Fig. 19: When only LminSpace is used together with Lrecon , the optimization becomes trapped by topological obstacles, manifested as two excessively long sensors, as shown in (B) and (C) after 100 and 200 epochs, respectively. Starting from (C), the optimization cannot escape this local minimum, either by (i) switching from LminSpace to Loverlap (light-blue optimization path, resulting in (D)) or by (ii) adding Loverlap to work together with LminSpace (red optimization path, resulting in (E)). In contrast, when all loss terms are applied from the beginning, the optimizer successfully converges to an intersection-free layout, as shown in (F).
Fig. 20: Study on whether symmetry in sensor layout needs to be explicitly enforced. (a, b) Method I: sensors restricted to one side. (c, d) Method II: symmetric distribution enforced. (e) Maximum and average shape prediction errors over 400 test shapes for layouts generated by these two methods and the free optimization method.
This paper presents the application and experimental evaluation of a systematic method for optimal sensor placement in soft robots. Existing methods either lack generalizability across different soft robot morphologies or do not account for system dynamics. The applied method uses convex optimization to find the optimal sensor configuration that maximizes an observability Gramian-based metric. The framework is experimentally evaluated using position and strain measurements on a soft continuum arm. Kalman filter state estimates using optimal sensor placements yield lower reconstruction error than a baseline across all sinusoidal input trials, with improvements on the order of millimeters. This case study shows that linear control theory tools can guide optimal sensor placement in soft robots, suggesting an interpretable approach to sensor placement that may extend to other morphologies.
Samuel Smocot, James Richard Forbes, Audrey Sedal
Dept. of Mechanical Engineering, McGill University, Montreal, Canada. · Mila – Québec AI Institute, Montreal, Canada.
We present a zero-shot deformation reconstruction framework for soft robots that operates without any visual supervision at inference time. In this work, zero-shot deformation reconstruction is defined as the ability to infer object-wide deformations on previously unseen soft robots without collecting object-specific deformation data or performing any retraining during deployment. Our method assumes access to a static geometric proxy of the undeformed object, which can be obtained from a STL model. During operation, the system relies exclusively on tactile sensing, enabling camera-free deformation inference. The proposed framework integrates a flexible piezoresistive sensor array with a geometry-aware, cage-based 3D Gaussian deformation model. Local tactile measurements are mapped to low-dimensional cage control signals and propagated to dense Gaussian primitives to generate globally consistent shape deformations. A graph attention network regresses cage displacements from tactile input, enforcing spatial smoothness and structural continuity via boundary-aware propagation. Given only a nominal geometric proxy and real-time tactile signals, the system performs zero-shot deformation reconstruction of unseen soft robots in bending and twisting motions, while rendering photorealistic RGB in real time. It achieves 0.67 IoU, 0.65 SSIM, and 3.48 mm Chamfer distance, demonstrating strong zero-shot generalization through explicit coupling of tactile sensing and structured geometric deformation.
Linrui Shou, Zilang Chen, Wenjia Xu +2
University of Notre Dame · University of Washington
Real-time prediction of deformation in highly compliant soft materials remains a significant challenge in soft robotics. While vision-based soft tactile sensors can track internal marker displacements, learning-based models for 3D contact estimation heavily depend on their training datasets, inherently limiting their ability to generalize to complex scenarios such as multi-point sensing. To address this limitation, we introduce TactiVerse, a U-Net-based framework that formulates contact geometry estimation as a spatial heatmap prediction task. Even when trained exclusively on a limited dataset of single-point indentations, our architecture achieves highly accurate single-point sensing, yielding a superior mean absolute error of 0.0589 mm compared to the 0.0612 mm of a conventional regression-based CNN baseline. Furthermore, we demonstrate that augmenting the training dataset with multi-point contact data substantially enhances the sensor's multi-point sensing capabilities, significantly improving the overall mean MAE for two-point discrimination from 1.214 mm to 0.383 mm. By successfully extrapolating complex contact geometries from fundamental interactions, this methodology unlocks advanced multi-point and large-area shape sensing. Ultimately, it significantly streamlines the development of marker-based soft sensors, offering a highly scalable solution for real-world tactile perception.
Junhui Lee, Hyosung Kim, Saekwang Nam
Graduate School of Data Science at Kyungpook National University, 41566 Daegu, Republic of Korea · Major in Data Science Convergence