Single-View RGB-D Force--Deformation Estimation for Robotic Manipulation of Elasto-Plastic Materials
Organizations: Robotics and Mechatronics, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, The Netherlands.
Abstract
Persistent deformation complicates elasto-plastic manipulation by coupling physical interaction with spatially distributed shape changes. We investigate the force--deformation relationship using a standard robot, a single eye-in-hand RGB-D camera, and intrinsic force estimates. A 7-DoF Franka Emika FR3 performs normal indentations while an Intel RealSense L515 captures pre- and post-contact surfaces. Point-cloud processing localizes deformation and estimates displaced volume. Force--volume characterization includes 27 measurements over nine force levels. A compact multilayer perceptron (MLP), trained on 52 force--volume pairs at four levels, achieves a held-out RMSE of 1.857 N and . A separate comparison shows lower error for a quadratic model at 16 N and for the MLP at 19 and 23.5 N. An eight-step experiment at 4 N per interaction demonstrates cumulative deformation localization from the second indentation onward. These results establish a practical measurement basis for deformation-aware manipulation using intrinsic force estimates and single-view geometry.
Figures & tables
| Protocol | Design | Evaluation role |
|---|---|---|
| Characterization | 5–25 N, 2.5 N steps; 3/level (27) | Force–volume response and quadratic fit |
| MLP | 15, 17.5, 20, 22.5 N; 13/level (52) | Grouped 70/15/15 train/validation/test split |
| Comparison | 16, 19, 23.5 N; 3/level (9) | Common inputs for both regressors |
| Trajectory | 16 N; five trials | End-effector repeatability |
| Sequential | One eight-step sequence; 4 N/step | Cumulative deformation localization |
| Camera | 5 poses at 5 heights (25) | Localization repeatability |
Deformable Object Manipulation
Deformable Object Manipulation studies how robots perceive, model, and control flexible objects such as cloth, ropes, cables, and soft tissue.
See all 112 papers