The Language of Deformation: Semantic Failure Analysis as a Prerequisite for Dexterous Manipulation of Deformable Objects
Authors: Ignacio Cuiral-Zueco, David Hardman, Michele Pierallini, Erfan Shahriari
Organizations: Clermont Auvergne INP, Institut Pascal, F-63000 Clermont-Ferrand, France. · Department of Engineering, University of Cambridge Cambridge CB2 1PZ, UK · Centro di Ricerca “Enrico Piaggio”, Universit`a di Pisa, Largo Lucio Lazzarino 1, 56126 Pisa, Italy. · RAI Institute, Zurich 8050, Switzerland
Dexterous manipulation of rigid objects has reached a level of maturity where grasp stability, control, and planning strategies are successfully applied in industry and logistics. In contrast, manipulating deformable objects (cables, plastic pieces, cloth) remains far more challenging, as rigid-object methods do not account for their additional complexity. This paper identifies semantic failure analysis as a key step toward mature deformable-object manipulation and supports this claim with proof-of-concept experiments. Unlike rigid-object manipulation, where goals are well-defined in terms of criteria such as force balance and slippage, deformable-object manipulation involves a broader variety of failure modes arising from differences in materials, geometry, and behaviour. Typical failures include buckling, plastic deformation, pressure variations, and tearing. These diverse and often hard-to-detect events require distinct corrective strategies, making semantic failure analysis necessary. We define its key components and challenges and present experimental results supporting its potential as a research direction.
Figures & tables
Figure 1: Schematic representation of the SFA-DOM framework. Sensor observations yt and contextual history ht are processed by the semantic failure classifier Φ to infer a semantic failure label st , which the corrective strategy mapper C:S→Γ then uses to select an appropriate corrective response. Combined with the control policy π:X×S→U , this selection yields failure-aware control actions ut . The depicted tactile sensors, signal conditioning, and controller selection are purely illustrative.
Figure 2: Experimental setup and tactile sensing results for interactions with deformable and fragile objects. (Top) Video frame sequences showing manipulation of a balloon, a clay piece, a thin paper sheet, and a filament, illustrating distinct events: compression, plastic deformation, tearing, and buckling. (Bottom left) Sensing configuration. (Bottom right) Recorded magnetic-field components x , y , and z (R, G, B) for each object interaction. Blue shaded regions highlight characteristic tactile signatures corresponding to each event: three compressions (balloon), plastic deformation and grip loss (clay), tearing (paper), and buckling with snapping (filament).