Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulation

Kei Ikemura; Yifei Dong; Florian T. Pokorny

cs.ROFeb 20, 2026

Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulation

Authors: Kei Ikemura, Yifei Dong, Florian T. Pokorny

Abstract

Manipulating deformable and fragile objects remains a fundamental challenge in robotics due to complex contact dynamics and strict requirements on object integrity. Existing approaches typically optimize either end-effector design or control strategies in isolation, limiting achievable performance. In this work, we present the first co-design framework that jointly optimizes end-effector morphology and manipulation control for deformable and fragile object manipulation. We introduce (1) a latent diffeomorphic shape parameterization enabling expressive yet tractable end-effector geometry optimization, (2) a stress-aware bi-level co-design pipeline coupling morphology and control optimization, and (3) a privileged-to-pointcloud policy distillation scheme for zero-shot real-world deployment. We evaluate our approach on challenging food manipulation tasks, including grasping and pushing jelly and scooping fillets. Simulation and real-world experiments demonstrate the effectiveness of the proposed method.

Figures & tables

Fig. 1: We jointly optimize end-effector morphology and motion-adaptive control to enable safe, gentle manipulation of deformable and fragile objects. The co-designed end-effector reshapes contact geometry and force distribution, achieving reliable grasping while preserving object integrity, where baseline designs fail and cause breakage.

Fig. 2: Overview of the proposed co-design framework for deformable and fragile object manipulation. (a) A diffeomorphic design space $\mathcal{D}$ parameterizes physically valid end-effector geometries. (b) A derivative-free evolutionary optimizer (CMA-ES) searches $\mathcal{D}$ to identify an optimal design $d^{*}$ that maximizes task performance while minimizing contact-induced damage. (c) For each sampled design $d_{i}$ , a design-conditioned controller $\pi^{*}_{d_{i},\text{prv}}$ is optimized using simulator’s privileged signals such as object stress. (d) Rollouts of the privileged policy on the optimized design $d$ are used to collect pointcloud observations and train a student diffusion policy $\pi^{*}_{d,\text{pcd}}$ . (e) The distilled policy is deployed zero-shot on the real robot using pointcloud observations, preserving gentle manipulation behaviors learned in simulation.

Fig. 3: Deformation of a 2D gripper contour under a stationary velocity field (SVF). The upper region is frozen (zero velocity), while the lower region deforms smoothly.

Fig. 4: Examples of diffeomorphic deformation parameter fitting. The target shape surface (a) and the fitted mesh (b) are close in terms of Chamfer distance.

	cylinder			onigiri			large cube
	score $\uparrow$	success $\uparrow$	stress $\downarrow$	score $\uparrow$	success $\uparrow$	stress $\downarrow$	score $\uparrow$	success $\uparrow$	stress $\downarrow$
Ours	143 $\pm$ 7	0.97 $\pm$ 0.03	9274 $\pm$ 124	64 $\pm$ 18	0.73 $\pm$ 0.08	13973 $\pm$ 490	19 $\pm$ 48	0.51 $\pm$ 0.21	12050 $\pm$ 1116
PJ	79 $\pm$ 12	0.63 $\pm$ 0.05	9366 $\pm$ 120	-33 $\pm$ 3	0.12 $\pm$ 0.04	10224 $\pm$ 332	-133 $\pm$ 17	0.18 $\pm$ 0.04	18893 $\pm$ 620
BO	106 $\pm$ 28	0.80 $\pm$ 0.11	9998 $\pm$ 1089	30 $\pm$ 66	0.59 $\pm$ 0.27	14946 $\pm$ 1658	33 $\pm$ 53	0.45 $\pm$ 0.40	13189 $\pm$ 1030
RL	-110 $\pm$ 62	0.04 $\pm$ 0.02	22015 $\pm$ 12105	-244 $\pm$ 97	0.09 $\pm$ 0.06	38107 $\pm$ 19327	-	-	-

TABLE I: Optimal design performance of the jelly grasping task

Fig. 5: End-effector design evolution for grasping a large jelly cube during a typical optimization run. High-scoring designs exhibit larger contact surfaces, which improve grasp stability and reduce contact-induced stress.

	cylinder			onigiri
	score $J$	success rate $q_{\text{succ}}$	stress $\sigma^{\mathrm{max}}$	score $J$	success rate $q_{\text{succ}}$	stress $\sigma^{\mathrm{max}}$
Ours	85 $\pm$ 5	0.760 $\pm$ 0.000	1236 $\pm$ 108	70 $\pm$ 16	0.627 $\pm$ 0.076	1177 $\pm$ 85
BO	63 $\pm$ 16	0.633 $\pm$ 0.064	1289 $\pm$ 46	65 $\pm$ 23	0.627 $\pm$ 0.115	1295 $\pm$ 64
RL	-39 $\pm$ 31	0.280 $\pm$ 0.100	2432 $\pm$ 522	-81 $\pm$ 33	0.287 $\pm$ 0.163	5032 $\pm$ 659

TABLE II: Optimal design performance of the jelly pushing task

	score $J\uparrow$	success rate $q_{\text{succ}}\uparrow$	stress $\sigma^{\mathrm{max}}\downarrow$
Ours	293 $\pm$ 97	0.779 $\pm$ 0.203	17154 $\pm$ 469
BO	282 $\pm$ 146	0.683 $\pm$ 0.322	13586 $\pm$ 2857

TABLE III: Optimal design performance of the fillet scooping task

Fig. 6: Visualization of the design-space landscape projected onto two principal components from the $M_{\text{latent}}$ -dimensional latent diffeomorphic space. Warmer colors indicate higher end-effector performance for the fillet scooping task. It suggests that thinner and longer end-effector geometries tend to perform better for scooping.

	score $J$	success rate $q_{\text{succ}}$	stress $\sigma^{\mathrm{max}}$
Ours	64 $\pm$ 18	0.727 $\pm$ 0.081	13973 $\pm$ 490
Cubic	10 $\pm$ 17	0.473 $\pm$ 0.083	14894 $\pm$ 306
Spherical	13 $\pm$ 23	0.540 $\pm$ 0.106	15321 $\pm$ 386

TABLE IV: Comparison of design spaces

Task	Method	success rate $\uparrow$	object damage $\downarrow$
Grasp	Ours	9 /9	1 /9
Grasp	BO	5/9	1 /9
Push	Ours	10 /12	0 /12
Push	BO	9/12	0 /12

TABLE V: Real-World Experiments

Fig. 7: (a) Task-specific end-effector designs optimized, fabricated, and mounted on the robot for real-world evaluation. Two design optimizers $\mathcal{O}_{d}$ are compared: CMA-ES (ours) and BO for jelly cylinder grasping and pushing tasks. The co-designed end-effector exhibits curved inner contact surfaces that improve form closure and reduce contact-induced damage to fragile objects. (b) & (c) The roll out of our policy and BO baseline in real world.

Fig. 8: Additional examples of diffeomorphic parameter fitting for jelly grasping and pushing. The target mesh (left, blue) and the fitted mesh (right, purple) closely match in terms of Chamfer distance. The target shapes are drawn from a subset of the $N_{0}$ designs used to learn the raw diffeomorphic parameters.

Fig. 9: Subset of the diffeomorphic training dataset and corresponding parameter fitting results for the fillet scooping task.

Paper	Design Space Dimension	Action Space Dimension	Co-Design Paradigm
SERL Co-Design [ 8 ]	2	9	Two-stage: RL + GA
Meta-RL Co-Design [ 4 ]	4	16	Two-stage: RL + CMA-ES
Object Adaptation [ 18 ]	4	7	Single-stage: Dual MDP
Object Adaptation [ 18 ]	4	7	Single-stage: Dual MDP
Cross-Embodiment Hand Co-Design [ 14 ]	5	20	Two-stage with Design-Conditioned Policy
CageCoOpt [ 12 ]	6	4	Two-stage: PPO + BO/GA

TABLE VI: Comparison of co-design methods by design and action space dimensionality and paradigm.

Fig. 10: End-effector design evolution for pushing a jelly onigiri during a typical optimization run. High-scoring designs exhibit larger contact surfaces, which improve push stability.

Fig. 11: Example base shape in the LDM training dataset (leftmost) and its augmented variants generated via random rotation, translation, and scaling.

Fig. 12: Average Chamfer distance between original mesh and fitted mesh, with respect to latent design space dimension

M_{\text{latent}}

. We choose

M_{\text{latent}}=15

in most of the experiments as a tradeoff.

Fig. 13: Real-world rollouts of the co-designed policies on the three DFOM tasks.

Fig. 14: Graphical user interface for curating end-effector designs used in diffeomorphic design space learning. Shapes from Thingi10K are attached to a standard finger mount at a user-specified pose.

Fig. 15: Parallel, independent simulation environments in Genesis.

Fig. 16: Illustration of synthesized pre-contact configurations $p_{d}^{\star}$ and the corresponding motion primitive sequences for jelly grasping, pushing, and fillet scooping. Each row shows the ordered execution stages, including approach, interaction, and post-contact lifting, pushing or insertion.

Fig. 17: illustration of the two-stage evaluation strategy in control optimization (line 6-9 in Algorithm 1 ). $p_{e,i},i\in\{1,2,..,I\}$ refers to $I$ candidate pre-contact configurations synthesized from rigid body surrogate. Here $I=5$ . $p^{*}_{e}$ refers to the best pre-contact configuration selected for execution.

Task	$\lambda_{\text{prog}}$	$\lambda_{\text{succ}}$	$\lambda^{2.5}_{\mathrm{max}}$	$\lambda^{0}_{\mathrm{max}}$	$\lambda_{\bar{\sigma}}$
Grasping	$1\times 10^{2}$	$2\times 10^{2}$	$2\times 10^{-3}$	$8\times 10^{-4}$	$1\times 10^{-2}$
Pushing	$3\times 10^{2}$	$5\times 10^{2}$	$2\times 10^{-3}$	$8\times 10^{-4}$	$1\times 10^{-2}$
Scooping	$1\times 10^{2}$	$2\times 10^{2}$	$2\times 10^{-3}$	$8\times 10^{-4}$	$1\times 10^{-2}$

TABLE VII: Design objective weights used for different tasks.

Explore similar work

Dec 3, 2025

House of Dextra: Cross-embodied Co-design for Dexterous Hands

Dexterous manipulation is limited by both control and design, without consensus as to what makes manipulators best for performing dexterous tasks. This raises a fundamental challenge: how should we design and control robot manipulators that are optimized for dexterity? We present a co-design framework that learns task-specific hand morphology and complementary dexterous control policies. The framework supports 1) an expansive morphology search space including joint, finger, and palm generation, 2) scalable evaluation across the wide design space via morphology-conditioned cross-embodied control, and 3) real-world fabrication with accessible components. We evaluate the approach across multiple dexterous tasks, including in-hand rotation with simulation and real deployment. Our framework enables an end-to-end pipeline that can design, train, fabricate, and deploy a new robotic hand in under 24 hours. The full framework and generated robot hands are open-sourced and available on our website.

Kehlani Fay, Darin Anthony Djapri, Anya Zorin +6

University of California, San Diego · University of California, Santa Barbara

Robotic Hand Design Dexterous Manipulation Robot Morphology-Control Co-Design Cross-Embodiment Robot Learning Robotics

Ask PDF

Sep 22, 2026

Manipulation with Stability Guarantees: Linear Deformable Objects with Non-negligible Physical Response Grasped at Multiple Location

Most research on the manipulation of deformable objects focuses on lightweight systems with negligible mechanical response, effectively restricting attention to quasi-static regimes. This assumption excludes a broad class of practically relevant objects, such as hoses, pipes, and wiring harnesses, whose dynamics cannot be ignored during manipulation. In this work, we address this limitation by introducing a closed-loop control architecture that explicitly accounts for object dynamics and recasts manipulation as a shape-regulation problem. Control is achieved by modulating forces and torques applied at multiple fixed points along the object. This approach builds on three methodological contributions: a fully dynamic model of linear deformable objects based on discrete strain parameterizations; an extension of the notion of actuation coordinates to SE(3), yielding a structured and inherently underactuated control architecture; and nonlinear feedback strategies providing explicit conditions for steady-state convergence to desired configurations. Extensive simulations on representative manipulation tasks demonstrate the performance gains enabled by the proposed modelbased formulation. We finally validate the approach experimentally through a real-time closed-loop implementation with online shape estimation, confirming its practical feasibility and effectiveness

Daniel Feliu-Talegon, Cosimo Della Santina

Department of Cognitive Robotics, Delft University of Technology, 2628 CN Delft, The Netherlands · Institute of Robotics and Mechatronics, German Aerospace Center (DLR), 82234 Oberpfaffenhofen, Germany

Deformable Object Manipulation Feedback Control Dynamic Robotic Manipulation

Ask PDF

Mar 18, 2026

Rapid Adaptation of Particle Dynamics for Generalized Deformable Object Mobile Manipulation

We address the challenge of learning to manipulate deformable objects with unknown dynamics. In non-rigid objects, the dynamics parameters define how they react to interactions -- how they stretch, bend, compress, and move -- and they are critical to determining the optimal actions to perform a manipulation task successfully. In other robotic domains, such as legged locomotion and in-hand rigid object manipulation, state-of-the-art approaches can handle unknown dynamics using Rapid Motor Adaptation (RMA). Through a supervised procedure in simulation that encodes each rigid object's dynamics, such as mass and position, these approaches learn a policy that conditions actions on a vector of latent dynamic parameters inferred from sequences of state-actions. However, in deformable object manipulation, the object's dynamics not only includes its mass and position, but also how the shape of the object changes. Our key insight is that the recent ground-truth particle positions of a deformable object in simulation capture changes in the object's shape, making it possible to extend RMA to deformable object manipulation. This key insight allows us to develop RAPiD, a two-phase method that learns to perform real-robot deformable object mobile manipulation by: 1) learning a visuomotor policy conditioned on the object's dynamics embedding, which is encoded from the object's privileged information in simulation, such as its mass and ground-truth particle positions, and 2) learning to infer this embedding using non-privileged information instead, such as robot visual observations and actions, so that the learned policy can transfer to the real world. On a mobile manipulator with 22 degrees of freedom, RAPiD enables over 80%+ success rates across two vision-based deformable object mobile manipulation tasks in the real world, under various object dynamics, categories, and instances.

Bohan Wu, Roberto Martín-Martín, Li Fei-Fei

the University of Texas at Austin, USA

Deformable Object Manipulation Generalization in Robotic Manipulation Mobile Manipulation Adaptive Control Robot Dynamics Learning

Ask PDF