Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulation

Kei Ikemura; Yifei Dong; Florian T. Pokorny

Fig. 1: We jointly optimize end-effector morphology and motion-adaptive control to enable safe, gentle manipulation of deformable and fragile objects. The co-designed end-effector reshapes contact geometry and force distribution, achieving reliable grasping while preserving object integrity, where baseline designs fail and cause breakage.

Fig. 2: Overview of the proposed co-design framework for deformable and fragile object manipulation. (a) A diffeomorphic design space

\mathcal{D}

parameterizes physically valid end-effector geometries. (b) A derivative-free evolutionary optimizer (CMA-ES) searches

\mathcal{D}

to identify an optimal design

d^{*}

that maximizes task performance while minimizing contact-induced damage. (c) For each sampled design

d_{i}

, a design-conditioned controller

\pi^{*}_{d_{i},\text{prv}}

is optimized using simulator’s privileged signals such as object stress. (d) Rollouts of the privileged policy on the optimized design

d

are used to collect pointcloud observations and train a student diffusion policy

\pi^{*}_{d,\text{pcd}}

. (e) The distilled policy is deployed zero-shot on the real robot using pointcloud observations, preserving gentle manipulation behaviors learned in simulation.

Fig. 3: Deformation of a 2D gripper contour under a stationary velocity field (SVF). The upper region is frozen (zero velocity), while the lower region deforms smoothly.

Fig. 4: Examples of diffeomorphic deformation parameter fitting. The target shape surface (a) and the fitted mesh (b) are close in terms of Chamfer distance.

	cylinder			onigiri			large cube
	score $\uparrow$	success $\uparrow$	stress $\downarrow$	score $\uparrow$	success $\uparrow$	stress $\downarrow$	score $\uparrow$	success $\uparrow$	stress $\downarrow$
Ours	143 $\pm$ 7	0.97 $\pm$ 0.03	9274 $\pm$ 124	64 $\pm$ 18	0.73 $\pm$ 0.08	13973 $\pm$ 490	19 $\pm$ 48	0.51 $\pm$ 0.21	12050 $\pm$ 1116
PJ	79 $\pm$ 12	0.63 $\pm$ 0.05	9366 $\pm$ 120	-33 $\pm$ 3	0.12 $\pm$ 0.04	10224 $\pm$ 332	-133 $\pm$ 17	0.18 $\pm$ 0.04	18893 $\pm$ 620
BO	106 $\pm$ 28	0.80 $\pm$ 0.11	9998 $\pm$ 1089	30 $\pm$ 66	0.59 $\pm$ 0.27	14946 $\pm$ 1658	33 $\pm$ 53	0.45 $\pm$ 0.40	13189 $\pm$ 1030
RL	-110 $\pm$ 62	0.04 $\pm$ 0.02	22015 $\pm$ 12105	-244 $\pm$ 97	0.09 $\pm$ 0.06	38107 $\pm$ 19327	-	-	-

TABLE I: Optimal design performance of the jelly grasping task

Fig. 5: End-effector design evolution for grasping a large jelly cube during a typical optimization run. High-scoring designs exhibit larger contact surfaces, which improve grasp stability and reduce contact-induced stress.

	cylinder			onigiri
	score $J$	success rate $q_{\text{succ}}$	stress $\sigma^{\mathrm{max}}$	score $J$	success rate $q_{\text{succ}}$	stress $\sigma^{\mathrm{max}}$
Ours	85 $\pm$ 5	0.760 $\pm$ 0.000	1236 $\pm$ 108	70 $\pm$ 16	0.627 $\pm$ 0.076	1177 $\pm$ 85
BO	63 $\pm$ 16	0.633 $\pm$ 0.064	1289 $\pm$ 46	65 $\pm$ 23	0.627 $\pm$ 0.115	1295 $\pm$ 64
RL	-39 $\pm$ 31	0.280 $\pm$ 0.100	2432 $\pm$ 522	-81 $\pm$ 33	0.287 $\pm$ 0.163	5032 $\pm$ 659

TABLE II: Optimal design performance of the jelly pushing task

	score $J\uparrow$	success rate $q_{\text{succ}}\uparrow$	stress $\sigma^{\mathrm{max}}\downarrow$
Ours	293 $\pm$ 97	0.779 $\pm$ 0.203	17154 $\pm$ 469
BO	282 $\pm$ 146	0.683 $\pm$ 0.322	13586 $\pm$ 2857

TABLE III: Optimal design performance of the fillet scooping task

Fig. 6: Visualization of the design-space landscape projected onto two principal components from the

M_{\text{latent}}

-dimensional latent diffeomorphic space. Warmer colors indicate higher end-effector performance for the fillet scooping task. It suggests that thinner and longer end-effector geometries tend to perform better for scooping.

	score $J$	success rate $q_{\text{succ}}$	stress $\sigma^{\mathrm{max}}$
Ours	64 $\pm$ 18	0.727 $\pm$ 0.081	13973 $\pm$ 490
Cubic	10 $\pm$ 17	0.473 $\pm$ 0.083	14894 $\pm$ 306
Spherical	13 $\pm$ 23	0.540 $\pm$ 0.106	15321 $\pm$ 386

TABLE IV: Comparison of design spaces

Task	Method	success rate $\uparrow$	object damage $\downarrow$
Grasp	Ours	9 /9	1 /9
Grasp	BO	5/9	1 /9
Push	Ours	10 /12	0 /12
Push	BO	9/12	0 /12

TABLE V: Real-World Experiments

Fig. 7: (a) Task-specific end-effector designs optimized, fabricated, and mounted on the robot for real-world evaluation. Two design optimizers

\mathcal{O}_{d}

are compared: CMA-ES (ours) and BO for jelly cylinder grasping and pushing tasks. The co-designed end-effector exhibits curved inner contact surfaces that improve form closure and reduce contact-induced damage to fragile objects. (b) & (c) The roll out of our policy and BO baseline in real world.

Fig. 8: Additional examples of diffeomorphic parameter fitting for jelly grasping and pushing. The target mesh (left, blue) and the fitted mesh (right, purple) closely match in terms of Chamfer distance. The target shapes are drawn from a subset of the

N_{0}

designs used to learn the raw diffeomorphic parameters.

Fig. 9: Subset of the diffeomorphic training dataset and corresponding parameter fitting results for the fillet scooping task.

Paper	Design Space Dimension	Action Space Dimension	Co-Design Paradigm
SERL Co-Design [ 8 ]	2	9	Two-stage: RL + GA
Meta-RL Co-Design [ 4 ]	4	16	Two-stage: RL + CMA-ES
Object Adaptation [ 18 ]	4	7	Single-stage: Dual MDP
Object Adaptation [ 18 ]	4	7	Single-stage: Dual MDP
Cross-Embodiment Hand Co-Design [ 14 ]	5	20	Two-stage with Design-Conditioned Policy
CageCoOpt [ 12 ]	6	4	Two-stage: PPO + BO/GA

TABLE VI: Comparison of co-design methods by design and action space dimensionality and paradigm.

Fig. 10: End-effector design evolution for pushing a jelly onigiri during a typical optimization run. High-scoring designs exhibit larger contact surfaces, which improve push stability.

Fig. 11: Example base shape in the LDM training dataset (leftmost) and its augmented variants generated via random rotation, translation, and scaling.

Fig. 12: Average Chamfer distance between original mesh and fitted mesh, with respect to latent design space dimension

M_{\text{latent}}

. We choose

M_{\text{latent}}=15

in most of the experiments as a tradeoff.

Fig. 13: Real-world rollouts of the co-designed policies on the three DFOM tasks.

Fig. 14: Graphical user interface for curating end-effector designs used in diffeomorphic design space learning. Shapes from Thingi10K are attached to a standard finger mount at a user-specified pose.

Fig. 15: Parallel, independent simulation environments in Genesis.

Fig. 16: Illustration of synthesized pre-contact configurations

p_{d}^{\star}

and the corresponding motion primitive sequences for jelly grasping, pushing, and fillet scooping. Each row shows the ordered execution stages, including approach, interaction, and post-contact lifting, pushing or insertion.

Fig. 17: illustration of the two-stage evaluation strategy in control optimization (line 6-9 in Algorithm 1 ).

p_{e,i},i\in\{1,2,..,I\}

refers to

I

candidate pre-contact configurations synthesized from rigid body surrogate. Here

I=5

.

p^{*}_{e}

refers to the best pre-contact configuration selected for execution.

Task	$\lambda_{\text{prog}}$	$\lambda_{\text{succ}}$	$\lambda^{2.5}_{\mathrm{max}}$	$\lambda^{0}_{\mathrm{max}}$	$\lambda_{\bar{\sigma}}$
Grasping	$1\times 10^{2}$	$2\times 10^{2}$	$2\times 10^{-3}$	$8\times 10^{-4}$	$1\times 10^{-2}$
Pushing	$3\times 10^{2}$	$5\times 10^{2}$	$2\times 10^{-3}$	$8\times 10^{-4}$	$1\times 10^{-2}$
Scooping	$1\times 10^{2}$	$2\times 10^{2}$	$2\times 10^{-3}$	$8\times 10^{-4}$	$1\times 10^{-2}$

TABLE VII: Design objective weights used for different tasks.

Latent Diffeomorphic Co-Design of End-Effectors for Deformable and Fragile Object Manipulation

Abstract

Figures & tables

Explore similar work

House of Dextra: Cross-embodied Co-design for Dexterous Hands

Manipulation with Stability Guarantees: Linear Deformable Objects with Non-negligible Physical Response Grasped at Multiple Location

Rapid Adaptation of Particle Dynamics for Generalized Deformable Object Mobile Manipulation