LEGO: Latent-space Exploration for Geometry-aware Optimization of Humanoid Kinematic Design

Jihwan Yoon; Taemoon Jeong; Jeongeun Park; Chanwoo Kim; Jaewoon Kwon; Yonghyeon Lee; Kyungjae Lee; Sungjoon Choi

cs.ROApr 9, 2026

LEGO: Latent-space Exploration for Geometry-aware Optimization of Humanoid Kinematic Design

Authors: Jihwan Yoon, Taemoon Jeong, Jeongeun Park, Chanwoo Kim, Jaewoon Kwon, Yonghyeon Lee, Kyungjae Lee, Sungjoon Choi

Organizations: Department of Artificial Intelligence, Korea University, Seoul 02841, Republic of Korea · Contoro Robotics, Austin, TX 78758, USA · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Statistics, Korea University, Seoul 02841, Republic of Korea

Abstract

Designing robot morphologies and kinematics has traditionally relied on human intuition, with little systematic foundation. Motion-design co-optimization offers a promising path toward automation, but two major challenges remain: (i) the vast, unstructured design space and (ii) the difficulty of constructing task-specific loss functions. We propose a new paradigm that minimizes human involvement by (i) learning the design search space from existing mechanical designs, rather than hand-crafting it, and (ii) defining the loss directly from human motion data via motion retargeting and Procrustes analysis. Using screw-theory-based joint axis representation and isometric manifold learning, we construct a compact, geometry-preserving latent space of humanoid upper body designs in which optimization is tractable. We then solve design optimization in this latent space using gradient-free optimization. Our approach establishes a principled framework for data-driven robot design and demonstrates that leveraging existing designs and human motion can effectively guide the automated discovery of novel robot design.

Figures & tables

Figure 1: Total pipeline for humanoid kinematic structure optimization. First, a dataset of robots is converted to a unified Screw Representation ( $(\omega,q)$ ). An autoencoder with an isometric regularization loss learns a compact 2D latent space from these representations. Next, Voronoi Optimistic Optimization (VOO) iteratively samples and decodes candidates from the latent space. Each candidate is evaluated by its motion retargeting performance using Procrustes analysis . The process converges to an optimal kinematic structure that best performs the given task.

Rep.	Motion( $\downarrow$ )			Motion Set( $\downarrow$ )
Rep.	Wave	Dance	Swimming	Motion Set( $\downarrow$ )
Baseline	$152.2\pm\pm 13.9$	$153.0\pm\pm 10.2$	$154.8\pm\pm 15.4$	$274.4\pm\pm 33.9$
DH [ 56 ]	$162.1\pm\pm 17.4$	$148.8\pm\pm 17.3$	$150.9\pm\pm 20.9$	$315.8\pm\pm 27.3$
Ours	$97.5\pm\pm 3.6$	$100.6\pm\pm 5.0$	$101.3\pm\pm 3.6$	$178.4\pm\pm 10.8$

Table I: Best-within-budget total objective on three motions (mean

\pm

std over 10 seeds; lower is better). Objective is

\mathrm{PA\text{-}MPJPE}+\lambda_{\text{joint}}N_{\mathrm{tot}}

with

\lambda_{\text{joint}}=3.5

and

\varepsilon=0.5

Figure 2: Mean best‑so‑far objective across ten shared seeds(lower is better). The error bars represent the interquartile range (thick) and min–max (thin). Left inset: oracle distribution with dashed mean $\mu$ on motion chicken dance .

Figure 3: Latent map of 30 robots (2D IsoAE). Colors show $k$ ‑means clusters ( $k{=}5$ ) with representative thumbnails, which demonstrate that the robots are semantically grouped based on their morphological similarity (e.g., full-body humanoids, dual-arm manipulators). The bottom strip interpolates eight points from rby1 to a starred design (sufficient for Swimming ); joint birth/death is visible as markers appear/disappear under the activation rule.

Figure 4: Hardware prototypes generated by our design framework under manufacturing constraints. Each prototype is optimized for a specific task (‘Waving Hello’, ‘Chicken Dance’, ‘Swimming’), resulting in different kinematic structures.

Figure 5: Demonstration of the three motion-optimized robots from Fig. 4 successfully executing Waving Hello, Chicken Dance, and Swimming via motion retargeting. For each robot snapshot, the human figure (upper right) shows the key pose from the source human motion (BVH file) used as input, while the photo shows the retargeted execution on the robot’s unique kinematics. The red and blue lines visualize the 3D end-effector trajectories of robot and source human motion.

#	Robot	Type	#	Robot	Type
1	Alex [ 35 ]	Non-bipedal	16	JVRC [ 40 ]	Humanoid
2	Apollo [ 31 ]	Humanoid	17	Nextage [ 43 ]	Non-bipedal
3	ARMAR-4 [ 46 ]	Humanoid	18	OP3 [ 31 ]	Humanoid
4	ARMAR-6 [ 47 ]	Non-bipedal	19	Pepper [ 48 ]	Non-bipedal
5	Baxter [ 32 ]	Non-bipedal	20	Poppy [ 41 ]	Humanoid
6	Booster [ 31 ]	Humanoid	21	PR2 [ 38 ]	Non-bipedal

Table II: Robots included in the dataset.

Group	Description
Torso	Trunk joints originating from the base link
Neck	Neck joints following the torso chain
Left / Right Shoulder Girdle	Clavicle and scapula region
Left / Right Shoulder	Primary shoulder joint
Left / Right Upper Arm	Upper arm rotation
Left / Right Elbow	Elbow flexion / extension

Table III: Joint classification groups (screw representation, 14 groups).

Group	Max	Group	Max
Torso	6	R. Shoulder Girdle	1
Neck	4	R. Shoulder	3
L. Shoulder Girdle	1	R. Upper Arm	1
L. Shoulder	3	R. Elbow	1
L. Upper Arm	1	R. Forearm	1
L. Elbow	1	R. Wrist	3

Table IV: Maximum joint count per anatomical group. For the DH representation, one TCP group per arm is appended (1 slot each), giving 16 groups and 32 slots in total.

Component	Specification
Input dimension	120 (20 joints $\times$ 6 screw parameters)
Hidden layers	[64, 32]
Latent dimension	2 – 3 (swept)
Activation	Tanh
Encoder path	120 $\to$ 64 $\to$ 32 $\to$ $z$
Decoder path	$z$ $\to$ 32 $\to$ 64 $\to$ 120

Table V: Autoencoder architecture.

Parameter	Value
Optimizer	Adam
Learning rate	$10^{-3}$
Epochs	1000
Batch size	10
Device	CUDA

Table VI: Training hyperparameters.

Parameter	Value
Initial samples ( $n_{\text{init}}$ )	16
Max iterations ( $T$ )	30
Total evaluations ( $n_{\text{init}}+T$ )	46
Global sampling probability ( $p_{\text{global}}$ )	0.55
Gaussian coefficient ( $\sigma_{c}$ )	0.6
Uniform-to-Gaussian switch ( $n_{\text{switch}}$ )	20

Table VII: VOO hyperparameters used in experiments.

Robot Morphology-Control Co-Design

Robot morphology-control co-design studies how to jointly shape a robot’s body and controller to improve task performance.

See all 60 papers

Identifying and Exploiting Structure in Robot Co-Design
Apr 13, 2026Apoorv Vaish, Oliver BrockRobot Morphology-Control Co-Design Robotic Manipulation
Robotics and Biology Laboratory, Technische Universität Berlin · Science of Intelligence (SCIoI), Cluster of Excellence, Berlin · Robotics Institute Germany (RIG)
MotionDisco: Motion Discovery for Extreme Humanoid Loco-Manipulation
Jun 4, 2026Ilyass Taouil, Michal Ciebelski, Shafeef Omar +4Robot Skill Learning Evolutionary Optimization
Technical University of Munich, Germany · New York University, USA · Carnegie Mellon University, USA
When Search Becomes Memory: Accelerating Robot Design Discovery with Self-Evolving Skills
May 25, 2026Yunfei Wang, Xiaohao Xu, Yang Li +1Robotics LLM Agent Skill Learning
University of Michigan, Ann Arbor

Apr 13, 2026J/K move · Enter open · S save

Identifying and Exploiting Structure in Robot Co-Design

Apoorv Vaish, Oliver Brock

Robotics and Biology Laboratory, Technische Universität Berlin · Science of Intelligence (SCIoI), Cluster of Excellence, Berlin · Robotics Institute Germany (RIG)

Robot Morphology-Control Co-Design Robotic Manipulation Robot Locomotion Hardware-Software Co-Design

Open paper AskPDF arXiv

Co-design is a high-dimensional search problem in the robot morphology and control design space. Efficient search requires exploiting the structure shaped by their interaction. To understand this structure, we analyze the landscapes of soft locomotion and manipulation tasks. We identify three patterns consistent across regions of their co-design spaces: 1) Within a region, quality varies along a low-dimensional manifold, with minimal variation orthogonal to it, reducing the effective search space dimensionality. 2) In higher-quality regions, the variance in quality is spread across more dimensions, necessitating search to expand dimensionality as quality improves. 3) In higher-quality regions, quality varies along joint morphology-control dimensions, requiring search along them. Using these insights, we devise an efficient co-design algorithm that yields 36% better co-designs than state-of-the-art baselines. We examine their exploration patterns and show that these baselines required an order of magnitude more function evaluations to find co-designs of comparable quality. Finally, we ablate our algorithm to verify that exploiting the identified structure was the key to efficient co-design.

Abstract

Robot Morphology-Control Co-Design

Explore similar work

Identifying and Exploiting Structure in Robot Co-Design

MotionDisco: Motion Discovery for Extreme Humanoid Loco-Manipulation

When Search Becomes Memory: Accelerating Robot Design Discovery with Self-Evolving Skills