LEGO: Latent-space Exploration for Geometry-aware Optimization of Humanoid Kinematic Design
Authors: Jihwan Yoon, Taemoon Jeong, Jeongeun Park, Chanwoo Kim, Jaewoon Kwon, Yonghyeon Lee, Kyungjae Lee, Sungjoon Choi
Organizations: Department of Artificial Intelligence, Korea University, Seoul 02841, Republic of Korea · Contoro Robotics, Austin, TX 78758, USA · Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Statistics, Korea University, Seoul 02841, Republic of Korea
Designing robot morphologies and kinematics has traditionally relied on human intuition, with little systematic foundation. Motion-design co-optimization offers a promising path toward automation, but two major challenges remain: (i) the vast, unstructured design space and (ii) the difficulty of constructing task-specific loss functions. We propose a new paradigm that minimizes human involvement by (i) learning the design search space from existing mechanical designs, rather than hand-crafting it, and (ii) defining the loss directly from human motion data via motion retargeting and Procrustes analysis. Using screw-theory-based joint axis representation and isometric manifold learning, we construct a compact, geometry-preserving latent space of humanoid upper body designs in which optimization is tractable. We then solve design optimization in this latent space using gradient-free optimization. Our approach establishes a principled framework for data-driven robot design and demonstrates that leveraging existing designs and human motion can effectively guide the automated discovery of novel robot design.
Figures & tables
Figure 1: Total pipeline for humanoid kinematic structure optimization. First, a dataset of robots is converted to a unified Screw Representation ( (ω,q) ). An autoencoder with an isometric regularization loss learns a compact 2D latent space from these representations. Next, Voronoi Optimistic Optimization (VOO) iteratively samples and decodes candidates from the latent space. Each candidate is evaluated by its motion retargeting performance using Procrustes analysis . The process converges to an optimal kinematic structure that best performs the given task.
Rep.
Motion( ↓ )
Motion Set( ↓ )
Wave
Dance
Swimming
Baseline
152.2±±13.9
153.0±±10.2
154.8±±15.4
274.4±±33.9
DH [ 56 ]
162.1±±17.4
148.8±±17.3
150.9±±20.9
315.8±±27.3
Ours
97.5±±3.6
100.6±±5.0
101.3±±3.6
178.4±±10.8
Table I: Best-within-budget total objective on three motions (mean ± std over 10 seeds; lower is better). Objective is PA-MPJPE+λjointNtot with λjoint=3.5 and ε=0.5 .
Figure 2: Mean best‑so‑far objective across ten shared seeds(lower is better). The error bars represent the interquartile range (thick) and min–max (thin). Left inset: oracle distribution with dashed mean μ on motion chicken dance .
Figure 3: Latent map of 30 robots (2D IsoAE). Colors show k ‑means clusters ( k=5 ) with representative thumbnails, which demonstrate that the robots are semantically grouped based on their morphological similarity (e.g., full-body humanoids, dual-arm manipulators). The bottom strip interpolates eight points from rby1 to a starred design (sufficient for Swimming ); joint birth/death is visible as markers appear/disappear under the activation rule.
Figure 4: Hardware prototypes generated by our design framework under manufacturing constraints. Each prototype is optimized for a specific task (‘Waving Hello’, ‘Chicken Dance’, ‘Swimming’), resulting in different kinematic structures.
Figure 5: Demonstration of the three motion-optimized robots from Fig. 4 successfully executing Waving Hello, Chicken Dance, and Swimming via motion retargeting. For each robot snapshot, the human figure (upper right) shows the key pose from the source human motion (BVH file) used as input, while the photo shows the retargeted execution on the robot’s unique kinematics. The red and blue lines visualize the 3D end-effector trajectories of robot and source human motion.
#
Robot
Type
#
Robot
Type
1
Alex [ 35 ]
Non-bipedal
16
JVRC [ 40 ]
Humanoid
2
Apollo [ 31 ]
Humanoid
17
Nextage [ 43 ]
Non-bipedal
3
ARMAR-4 [ 46 ]
Humanoid
18
OP3 [ 31 ]
Humanoid
4
ARMAR-6 [ 47 ]
Non-bipedal
19
Pepper [ 48 ]
Non-bipedal
5
Baxter [ 32 ]
Non-bipedal
20
Poppy [ 41 ]
Humanoid
6
Booster [ 31 ]
Humanoid
21
PR2 [ 38 ]
Non-bipedal
Table II: Robots included in the dataset.
Group
Description
Torso
Trunk joints originating from the base link
Neck
Neck joints following the torso chain
Left / Right Shoulder Girdle
Clavicle and scapula region
Left / Right Shoulder
Primary shoulder joint
Left / Right Upper Arm
Upper arm rotation
Left / Right Elbow
Elbow flexion / extension
Table III: Joint classification groups (screw representation, 14 groups).
Group
Max
Group
Max
Torso
6
R. Shoulder Girdle
1
Neck
4
R. Shoulder
3
L. Shoulder Girdle
1
R. Upper Arm
1
L. Shoulder
3
R. Elbow
1
L. Upper Arm
1
R. Forearm
1
L. Elbow
1
R. Wrist
3
Table IV: Maximum joint count per anatomical group. For the DH representation, one TCP group per arm is appended (1 slot each), giving 16 groups and 32 slots in total.
Component
Specification
Input dimension
120 (20 joints × 6 screw parameters)
Hidden layers
[64, 32]
Latent dimension
2 – 3 (swept)
Activation
Tanh
Encoder path
120 → 64 → 32 →z
Decoder path
z→ 32 → 64 → 120
Table V: Autoencoder architecture.
Parameter
Value
Optimizer
Adam
Learning rate
10−3
Epochs
1000
Batch size
10
Device
CUDA
Table VI: Training hyperparameters.
Parameter
Value
Initial samples ( ninit )
16
Max iterations ( T )
30
Total evaluations ( ninit+T )
46
Global sampling probability ( pglobal )
0.55
Gaussian coefficient ( σc )
0.6
Uniform-to-Gaussian switch ( nswitch )
20
Table VII: VOO hyperparameters used in experiments.
Co-design is a high-dimensional search problem in the robot morphology and control design space. Efficient search requires exploiting the structure shaped by their interaction. To understand this structure, we analyze the landscapes of soft locomotion and manipulation tasks. We identify three patterns consistent across regions of their co-design spaces: 1) Within a region, quality varies along a low-dimensional manifold, with minimal variation orthogonal to it, reducing the effective search space dimensionality. 2) In higher-quality regions, the variance in quality is spread across more dimensions, necessitating search to expand dimensionality as quality improves. 3) In higher-quality regions, quality varies along joint morphology-control dimensions, requiring search along them. Using these insights, we devise an efficient co-design algorithm that yields 36% better co-designs than state-of-the-art baselines. We examine their exploration patterns and show that these baselines required an order of magnitude more function evaluations to find co-designs of comparable quality. Finally, we ablate our algorithm to verify that exploiting the identified structure was the key to efficient co-design.
Apoorv Vaish, Oliver Brock
Robotics and Biology Laboratory, Technische Universität Berlin · Science of Intelligence (SCIoI), Cluster of Excellence, Berlin · Robotics Institute Germany (RIG)
We present MotionDisco, a framework that discovers contact-rich, long-horizon humanoid loco-manipulation motions from scratch, without relying on teleoperation or motion retargeting from human demonstrations. This is challenging because the space of possible contact interactions grows combinatorially with the task horizon and the number of objects in the scene. MotionDisco enables rapid discovery of novel motions by coupling a large language model (LLM) guided evolutionary search over sequences of interactions with an efficient sequential kinodynamic trajectory optimizer and pruning strategy, enabling the rapid discovery of novel skills. Through extensive ablation studies, we show that our LLM-guided search discovers successful whole-body trajectories across several challenging long-horizon tasks. Finally, by training reinforcement learning tracking policies on the discovered trajectories, we transfer the motions to a real humanoid robot. This is the first work to discover and deploy long-horizon humanoid loco-manipulation skills entirely through automated evolutionary search. Supplementary videos of the experiments are available at: https://youtu.be/DHiVz34QYlw.
Ilyass Taouil, Michal Ciebelski, Shafeef Omar +4
Technical University of Munich, Germany · New York University, USA · Carnegie Mellon University, USA
Large language models (LLMs) are increasingly used as proposal generators for evolutionary robot design, yet most loops remain memoryless: simulator results shape the next population but are not preserved as reusable design knowledge. We present Auto-Robotist, a self-evolving LLM agent that distills morphology-search traces into an explicit natural-language skill library. Each skill stores a structural archetype, evidence-grounded positive and negative rules, and the evaluated designs that support them, making design memory inspectable rather than implicit in a population. During search, the agent retrieves skills to condition LLM edits of elite bodies while retaining a Genetic Algorithm (GA) mutation path for exploration; after evaluation, it updates the library through Add, Diagnose, and Merge. Across seven EvoGym tasks spanning locomotion, traversal, and object interaction, Auto-Robotist improves cold-start 5x5 search and transfers learned skills to 10x10 design spaces, where reference-conditioned transfer outperforms GA on every task. These results suggest that LLM agents can convert expensive physical evaluations into reusable, auditable design principles. Our code is publicly available at https://github.com/wangyf9/Auto-Robotist .