Organizations: School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Intelligent Game and Decision Laboratory, Beijing, 100071, China · School of Statistics, Renmin University of China, Beijing, 100872, China · School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, 200030, China · Shanghai Innovation Institute, Shanghai, China · Center for Applied Statistics, Renmin University of China, Beijing, 100872, China
Recent advances in robotics highlight hierarchical configurations of robot morphology, where multiple levels of functional substructures synergize to facilitate intelligent behaviors. This hierarchical perspective, while particularly advantageous for voxel-based soft robots (VSRs) to ease design and control complexities, is hindered by its heavy reliance on domain expertise. In this work, we address the following question: can we derive such hierarchical design principles solely from existing successful designs? We answer affirmatively by presenting RoboLDA, a Bayesian probabilistic model that decomposes VSR morphology generation into a four-level hierarchy: "task-robot-organ-voxel", and is trained via variational inference. Through extensive experiments on simulated VSRs, we verify the presence of consistent, intuitive hierarchical patterns underlying high-performing VSR designs and showcase RoboLDA's proficiency to extract and leverage these hierarchical priors for zero-shot robot design in unseen tasks. The generated designs, even without further optimization, achieve on average 106.4% of the optimized performance produced by evolutionary algorithms. Additionally, the organ structures inferred by RoboLDA serve as valid functional substructures, significantly enhancing synergistic motion control when integrated with modular control policies. Our work pioneers hierarchical generative modeling of robot morphology, offering a promising pathway towards more interpretable and generalizable development of embodied agents.
Figures & tables
Figure 1: VSR hierarchy from tasks to voxels. Robot types specialized for different tasks exhibit distinct features, but also share some common functional substructures (colored dashed boxes). “V" and “H" represent “vertical" and “horizontal", respectively.
Figure 2: The architecture of RoboLDA, including a decoder (generative process) and an encoder (approximate posterior). The generative process consists of four levels namely task, robot, organ and voxel. The approximate posterior infers latent variables to support variational inference.
Figure 3: MetaMorph with organ synergy. The upper half shows the architecture of MetaMorph. The lower half demonstrates how organs are converted into attention masks and imposed onto the attention layer.
Figure 4: An example of transformation between voxel matrices and rigid robots. For the purpose of illustration, robot joints are torqued so that the directions of rotation become more apparent. Different limb lengths are marked as different colors, while different rotation directions as different shades. Apart from ten different limb types (0 ∼ 9), we use a distinct number (10) to denote the torso.
Figure 5: Illustration of the hierarchical generative process learned by RoboLDA. The distributions of robot type are illustrated as histograms, whereas for organ and voxel layers, each position is labeled with the most likely outcome.
Figure 6: Learning curves of RoboLDA when being trained on morphologies with different performance levels.
Table 1: Task combinations for zero-shot evaluation. Each task is annotated with its difficulty level (easy/medium/hard) and task category (loc: locomotion, man: manipulation, cli: climbing). Following the task-relation perspective of PreCo [ 63 ] , these settings broadly cover three transfer types: more challenging variants (e.g., Climber-v0 → Climber-v2), transfers of comparable difficulty (e.g., Walker-v0 ↔ BridgeWalker-v0; PlatformJumper-v0 → GapJumper-v0), and reverse scenarios (e.g., UpStepper-v0 ↔ DownStepper-v0).
Task
RoboLDA
LASeR
MorphVAE
c-DM
RoboGAN
BO/GA
CPPN-NEAT
POET
High-similarity transfers
Pusher-v0 (easy)
11.42
10.97
11.22
6.99
7.60
7.79
9.67
12.50
Carrier-v0 (easy)
8.47
7.72
8.61
5.70
4.70
5.98
6.58
8.19
Climber-v1 (medium)
5.78
0.72
3.25
0.26
0.21
0.54
0.25
-
Medium-similarity transfers
Walker-v0 (easy)
10.65
10.62
10.64
10.21
8.45
9.85
6.42
-
Table 2: Comparison of zero-shot fitness between RoboLDA and baseline algorithms, grouped by train–test task similarity measured by average edit distance (Table 1 ). As both GA and BO use random sampling for their first generation, their results are averaged. The task names are followed by their difficulty levels. Harder tasks require more complex morphologies and thus present bigger challenges for robot design.
Task
BO
GA
CPPN-NEAT
BridgeWalker-v0
11
18
8
Walker-v0
N/A
N/A
N/A
Pusher-v0
N/A
29
N/A
Carrier-v0
N/A
18
N/A
UpStepper-v0
N/A
28
N/A
DownStepper-v0
23
17
3
Table 3: The number of generations for baseline EAs to reach RoboLDA’s zero-shot fitness (using the average-then-threshold protocol; see Section 4.4). “N/A" indicates that the EA fails to do so before evolution terminates.
Figure 7: Relative performance of zero-shot RoboLDA compared against optimal results evolved by baseline EAs (BO, GA, and CPPN-NEAT), using the ratio-then-average protocol (see Section 4.4).
Figure 8: Comparative results of robotic control.
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Results of comparison study on robotic control (training robots).
Task
BW-v0
Walker-v0
Pusher-v0
Carrier-v0
US-v0
DS-v0
Climber-v1
Climber-v2
GJ-v0
Sensitivity regarding hyperparameter settings
Std
0.222
0.008
0.990
1.246
0.983
0.641
0.831
0.313
0.776
COV
0.034
< 0.001
0.082
0.141
0.236
0.072
0.142
0.159
0.178
Sensitivity regarding random initialization
Std
0.004
0.003
1.213
1.098
0.006
0.277
0.050
0.027
0.375
COV
< 0.001
< 0.001
0.106
0.130
< 0.001
0.054
0.009
0.015
0.102
Appendix
Table 4: Sensitivity analysis of zero-shot fitness. “Std" and “COV" represent standard deviation and coefficient of variation, respectively. BW, US, DS and GJ stand for BridgeWalker, UpStepper, DownStepper and GapJumper, respectively.
Figure 10: Case studies on coordination and dexterity.
Figure 11: Locomotion environments of rigid robots.
Table 6: Zero-shot transfer results on 10×10 Walker-v0.
Figure 12: Organ mask ablation on four tasks. Each row: one task; left: training split; right: held-out test split.
Method
Climber-v1 (high)
Walker-v0 (med.)
UpStepper-v0 (low)
RoboLDA
5.78
10.65
5.13
w/o position modeling
4.13
10.59
4.02
task-organ-voxel
5.55
10.62
4.16
task-robot-voxel
5.72
10.62
3.93
task-voxel
5.70
10.63
4.29
Appendix
Table 7: Hierarchy ablation on three representative transfer settings.
Task
RoboLDA
MorphVAE
LASeR
BO/GA
CPPN-NEAT
UpStepper-v0
4.62 ± 0.71
3.72 ± 1.81
1.55 ± 0.63
2.86 ± 0.82
4.03 ± 0.56
Climber-v1
5.92 ± 0.84
4.35 ± 0.71
0.89 ± 0.08
0.27 ± 0.04
1.88 ± 2.20
Climber-v2
1.78 ± 0.07
1.11 ± 0.43
1.43 ± 0.39
0.27 ± 0.04
0.97 ± 0.57
GapJumper-v0
5.78 ± 0.86
4.45 ± 1.10
4.29 ± 0.14
2.40 ± 0.67
4.47 ± 1.39
Appendix
Table 8: Zero-shot fitness (mean ± std) over 10 independent runs on four challenging transfer settings.
Figure 13: Walker-v0
Figure 14: Pusher-v0
Figure 15: Carrier-v0
Figure 16: BridgeWalker-v0
Figure 17: UpStepper-v0
Figure 18: DownStepper-v0
Figure 19: Climber-v0. The figure has been rotated 90 degrees clockwise to save space, where the vertical line on the left represents the ground. The same is true for Figures 20 and 21 .
Figure 20: Climber-v1
Figure 21: Climber-v2
Figure 22: PlatformJumper-v0
Figure 23: GapJumper-v0
Hyperparameter
Value
RoboLDA
robot size
5×5
number of robot types
6
number of organ types
6
number of organ components
10
hidden dims of MLPs
[128]*3
Appendix
Table 9: Hyperparameter settings.
Figure 24: Visualization of the best-performing rigid robot morphologies generated by different algorithms in each trial, as well as the ones optimized by the genetic algorithm. f denotes fitness of each robot, which is measured as the cumulative sum of rewards over a complete episode of 400 timesteps, when controlled by the optimized controller. Reward of each step is in turn assigned according to the velocity of the robot along the x -axis.
Figure 25: Additional visualizations of hierarchical structures learned by RoboLDA.
School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China · School of Statistics, Renmin University of China, Beijing 100872, China · Intelligent Game and Decision Laboratory, Beijing 100071, China +1