Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization

Yanning Dai; Yuhui Wang; Dylan R. Ashley; Jürgen Schmidhuber

cs.LGMar 16, 2026

Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization

Authors: Yanning Dai, Yuhui Wang, Dylan R. Ashley, Jürgen Schmidhuber

Organizations: Center of Excellence for Generative AI, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia. · Dalle Molle Institute for Artificial Intelligence Research (IDSIA), Lugano, Switzerland. · Università della Svizzera italiana (USI), Lugano, Switzerland. · Scuola universitaria professionale della Svizzera italiana (SUPSI), Lugano, Switzerland.

Abstract

Morphology-control co-design concerns the coupled optimization of an agent's body structure and control policy. This problem exhibits a bi-level structure, where the control dynamically adapts to the morphology to maximize performance. Existing methods typically neglect the control's adaptation dynamics by adopting a single-level formulation that treats the control policy as fixed when optimizing morphology. This can lead to inefficient optimization, as morphology updates may be misaligned with control adaptation. In this paper, we revisit the co-design problem from a game-theoretic perspective, modeling the intrinsic coupling between morphology and control as a novel variant of a Stackelberg game. We propose Stackelberg Proximal Policy Optimization (Stackelberg PPO), which explicitly incorporates the control's adaptation dynamics into morphology optimization. By modeling this intrinsic coupling, our method aligns morphology updates with control adaptation, thereby stabilizing training and improving learning efficiency. Experiments across diverse co-design tasks demonstrate that Stackelberg PPO outperforms standard PPO in both stability and final performance, opening the way for dramatically more efficient robotics designs.

Figures & tables

Figure 1: Showcasing how our Stackelberg PPO can autonomously design task-specific robots for the “Pusher” task, starting with a bare-bones structure and ultimately evolving into a sophisticated design with arm-like structures for pushing boxes and leg-like limbs for movement. This highlights the method’s ability to create adaptive and complex designs. In comparison, the traditional PPO method generates simpler structures that can’t support more complex behaviors. For more examples of evolved designs and animations, as well as open-source code, visit: https://yanningdai.github.io/stackelberg-ppo-co-design .

Figure 2: Illustration of the phase-separated Stackelberg Markov Game for morphology–control co-design. In the leader phase (blue part), the agent incrementally edits the morphology via discrete topology-altering actions, producing a terminal morphology $s_{T}^{L}$ . In the follower phase (green part), the control policy is optimized based on this morphology.

Panel of Figure 3 — Figure 3: Performance curve with respect to the number of follower steps during training. Shaded regions denote standard error across seven random seeds.

Panel of Figure 4 — Figure 4: (a) Evolved morphologies. Ablation studies on (b) $\lambda$ sweep from 0 to $\infty$ and (c) Fisher information matrix on/off, both evaluated on Stepper-Regular task.

Panel of Figure 5 — Figure 5: (a) Reward curves and (b) KL-divergence traces for different clipping thresholds $\epsilon$ . (c) Performance comparison under varying leader horizons $T$ . All evaluated on Stepper-Regular.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Visualization of the nine benchmark environments used in our experiments. Crawler, Stepper-Regular, Stepper-Hard, and Pusher are 3D tasks; others are 2D (x-z or x-y plane). The environments differ substantially in required morphology depth, symmetry, and limb arrangement, enabling evaluation on the generality of morphology–control co-design across locomotion and manipulation tasks.

Hyperparameter	Value
Fisher Regularization Coefficient $\lambda$	5.0
Maximum Conjugate Gradient Steps	20
CG Relative Error Tolerance	$10^{-3}$
Follower Sampling Steps per Episode	6
Gradient Normalization Ratio $\alpha$	1.0
Structure Design Steps $T^{stru}$	5

Appendix

Table 1: Hyperparameters of Stackelberg PPO adopted in all experiments

Figure 7: Visualization of co-evolved body designs generated through our Stackelberg PPO.

Methods	Crawler	Cheetah	Swimmer
Stackelberg PPO (Ours)	$11047.90_{\pm 126.20}$	$13514.94_{\pm 653.62}$	$1334.98_{\pm 16.06}$
BodyGen ( Lu et al., 2025 )	$9098.72_{\pm 558.26}$	$11575.87_{\pm 640.65}$	$1302.64_{\pm 3.71}$
Transform2Act ( Yuan et al., 2022 )	$3950.80_{\pm 268.43}$	$8297.90_{\pm 825.02}$	$737.90_{\pm 21.04}$
NGE ( Wang et al., 2019 )	$1482.45_{\pm 524.97}$	$2534.76_{\pm 428.68}$	$384.45_{\pm 112.03}$
ESS ( Sims, 1994 )	$631.67_{\pm 122.41}$	$671.67_{\pm 134.65}$	$190.62_{\pm 37.84}$
Methods	Walker-Hard	Glider-Hard	TerrainCrosser

Appendix

Table 2: Performance comparison of Stackelberg PPO against baseline methods across morphology-control co-design environments. Results show mean episode rewards and standard deviations over seven random seeds.

Panel of Figure 8 — Figure 8: Extended evaluation on Glider and Walker environments under different morphology complexity budgets.(a) Training curves for the regular, medium, and hard variants of each environment. (b) Final generated morphologies under each complexity tier.

Clipping Parameter	Performance	Likelihood Ratio Violations (%)	Average KL Divergence
$\epsilon=0.1$	$4934.52_{\pm 646.40}$	$14.82_{\pm 0.55}$	$0.0030_{\pm 0.0006}$
$\epsilon=0.2$	$7215.20_{\pm 449.02}$	$13.39_{\pm 0.83}$	$0.0196_{\pm 0.0025}$
$\epsilon=0.4$	$\mathbf{7907.01_{\pm 208.02}}$	$9.92_{\pm 0.94}$	$0.0343_{\pm 0.0153}$
$\epsilon=0.6$	$4778.18_{\pm 407.84}$	$9.10_{\pm 1.30}$	$0.0665_{\pm 0.0188}$
$\epsilon=0.8$	$2656.92_{\pm 503.93}$	$7.02_{\pm 0.81}$	$0.1340_{\pm 0.0388}$
No Clipping	$1233.26_{\pm 443.98}$	$0$	$1.7726_{\pm 0.1539}$

Appendix

Table 3: Sensitivity of Stackelberg PPO to the clipping threshold

\epsilon

Environment	SID+PPO (full)	PPO-only (no SID)	SID-only (no clipping)
Stepper-Regular	$\mathbf{7215.20_{\pm 449.02}}$	$4685.94_{\pm 845.23}$	$1257.33_{\pm 530.25}$
Crawler	$\mathbf{11047.90_{\pm 126.20}}$	$9098.72_{\pm 558.26}$	$35.77_{\pm 12.25}$
Cheetah	$\mathbf{13514.94_{\pm 653.62}}$	$11575.87_{\pm 640.65}$	$472.89_{\pm 77.40}$
Glider	$\mathbf{12414.50_{\pm 498.53}}$	$11049.95_{\pm 468.44}$	$566.81_{\pm 89.96}$

Appendix

Table 4: Ablation studies on the components of our SID estimator and PPO clipping, evaluated under the same phase-separated, non-differentiable Stackelberg setting.

Figure 9: Comparison of morphology evolution between Stackelberg PPO (ours) and Standard PPO (BodyGen) across four environments. BodyGen tends to collapse early into low-complexity designs, while Stackelberg PPO continues exploring structurally richer morphologies, yielding more capable final structures.

	Stackelberg PPO (Ours)	BodyGen (PPO)
Performance Change After 1 Epoch	$\mathbf{+0.392_{\pm 0.075}\%}$	$+0.224_{\pm 0.043}\%$

Appendix

Table 5: Average performance change after one epoch of training from the same checkpoint model, averaged over 10 checkpoints and 7 seeds, evaluated on Stepper-Regular.

	Stackelberg PPO (10 cores)	BodyGen (10 cores)	NGE (10 cores)	NGE (64 cores)
Wall-clock Time	$33.07_{\pm 0.49}$ h	$29.43_{\pm 0.97}$ h	$45.16_{\pm 3.72}$ h	$13.52_{\pm 1.52}$ h

Appendix

Table 8: Wall-clock training time across methods. NGE results are shown under both 10 CPU cores and 64 cores to illustrate parallelization effects.

Figure 10: Morphological evolution trajectories across eight environments. Each row represents a distinct robot embodiment, with columns showing progressive stages of morphological adaptation from 5% to 100% training progress.

Figure 11: Power constraint under different penalty coefficients. As the penalty increases, the co-designed morphologies transition toward shorter, thicker, and more symmetric limbs.

Penalty Coef.	Performance		Power Penalty
Penalty Coef.	Stackelberg PPO (ours)	BodyGen	Stackelberg PPO (ours)	BodyGen
$0.0001$	$\mathbf{11047.90_{\pm 126.20}}$	$9098.72_{\pm 558.26}$	$5631.42_{\pm 674.03}$	$2745.84_{\pm 284.55}$
$0.001$	$\mathbf{10191.15_{\pm 371.81}}$	$7501.61_{\pm 671.33}$	$5342.16_{\pm 584.25}$	$5948.16_{\pm 254.84}$
$0.01$	$\mathbf{9853.19_{\pm 229.37}}$	$8304.00_{\pm 497.56}$	$1582.48_{\pm 697.61}$	$468.12_{\pm 516.33}$
$0.1$	$\mathbf{10585.25_{\pm 146.80}}$	$8974.48_{\pm 574.29}$	$25.50_{\pm 22.27}$	$26.34_{\pm 23.64}$

Appendix

Table 9: Power-constraint setting: performance and power penalties under different penalty coefficients.

Figure 12: Manufacturability constraint under different penalty coefficients. Higher penalties on structure complexity and material mass encourage designs with fewer body elements, reduced branching, and mass concentrated near the root, producing compact morphologies that are easier to fabricate.

Penalty Coef.	Performance		Morphology Complexity		Material Cost
Penalty Coef.	Stackelberg PPO (ours)	BodyGen	Stackelberg PPO (ours)	BodyGen	Stackelberg PPO (ours)	BodyGen
$0$	$\mathbf{11047.90_{\pm 126.20}}$	$9098.72_{\pm 558.26}$	$16.40_{\pm 2.45}$	$13.67_{\pm 2.08}$	$1.71_{\pm 0.23}$	$1.57_{\pm 0.32}$
$1$	$\mathbf{7892.93_{\pm 349.84}}$	$6531.37_{\pm 437.26}$	$13.67_{\pm 2.03}$	$9.67_{\pm 1.61}$	$1.59_{\pm 0.18}$	$1.32_{\pm 0.16}$
$10$	$\mathbf{6825.47_{\pm 303.09}}$	$5372.10_{\pm 364.79}$	$8.25_{\pm 0.91}$	$7.50_{\pm 1.24}$	$0.94_{\pm 0.08}$	$0.93_{\pm 0.10}$

Appendix

Table 10: Manufacturability constraint setting: performance, morphology complexity, and material cost under different penalty levels.

Figure 13: Torque limits constraint under different penalty coefficients. Tighter actuation limits lead to noticeably simpler and more compact structures, with shorter limbs and reduced distal branching.

Penalty Coef.	Performance		Limit Violation Penalty
Penalty Coef.	Stackelberg PPO (ours)	BodyGen	Stackelberg PPO (ours)	BodyGen
$0.01$	$\mathbf{7893.42_{\pm 84.62}}$	$6311.75_{\pm 98.31}$	$20210.50_{\pm 6503.22}$	$11350.45_{\pm 5338.31}$
$0.1$	$\mathbf{3133.01_{\pm 70.44}}$	$3121.80_{\pm 54.03}$	$1106.45_{\pm 64.69}$	$899.35_{\pm 49.40}$

Appendix

Table 11: Torque limits constraint: performance and torque-violation penalties under different torque-penalty coefficients.

Figure 14: Morphology comparison trained with and without payload. Payload induces more symmetric and load-supporting structures.

Payload Weight	Stackelberg PPO (ours)	BodyGen
$0.2$ kg	$\mathbf{8675.08_{\pm 286.42}}$	$5186.31_{\pm 659.87}$
$0.4$ kg	$\mathbf{6966.34_{\pm 473.43}}$	$5116.54_{\pm 546.55}$
$0.6$ kg	$\mathbf{7347.46_{\pm 478.01}}$	$4523.89_{\pm 536.99}$

Appendix

Table 12: Payload constraint: performance comparison under different payload weights.

Level	Stackelberg PPO (ours)	BodyGen
$2.0$ N	$\mathbf{11557.31_{\pm 124.68}}$	$6963.05_{\pm 450.48}$
$4.0$ N	$\mathbf{11290.13_{\pm 164.54}}$	$4621.82_{\pm 597.71}$
$6.0$ N	$\mathbf{10875.23_{\pm 250.97}}$	$2816.16_{\pm 857.83}$

Appendix

Table 13: Robustness evaluation: performance under different levels of external disturbance forces.

Level	Stackelberg PPO (ours)	BodyGen
$30\%$	$\mathbf{11424.66_{\pm 112.08}}$	$7326.04_{\pm 421.73}$
$50\%$	$\mathbf{11333.43_{\pm 141.42}}$	$6795.55_{\pm 493.31}$
$70\%$	$\mathbf{10892.09_{\pm 149.72}}$	$5062.85_{\pm 579.66}$

Appendix

Table 14: Robustness evaluation: performance under different levels of terrain friction noise.

Figure 15: Visualization the unseen obstacle-navigation task environment.

Obstacle Type	Spacing	Performance
Obstacle Type	Spacing	Stackelberg PPO (ours)	BodyGen
Sparse Obstacle	$16$ m ( $\sim$ 4× robot width)	$\mathbf{1790.45_{\pm 161.77}}$	$1061.55_{\pm 228.40}$
Dense Obstacle	$8$ m ( $\sim$ 2× robot width)	$\mathbf{1698.52_{\pm 733.02}}$	$1007.21_{\pm 157.42}$

Appendix

Table 15: Performance in the unseen obstacle-navigation task under two obstacle densities.

Figure 16: Cross-task reuse of morphology priors: Crawler prior (left) and the resulting Pusher morphology (right).

Condition	Performance		Steps to Threshold (2500 Reward)
Condition	Stackelberg PPO (ours)	BodyGen	Stackelberg PPO (ours)	BodyGen
With Prior	$\mathbf{4822.59_{\pm 114.32}}$	$4575.52_{\pm 112.78}$	$\mathbf{\sim 8\text{M}}$	$\sim 9\text{M}$
Without Prior	$\mathbf{3462.77_{\pm 368.09}}$	$2779.95_{\pm 509.18}$	$\mathbf{\sim 32\text{M}}$	$\sim 44\text{M}$

Appendix

Table 16: Performance and sample efficiency in the Pusher task with and without morphology priors.

Robot Morphology-Control Co-Design

Robot morphology-control co-design studies how to jointly shape a robot’s body and controller to improve task performance.

See all 60 papers

Bridging Body and Brain: Gene-Driven Morphology--Control Co-Design
Sep 25, 2026Fu Feng, Ruixiao Shi, Yucheng Xie +2Robot Morphology-Control Co-Design Latent Space Optimization
School of Computer Science and Engineering, Southeast University, Nanjing, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China
Evolving Embodied Intelligence: Graph Neural Network--Driven Co-Design of Morphology and Control in Soft Robotics
Mar 20, 2026Jianqiang Wang, Shuaiqun Pan, Alvaro Serra-Gomez +2Robot Morphology-Control Co-Design Robot Policy Adaptation
LIACS, Leiden University · School of Mechanical Engineering, Xi’an Jiaotong University · Department of Computer Science, Loughborough University
Shaping the Evolutionary Dynamics of Robot Morphology via Adaptive Control Learning
Aug 24, 2026Junru Song, Yang Yang, Yaqing Xu +5Robot Morphology-Control Co-Design Evolutionary Robotics
Shanghai Jiao Tong University, Shanghai 200240, China · Intelligent Game and Decision Laboratory, Beijing 100048, China · Renmin University of China, Beijing 100872, China +1

Sep 25, 2026J/K move · Enter open · S save

Bridging Body and Brain: Gene-Driven Morphology--Control Co-Design

Fu Feng, Ruixiao Shi, Yucheng Xie, Jing Wang+1

School of Computer Science and Engineering, Southeast University, Nanjing, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China

Robot Morphology-Control Co-Design Latent Space Optimization Evolutionary Robotics

Open paper AskPDF arXiv

Morphology--control co-design jointly optimizes an agent's body structure and control policy as an integrated embodied system. However, existing methods typically model morphology design and control with separate networks coupled only indirectly through a shared task objective, limiting explicit high-level coordination. Inspired by natural genes that coordinate biological development, we introduce Morphogene, a compact latent blueprint that bridges an agent's body and brain. Through AdaConcat, Morphogene jointly conditions morphology and control generation at the limb level, allowing its variations to induce coordinated changes in both components. Building on this representation, we propose GeCode, which formulates co-design as exploration in the compact Morphogene space. Each Morphogene anchors a local design region in which nearby body--brain designs are explored, while performance-guided updates move these anchors toward promising regions for more efficient exploration of the broader design space. This process combines local refinement with global exploration while preserving body--brain compatibility. Extensive experiments across diverse 2D and 3D co-design tasks demonstrate that GeCode consistently outperforms existing state-of-the-art methods, achieving substantially faster convergence and higher final performance.

Abstract

Robot Morphology-Control Co-Design

Explore similar work

Bridging Body and Brain: Gene-Driven Morphology--Control Co-Design

Evolving Embodied Intelligence: Graph Neural Network--Driven Co-Design of Morphology and Control in Soft Robotics

Shaping the Evolutionary Dynamics of Robot Morphology via Adaptive Control Learning