Finding an optimal behaviour policy within a given environment is a widely studied problem in domains as diverse as games, robotics, energy infrastructure, communication networks, and multi-agent systems. Numerous benchmarks have been developed to support such research, but the vast majority assume that the agent's embodiment (design) is fixed, focusing instead on policy learning alone. Lifting this assumption gives rise to a broader class of problems in which optimizing embodiment and policy separately is highly suboptimal. An agent's embodiment strongly shapes which control policies can be discovered, while the optimal embodiment is in turn defined by the policies it admits. To help the research community study this class of problems explicitly and systematically, we introduce Co-Design Gym - a suite of benchmark environments for jointly optimizing embodiment and policy. Our environments span domains such as robotic manipulation and locomotion, multi-robot cooperation, deformable and soft dynamics, video games, electricity grids, wireless networks, F1 racing, multi-agent warehouses, and optimal control, offering 20 environment families (domains), with over 85 distinct co-design presets in total. We further contribute a systematic evaluation of representative co-design algorithms, characterizing the current state of the art. Together, these contributions lay the groundwork for cumulative, comparable progress in co-design.
Figures & tables
Figure 1: Co-Design Gym overview with three environment categories. Native: new domains we developed to study the dynamic interplay between embodiment and policy, covering manipulation, locomotion and deformable body dynamics. Extended: classic MuJoCo control tasks from OpenAI Gym extended to support morphology variations. Reframed: existing problems from games, energy , networking, racing, logistics and control, recast as co-design tasks. Co-Design Gym is available at https://github.com/camral/codesign-gym
Figure 2: An illustrative embodiment-policy optimization landscape. The axes Π and E represent policy and embodiment spaces; surface height represents expected return. The panels eα and eβ label two fixed-embodiment slices. Each J⋆(e) refers to optimization over only its panel’s embodiments. In this example, the green slice contains a joint optimum, giving J⋆(e)=J⋆⋆(I) .
Figure 3: Overview of the available environments. For additional illustrations of the environment presets, refer to Appendix A: Environments .
Figure 4: Solar Panel Cleaning environment.
Figure 5: The MorphHand. Colors show elevation at each vertex.
Figure 6: Soft Walkers environment.
Figure 7: Minimal usage of codesign_gym . A single entry point, make_vec , returns a natively parallelized environment together with its base embodiment supporting GPU-acceleration. An outer design optimizer proposes an embodiment that is passed to reset , and an inner policy then acts against it. The embodiment can optionally be re-proposed and passed to step at every timestep.
Preset
CMA-ES
FastTD3
PPO+NGOpt
LOKI
Native
BallCatcher-Pitch
0.296 ± 0.121
3.42 ± 1.62
0.435 ± 0.325
0.0633 ± 0.0021
BallCatcher-Arc
0.316 ± 0.107
4.15 ± 0.65
1.59 ± 0.23
0.0875 ± 0.0140
BallCatcher-Fielding
0.143 ± 0.045
3.15 ± 2.48
0.461 ± 0.400
0.0739 ± 0.0037
NeroGraspAll
2.34 ± 0.43
− 250 ± 0
0.147 ± 0.181
0.0899 ± 0.0195
SolarCleanerReach
0.0249 ± 0.0015
0.00639 ± 0.00182
0.186 ± 0.270
0.00613 ± 0.00207
Table 1: The Co-Design Gym’s first scoreboard: Final evaluation return (mean ± std over 5 seeds; higher is better) of each baseline under a matched 4-hour wall-clock budget (LOKI: 3 hours online after 1 hour of preparation). FastTD3 on NeroGraspAll: every proposed design failed structural validation, so every episode received the rejection reward ( −250 ). Bold: best mean.
Figure 8: Leading baseline over training on the 16 presets run with all four baselines. Each cell names the method with the highest mean eval return (5 seeds) at that point, using the latest evaluation so far. CMA-ES leads early on 9 presets but at the end on only 4; FastTD3 ends ahead on 6, PPO+NGOpt and LOKI on 3 each. The leader changes at least once on 12 of 16 presets, and no method leads on every domain group, so the ranking of co-design baselines depends on both the task and the budget.
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Ball Catching environment.
Figure 10: Solar Panel Cleaning environment.
Figure 11: Truck Unloading environment.
Figure 12: Nero-LEAP Grasping environment.
Figure 13: MorphHand Rotation environment.
Figure 14: MorphHand Pick-Up environment.
Figure 15: Soft Walkers environment.
Figure 16: The Pokemon Showdown co-design environment.
Figure 17: Electrical Microgrid environment.
Figure 18: Wireless Network environment.
Figure 19: Car Racing environment.
Figure 20: Robotic Warehouse environment.
Figure 21: Linear-Quadratic Control environment.
Figure 22: WIP: An overview of the space of co-design algorithms highlighting our chosen baselines.
Preset
CMA-ES
FastTD3
PPO+NGOpt
Native
SolarCleanerTraverse
0.0185 ± 0.0159
0.00431 ± 0.00335
0.00043 ± 0.00133 (6)
Reframed
RacingLevine
6.07 ± 1.03
48.3 ± 11.6
39.6 ± 10.3
WarehouseMedium
0 ± 0
0 ± 0
0.0172 ± 0.0125
WarehouseCongested
0 ± 0
− 50 ± 100
0.00313 ± 0.00625
Appendix
Table 2: Final evaluation return (mean ± std over 5 seeds; higher is better) on the remaining presets, run with CMA-ES, FastTD3 and PPO+NGOpt under a matched 4-hour wall-clock budget. Bold: best mean. † : runs still in progress. (n) : n=5 seeds.
Figure 23: BallCatcher-Pitch. Only FastTD3, which emits the design as the first action of each episode, learns to catch (16.6% full catches vs. ≤ 1.1% for the others; final return 3.42 vs. ≤ 0.44). Its final return varies widely across seeds (1.35–5.98), so the joint design-and-policy agent is effective here but unreliable. None of the outer-loop methods reaches catching behaviour within the budget.
Figure 24: Hopper. The four families trace four different learning curves on the same task, and the leader changes six times in four hours. FastTD3 peaks early (2,800 at 0.6 h) and decays to 897; PPO+NGOpt jumps late (761 → 1,588 in the final hour, seeds 462–2,885); LOKI improves steadily throughout; CMA-ES plateaus at ≈ 1,260 almost immediately. A single budget checkpoint would rank the methods differently at almost every time.
Figure 25: RacingSpielberg. PPO+NGOpt finds a strong vehicle setup within the first hour (eval 157 at 1 h) and then loses it, ending at 64 after ≈ 245 design rounds: more outer-loop design updates are not monotonically better, and a good design found early is not retained. Its train return ( ≈ 13) stays far below eval because most rounds train on exploratory setups. FastTD3 is stable at ≈ 77; CMA-ES never leaves ≈ 7.
Figure 26: RacingLevine. The same pattern as on RacingSpielberg on a second track: PPO+NGOpt peaks within the first hour (63) and declines to 40 over ≈ 274 design rounds, while FastTD3 improves steadily, overtakes it and ends with the most completed laps (37% vs. 20%). Early design-search gains that are not retained recur across tracks, so they reflect the algorithm rather than a single environment.
Figure 27: Final designs on BallCatcher-Pitch , one per run, taken from the first frame of the run’s final evaluation rollout and shown at a common scale; seeds are sorted by final eval return (above each image) and the best is framed thick. The methods converge to visibly different bodies, and FastTD3 and LOKI also differ from seed to seed, whereas CMA-ES’s designs look alike: similar returns can come from different morphologies.
Figure 28: Train vs. eval return on every preset run with all four baselines (mean over 5 seeds; one block of panels per preset, sharing a y-axis). Train return averages over the designs being trained or explored and eval return uses the selected design, so the gap measures each method’s cost of design exploration; for CMA-ES on the Warehouse presets the train return also includes the penalty for rejected designs. PPO+NGOpt does not log a train return on HalfCheetah, Swimmer and HumanoidStandup, and its HumanoidStandup runs were still in progress when this figure was generated.
Figure 29: Prompt used by the PPO+LLM outer loop on BallCatcher-Pitch. Placeholders of the form {VAR} are replaced with the corresponding values at query time.
Run
Stopped at
PPO iters
Design rounds
Final eval
Peak eval
PPO+NGOpt (5 seeds)
4 h (budget)
70
69
0.43 ± 0.33
0.59 ± 0.46
Appendix
Table 3: PPO+LLM (Gemma 4 31B, 16 templates per round, perturbation std 0.05, 16,384 environments) on BallCatcher-Pitch, per seed, against the PPO+NGOpt baseline.
Figure 30: PPO+LLM vs. PPO+NGOpt on BallCatcher-Pitch , plotted against PPO iterations because both algorithms run the same PPO inner loop and the PPO+LLM runs stopped early ( × ). PPO+NGOpt’s return (5-seed mean ± std) rises from ≈ 0.08 to 0.43 at evaluation, whereas both PPO+LLM runs stay at or near zero in training ( ≤ 0.005) and at every evaluation ( ≤ 0.02), over a comparable number of PPO iterations (68 and 59 vs. 68–71).
Figure 31: Every distinct evaluated design (seed 0, illustrative) for PPO+LLM (amber frames) and PPO+NGOpt (aqua frames): the initial design and the design after each time a better-scoring candidate replaced it (PPO+LLM at 0.21, 0.64 and 2.84 h; PPO+NGOpt once, at 1.91 h), each taken from the first frame of the next evaluation video and shown at a common scale (title: PPO iteration and eval return of that evaluation).
We propose to turn generalist multi-embodiment value functions into reusable models for robot design. Instead of running a new reinforcement learning co-design loop for each robot, we first train an embodiment-aware policy and value function across many robot designs. After training, the frozen value function is used as a differentiable surrogate to optimize candidate embodiments through value gradients. We evaluate our approach across different robot design settings, from perturbed single robots to held-out robots across morphology classes, with single models trained on up to 50 robots and design spaces of over 1100 continuous embodiment parameters. Beyond optimizing complete embodiments, we show that value gradients can identify performance-limiting design and control parameters, enabling both the optimization and the analysis of new robot designs.
Nico Bohlinger, Jan Peters
Technical University of Darmstadt, Germany · Robotics Institute Germany (RIG); German Research Center for AI (DFKI); hessian.AI
Morphology--control co-design jointly optimizes an agent's body structure and control policy as an integrated embodied system. However, existing methods typically model morphology design and control with separate networks coupled only indirectly through a shared task objective, limiting explicit high-level coordination. Inspired by natural genes that coordinate biological development, we introduce Morphogene, a compact latent blueprint that bridges an agent's body and brain. Through AdaConcat, Morphogene jointly conditions morphology and control generation at the limb level, allowing its variations to induce coordinated changes in both components. Building on this representation, we propose GeCode, which formulates co-design as exploration in the compact Morphogene space. Each Morphogene anchors a local design region in which nearby body--brain designs are explored, while performance-guided updates move these anchors toward promising regions for more efficient exploration of the broader design space. This process combines local refinement with global exploration while preserving body--brain compatibility. Extensive experiments across diverse 2D and 3D co-design tasks demonstrate that GeCode consistently outperforms existing state-of-the-art methods, achieving substantially faster convergence and higher final performance.
Fu Feng, Ruixiao Shi, Yucheng Xie +2
School of Computer Science and Engineering, Southeast University, Nanjing, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China
Co-design is a high-dimensional search problem in the robot morphology and control design space. Efficient search requires exploiting the structure shaped by their interaction. To understand this structure, we analyze the landscapes of soft locomotion and manipulation tasks. We identify three patterns consistent across regions of their co-design spaces: 1) Within a region, quality varies along a low-dimensional manifold, with minimal variation orthogonal to it, reducing the effective search space dimensionality. 2) In higher-quality regions, the variance in quality is spread across more dimensions, necessitating search to expand dimensionality as quality improves. 3) In higher-quality regions, quality varies along joint morphology-control dimensions, requiring search along them. Using these insights, we devise an efficient co-design algorithm that yields 36% better co-designs than state-of-the-art baselines. We examine their exploration patterns and show that these baselines required an order of magnitude more function evaluations to find co-designs of comparable quality. Finally, we ablate our algorithm to verify that exploiting the identified structure was the key to efficient co-design.
Apoorv Vaish, Oliver Brock
Robotics and Biology Laboratory, Technische Universität Berlin · Science of Intelligence (SCIoI), Cluster of Excellence, Berlin · Robotics Institute Germany (RIG)