cs.LGOct 1, 2026
SaveCo-design Gym: A Unified Benchmark for Embodiment-Policy Co-optimization
Organizations: University of Cambridge · Technical University of Munich · Carnegie Mellon University · MorphoAI
Abstract
Finding an optimal behaviour policy within a given environment is a widely studied problem in domains as diverse as games, robotics, energy infrastructure, communication networks, and multi-agent systems. Numerous benchmarks have been developed to support such research, but the vast majority assume that the agent's embodiment (design) is fixed, focusing instead on policy learning alone. Lifting this assumption gives rise to a broader class of problems in which optimizing embodiment and policy separately is highly suboptimal. An agent's embodiment strongly shapes which control policies can be discovered, while the optimal embodiment is in turn defined by the policies it admits. To help the research community study this class of problems explicitly and systematically, we introduce Co-Design Gym - a suite of benchmark environments for jointly optimizing embodiment and policy. Our environments span domains such as robotic manipulation and locomotion, multi-robot cooperation, deformable and soft dynamics, video games, electricity grids, wireless networks, F1 racing, multi-agent warehouses, and optimal control, offering 20 environment families (domains), with over 85 distinct co-design presets in total. We further contribute a systematic evaluation of representative co-design algorithms, characterizing the current state of the art. Together, these contributions lay the groundwork for cumulative, comparable progress in co-design.
Figures & tables
Figure 1: Co-Design Gym overview with three environment categories. Native: new domains we developed to study the dynamic interplay between embodiment and policy, covering manipulation, locomotion and deformable body dynamics. Extended: classic MuJoCo control tasks from OpenAI Gym extended to support morphology variations. Reframed: existing problems from games, energy , networking, racing, logistics and control, recast as co-design tasks. Co-Design Gym is available at https://github.com/camral/codesign-gym
Figure 2: An illustrative embodiment-policy optimization landscape. The axes and represent policy and embodiment spaces; surface height represents expected return. The panels and label two fixed-embodiment slices. Each refers to optimization over only its panel’s embodiments. In this example, the green slice contains a joint optimum, giving .
Figure 3: Overview of the available environments. For additional illustrations of the environment presets, refer to Appendix A: Environments .
Figure 4: Solar Panel Cleaning environment.
Figure 5: The MorphHand. Colors show elevation at each vertex.
Figure 6: Soft Walkers environment.
Figure 7: Minimal usage of codesign_gym . A single entry point, make_vec , returns a natively parallelized environment together with its base embodiment supporting GPU-acceleration. An outer design optimizer proposes an embodiment that is passed to reset , and an inner policy then acts against it. The embodiment can optionally be re-proposed and passed to step at every timestep.
| Preset | CMA-ES | FastTD3 | PPO+NGOpt | LOKI |
|---|---|---|---|---|
| Native | ||||
| BallCatcher-Pitch | 0.296 0.121 | 3.42 1.62 | 0.435 0.325 | 0.0633 0.0021 |
| BallCatcher-Arc | 0.316 0.107 | 4.15 0.65 | 1.59 0.23 | 0.0875 0.0140 |
| BallCatcher-Fielding | 0.143 0.045 | 3.15 2.48 | 0.461 0.400 | 0.0739 0.0037 |
| NeroGraspAll | 2.34 0.43 | 250 0 | 0.147 0.181 | 0.0899 0.0195 |
| SolarCleanerReach | 0.0249 0.0015 | 0.00639 0.00182 | 0.186 0.270 | 0.00613 0.00207 |
Table 1: The Co-Design Gym’s first scoreboard: Final evaluation return (mean std over 5 seeds; higher is better) of each baseline under a matched 4-hour wall-clock budget (LOKI: 3 hours online after 1 hour of preparation). FastTD3 on NeroGraspAll: every proposed design failed structural validation, so every episode received the rejection reward ( ). Bold: best mean.
Figure 8: Leading baseline over training on the 16 presets run with all four baselines. Each cell names the method with the highest mean eval return (5 seeds) at that point, using the latest evaluation so far. CMA-ES leads early on 9 presets but at the end on only 4; FastTD3 ends ahead on 6, PPO+NGOpt and LOKI on 3 each. The leader changes at least once on 12 of 16 presets, and no method leads on every domain group, so the ranking of co-design baselines depends on both the task and the budget.
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Ball Catching environment.
Figure 10: Solar Panel Cleaning environment.
Figure 11: Truck Unloading environment.
Figure 12: Nero-LEAP Grasping environment.
Figure 13: MorphHand Rotation environment.
Figure 14: MorphHand Pick-Up environment.
Figure 15: Soft Walkers environment.
Figure 16: The Pokemon Showdown co-design environment.
Figure 17: Electrical Microgrid environment.
Figure 18: Wireless Network environment.
Figure 19: Car Racing environment.
Figure 20: Robotic Warehouse environment.
Figure 21: Linear-Quadratic Control environment.
Figure 22: WIP: An overview of the space of co-design algorithms highlighting our chosen baselines.
| Preset | CMA-ES | FastTD3 | PPO+NGOpt |
|---|---|---|---|
| Native | |||
| SolarCleanerTraverse | 0.0185 0.0159 | 0.00431 0.00335 | 0.00043 0.00133 (6) |
| Reframed | |||
| RacingLevine | 6.07 1.03 | 48.3 11.6 | 39.6 10.3 |
| WarehouseMedium | 0 0 | 0 0 | 0.0172 0.0125 |
| WarehouseCongested | 0 0 | 50 100 | 0.00313 0.00625 |
Table 2: Final evaluation return (mean std over 5 seeds; higher is better) on the remaining presets, run with CMA-ES, FastTD3 and PPO+NGOpt under a matched 4-hour wall-clock budget. Bold: best mean. † : runs still in progress. (n) : seeds.
Figure 23: BallCatcher-Pitch. Only FastTD3, which emits the design as the first action of each episode, learns to catch (16.6% full catches vs. 1.1% for the others; final return 3.42 vs. 0.44). Its final return varies widely across seeds (1.35–5.98), so the joint design-and-policy agent is effective here but unreliable. None of the outer-loop methods reaches catching behaviour within the budget.
Figure 24: Hopper. The four families trace four different learning curves on the same task, and the leader changes six times in four hours. FastTD3 peaks early (2,800 at 0.6 h) and decays to 897; PPO+NGOpt jumps late (761 1,588 in the final hour, seeds 462–2,885); LOKI improves steadily throughout; CMA-ES plateaus at 1,260 almost immediately. A single budget checkpoint would rank the methods differently at almost every time.
Figure 25: RacingSpielberg. PPO+NGOpt finds a strong vehicle setup within the first hour (eval 157 at 1 h) and then loses it, ending at 64 after 245 design rounds: more outer-loop design updates are not monotonically better, and a good design found early is not retained. Its train return ( 13) stays far below eval because most rounds train on exploratory setups. FastTD3 is stable at 77; CMA-ES never leaves 7.
Figure 26: RacingLevine. The same pattern as on RacingSpielberg on a second track: PPO+NGOpt peaks within the first hour (63) and declines to 40 over 274 design rounds, while FastTD3 improves steadily, overtakes it and ends with the most completed laps (37% vs. 20%). Early design-search gains that are not retained recur across tracks, so they reflect the algorithm rather than a single environment.
Figure 27: Final designs on BallCatcher-Pitch , one per run, taken from the first frame of the run’s final evaluation rollout and shown at a common scale; seeds are sorted by final eval return (above each image) and the best is framed thick. The methods converge to visibly different bodies, and FastTD3 and LOKI also differ from seed to seed, whereas CMA-ES’s designs look alike: similar returns can come from different morphologies.
Figure 28: Train vs. eval return on every preset run with all four baselines (mean over 5 seeds; one block of panels per preset, sharing a y-axis). Train return averages over the designs being trained or explored and eval return uses the selected design, so the gap measures each method’s cost of design exploration; for CMA-ES on the Warehouse presets the train return also includes the penalty for rejected designs. PPO+NGOpt does not log a train return on HalfCheetah, Swimmer and HumanoidStandup, and its HumanoidStandup runs were still in progress when this figure was generated.
Figure 29: Prompt used by the PPO+LLM outer loop on BallCatcher-Pitch. Placeholders of the form {VAR} are replaced with the corresponding values at query time.
| Run | Stopped at | PPO iters | Design rounds | Final eval | Peak eval |
|---|---|---|---|---|---|
| PPO+NGOpt (5 seeds) | 4 h (budget) | 70 | 69 | 0.43 0.33 | 0.59 0.46 |
Table 3: PPO+LLM (Gemma 4 31B, 16 templates per round, perturbation std 0.05, 16,384 environments) on BallCatcher-Pitch, per seed, against the PPO+NGOpt baseline.
Figure 30: PPO+LLM vs. PPO+NGOpt on BallCatcher-Pitch , plotted against PPO iterations because both algorithms run the same PPO inner loop and the PPO+LLM runs stopped early ( ). PPO+NGOpt’s return (5-seed mean std) rises from 0.08 to 0.43 at evaluation, whereas both PPO+LLM runs stay at or near zero in training ( 0.005) and at every evaluation ( 0.02), over a comparable number of PPO iterations (68 and 59 vs. 68–71).
Figure 31: Every distinct evaluated design (seed 0, illustrative) for PPO+LLM (amber frames) and PPO+NGOpt (aqua frames): the initial design and the design after each time a better-scoring candidate replaced it (PPO+LLM at 0.21, 0.64 and 2.84 h; PPO+NGOpt once, at 1.91 h), each taken from the first frame of the next evaluation video and shown at a common scale (title: PPO iteration and eval return of that evaluation).
Figure 32: Co-design Gym API flowchart.
More in this topic
Robot Morphology-Control Co-Design
Robot morphology-control co-design studies how to jointly shape a robot’s body and controller to improve task performance.
See all 60 papers