MOCHA: Multi-Objective Co-Design using Hypernetwork Architectures

Varun Madabushi; Neil Janwani; Maegan Tucker

cs.ROSep 24, 2026

MOCHA: Multi-Objective Co-Design using Hypernetwork Architectures

Authors: Varun Madabushi, Neil Janwani, Maegan Tucker

Organizations: Dynamic Mobility Lab at Georgia Tech, Atlanta, U.S.

Abstract

In this work, we present MOCHA, the first, to our knowledge, reinforcement learning based approach to computing a family of Pareto-optimal policies across the design space of a robot using a single network. Specifically, MOCHA leverages the hypernetwork architecture to learn a network that produces specialized network parameters optimized for a given objective and parameterized robot design; we term this a multi-objective design hypernetwork (MDH). We demonstrate the capabilities of MDHs to represent a complex family of design-dependent strategies on two distinct robot morphologies, each with six design dimensions and across 2-3 objectives. Moreover, we propose an approach for efficiently producing a Design Pareto set using evolutionary search of the learned policy network, generating the optimal design-policy combination for each objective prioritization. Lastly, we provide an efficient method for computing generalist robot designs which achieve the best cumulative performance across the entire set of objectives.

Figures & tables

Fig. 1 : By learning a control policy over multiple objectives and designs, our algorithm is capable of finding the optimal design for a given objective prioritization. For example, (a) is optimized for jumping, (b) is optimized for energy consumption, and (c) is optimized for running speed.

Fig. 2 : The MO-UP Hypernetwork is trained by first sampling $M$ designs and $K$ trade-offs. The rewards obtained from rolling out these designs and trade-offs are used to update the value and policy networks using Multi-Objective PPO.

Fig. 3 : Some examples of procedurally generated Cheetah and Walker robots.

Fig. 4 : The Design Pareto frontier of the Cheetah sliced along the Run-Energy, Run-Height, and Energy-Height axes.

Network	Env.	Hypervolume	Spacing	Train Time
MDH	Cheetah	6.838e9	5.195e-2	2h 6m
MDMLP	Cheetah	5.353e9	0.1224	6h 20m
MDH	Walker	2.715e6	5.656e-2	1h 48m
MDMLP	Walker	2.576e6	9.566e-2	5h 55m

TABLE I : Performance of the hypernetwork algorithm on Walker and Cheetah Environments compared to the MLP baseline

Fig. 5 : Design Pareto frontier for the 2-objective Walker, and some rollouts of Pareto-optimal designs.

Fig. 6 : Comparison of the Design Pareto Frontier metrics attained by a Hypernetwork and MLP policy for the Cheetah (top) and Walker (bottom) environments. The Hypernetwork achieves a higher hypervolume and lower spacing than the MLP in both environments.

Fig. 7 : Fixed-Design Pareto frontiers for the generalist Cheetah, and a few selected specialists. Note that the run-optimized design far outperforms the others at running and energy savings, and the generalist design performs approximately similar to the height-optimized design.

Fig. 8 : Procedurally-generated generalist and specialist Cheetah robots obtained using MOCHA. The generalist robot maximizes the hypervolume of its associated Pareto frontier, while each task-optimized design maximizes the scalar reward according to its objective prioritization.

Robot Morphology-Control Co-Design

Robot morphology-control co-design studies how to jointly shape a robot’s body and controller to improve task performance.

See all 60 papers

MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Robotics
Mar 10, 2026Neil Janwani, Ellen Novoseller, Vernon J. Lawhern +1GPU-Accelerated RL Multi-Objective Reinforcement Learning
Georgia Institute of Technology, Atlanta, GA USA · DEVCOM Army Research Laboratory, MD, USA
Pareto Q-Learning with Reward Machines
Jun 17, 2026Arnaud Lequen, Clément Legrand-Lixon, Léo SaulièresMulti-Objective Reinforcement Learning Reinforcement Learning
Linköping University, Sweden · Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStal, F-59000 Lille, France · Univ. Toulouse, INRAE-MIAT, Toulouse, France
ECo-MoE: Embodiment-Conditioned Mixture of Experts Increases the Evolvability of Robots
May 22, 2026Yibin Wang, Muhan Li, Zihan Guo +1Robot Morphology-Control Co-Design Mixture of Experts
Northwestern University, Evanston, IL, USA.

Mar 10, 2026J/K move · Enter open · S save

MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Robotics

Neil Janwani, Ellen Novoseller, Vernon J. Lawhern, Maegan Tucker

Georgia Institute of Technology, Atlanta, GA USA · DEVCOM Army Research Laboratory, MD, USA

GPU-Accelerated RL Multi-Objective Reinforcement Learning Reinforcement Learning RL for Robotics

Open paper AskPDF arXiv

Multi-objective reinforcement learning (MORL) is a powerful tool to learn Pareto-optimal policy families across conflicting objectives. However, unlike traditional RL algorithms, existing MORL algorithms do not effectively leverage large-scale parallelization to concurrently simulate thousands of environments, thus facing vastly increased computation time. Ultimately, this has limited the application of MORL towards complex multi-objective robotics problems. To address these challenges, we present 1) MORLAX, a new GPU-native, fast MORL algorithm, and 2) MO-Playground, a pip-installable playground of GPU-accelerated multi-objective environments. Together, MORLAX and MO-Playground approximate Pareto sets within minutes, offering 26-271x speed-ups compared to legacy CPU-based approaches and up to 19x speed-ups over prior GPU-based approaches whilst learning superior Pareto front hypervolumes. We demonstrate MO-Playground's versatility by implementing a custom BRUCE humanoid robot environment and learning Pareto-optimal locomotion policies across 6 practical objectives in simulation, such as smoothness, efficiency and arm swinging. Our toolbox is available at https://dynamicmobility.github.io/moplayground/.

Abstract

Robot Morphology-Control Co-Design

Explore similar work

MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Robotics

Pareto Q-Learning with Reward Machines

ECo-MoE: Embodiment-Conditioned Mixture of Experts Increases the Evolvability of Robots