Social Learning Strategies for Evolved Virtual Soft Robots
Authors: K. Ege de Bruin, Kyrre Glette, Kai Olav Ellefsen, Giorgia Nadizar, Eric Medvet
Organizations: Department of Informatics, University of Oslo, Oslo, Norway · Also with RITMO, University of Oslo. · University Toulouse Capitole, IRIT - CNRS UMR5505, Toulouse, France · University of Trieste, Trieste, Italy
Optimizing the body and brain of a robot is a coupled challenge: the morphology determines what control strategies are effective, while the control parameters influence how well the morphology performs. This joint optimization can be done through nested loops of evolutionary and learning processes, where the control parameters of each robot are learned independently. However, the control parameters learned by one robot may contain valuable information for others. Thus, we introduce a social learning approach in which robots can exploit optimized parameters from their peers to accelerate their own brain optimization. Within this framework, we systematically investigate how the selection of teachers, deciding which and how many robots to learn from, affects performance, experimenting with virtual soft robots in four tasks and environments. In particular, we study the effect of inheriting experience from morphologically similar robots due to the tightly coupled body and brain in robot optimization. Our results confirm the effectiveness of building on others' experience, as social learning clearly outperforms learning from scratch under equivalent computational budgets. In addition, while the optimal teacher selection strategy remains open, our findings suggest that incorporating knowledge from multiple teachers can yield more consistent and robust improvements.
Figures & tables
Figure 1. Overview of the SL approach. When a new VSR is generated through mutation, we need to optimize its brain. We use Bayesian optimization , which works by using initial samples ( i.e. , initial evaluated brains) to predict new candidates ( i.e. , new brains). The best sample is then used as the VSR brain to determine its fitness. With SL , initial candidates are obtained by inheriting samples from other VSRs in the population. This differs from a standard case, IL , where initial candidates are independent from other robots. Several strategies exist for selecting which samples to inherit; in this figure we show inheritance of one sample from the most similar robots in the previous generation (we call this “Similar-many”, see Section 4.2 ).
Figure 2. A snapshot of four episodes where a VSR performs the four tasks.
Figure 3. Distribution of the quality q⋆ of the best-performing VSR in the four tasks for the eight approaches. Stars above the boxes show significant differences: for example, a red star on top of a gray box indicates that IL is significantly different from Best-One.
Figure 4. The evolution of the population diversity, averaged over runs and environments. The line corresponds to the mean value, the shaded area to the first-third quartile range.
Figure 5. Body features for all the optimized VSRs .
Figure 7. A selection of the optimized VSR bodies, one per each approach (column) and task (row).
Figure 8. Distribution of the quality q⋆ of the best-performing VSR subjected to the re-learning of the brain with a large budget for BO .
Figure 9. Distribution of the quality q⋆ of the best-performing VSR optimized on a source task (row of plots) subjected to the re-learning of the brain with a large budget for BO and for another destination task (groups of boxes).
Figure 10. Performance over learning iterations for three brain learning approaches evaluated on eight fixed robot bodies. The vertical red dotted line corresponds to the learning budget of 50 in our main experiment.
Robot co-design via bi-level optimization couples within-lifetime controller learning for fitness evaluation with cross-generational morphological evolution. Prior work has established that well-adapted morphology facilitates faster control learning, a property termed morphological intelligence. Yet how control learning reciprocally shapes morphological evolution remains unexplored. This paper examines both directions for a holistic account of brain-body interplay. We first show that morphological contributions to control learning decouple into two orthogonal dimensions. We formalize the convergence speed as morphological intelligence and identify the performance ceiling as a complementary quantity termed true potential. A concise functional relation is then established to jointly characterize both quantities from individual learning curves, which, when aggregated at the population level, capture evolutionary profiles. Through extensive experiments on simulated voxel-based soft robots, we reveal that premature fitness evaluation systematically underestimates true potential and biases selection towards fast learners. This restricts design space exploration, compromising both optimization efficiency and morphological diversity. Notably, the widely recognized morphological Baldwin effect emerges as an artifact of this bias rather than a general evolutionary tendency. We therefore propose AdaControl, which monitors disproportionate selection for morphological intelligence during evolution and allocates minimally sufficient control learning for unbiased fitness evaluation. With AdaControl, a simple genetic algorithm rivals state-of-the-art generative-model-based co-design methods in discovering diverse high-performing designs while cutting computation by up to 80% versus exhaustive control.
Junru Song, Yang Yang, Yaqing Xu +5
Shanghai Jiao Tong University, Shanghai 200240, China · Intelligent Game and Decision Laboratory, Beijing 100048, China · Renmin University of China, Beijing 100872, China +1
The co-optimization of a robot's body and brain presents a coupled challenge: the morphology constrains which control strategies are effective, while the control determines how well the morphology performs. To address this, we combine morphology optimization as evolution with controller optimization as lifetime learning, utilizing Lamarckian inheritance to transfer learned controller parameters from parent to offspring. In dynamic environments, existing literature presents conflicting evidence: while traditional evolutionary theory often suggests Lamarckian inheritance lacks benefit, recent studies in evolutionary robotics indicate it can improve performance. We hypothesize that this is because previous works have not included all relevant variables with dynamic environments. In this work, we show that the benefit of Lamarckian inheritance depends on two variables: how conflicting the environmental changes are to robot control, and the predictability of those changes for the robotic agent. Using virtual soft robots and two different learning approaches, Bayesian optimization and reinforcement learning, we show that Lamarckian inheritance only underperforms Darwinian inheritance when the changes are both conflicting and unpredictable. We find that adding a sensor to detect environmental changes restores the benefits for Lamarckian inheritance in conflicting environments, by allowing robotic agents to predict the need for a different behavior, thereby generalizing their control.
K. Ege de Bruin, Kyrre Glette, Kai Olav Ellefsen
Department of Informatics, University of Oslo, Norway · RITMO, University of Oslo, Norway
In this paper, we introduce a model of evolution and learning in robots that co-optimizes a distribution of latent design vectors (genotypes) and a mixture of control experts (neural modules), which are gated by the latent coordinates of each decoded design (phenotype). This provides a scalable alternative to co-design algorithms that either train an individual policy for every robot, which is inefficient, or a monolithic universal controller for all robots, which results in overly conservative structures and behaviors. Our approach lies somewhere between these two extremes, preserving ancestral knowledge in a unified yet modular framework in which different body plans activate and deactivate different combinations of learned sensorimotor circuits for goal-directed behavior. This allows one part of the controller to be overhauled to better suit new species of designs as they emerge without disrupting the hard-earned knowledge contained within other expert modules. It also allows pretrained expert policies to be directly plugged into the mixture, which can steer evolution into otherwise unexplored areas of latent space containing desired morphological traits. We refer to this process as "evo by demo" and explore how it may be used to guide freeform evolution toward canonical structures defined by the pretrained model. Videos and code can be found at: https://eco-moe.github.io.