cs.NEOct 7, 2026
SaveThe Modular CMA-ES: A Framework for Modern Evolution Strategies
Abstract
Since their introduction, modern evolution strategies, such as the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), have become established as powerful methods for continuous black-box optimization. This success has led to a wide range of proposed modifications, each designed to improve performance or behavior in specific optimization scenarios. However, because these developments have largely been introduced and studied in isolation, their interactions remain comparatively underexplored. In this paper, we present the Modular CMA-ES (ModCMA), a configurable framework that integrates a wide range of mechanisms from modern evolution strategies within a single implementation. By decomposing CMA-ES into modules with interchangeable options for sampling, selection and recombination, step-size adaptation, matrix adaptation, and restarting, ModCMA enables systematic exploration of a large design space of modern evolution strategies and facilitates the construction, comparison, and automated configuration of new algorithm variants. We illustrate the benefits of this modular approach through two example studies. First, we compare several matrix-adaptation mechanisms in terms of their computational cost and optimization performance. Second, we use automated algorithm configuration to specialize ModCMA to individual benchmark problems and analyze the resulting configurations. Together, these examples demonstrate how the framework can be used both to study individual algorithmic design choices and to explore their combinations in a systematic and reproducible manner.
Figures & tables
| Module | Component | # | Available Options |
|---|---|---|---|
| Base Sampler | Sampling | 3 | Uniform , Sobol , Halton |
| Sample Transformation | Sampling | 6 | Gaussian , Uniform , Laplace , Logistic , Cauchy , Double Weibull |
| Mirrored Sampling | Sampling | 3 | None , Mirrored , Pairwise |
| Orthogonal Sampling | Sampling | 2 | Disabled , Enabled |
| Threshold Convergence | Sampling | 2 | Disabled , Enabled |
| Boundary Correction | Sampling | 7 | None , Resample , Mirror , COTN , Saturate , Uniform Resample , Toroidal |
Table 1: Implemented modules and configuration options available in ModCMA. The default option is provided in boldface. The component affected by the algorithm is shown alongside each module.
| Name | Type | LB | Default | UB | Description |
|---|---|---|---|---|---|
| Initial offspring population size. | |||||
| Initial number of selected parents. | |||||
| Initial global step size. | |||||
| Learning rate of the step-size evolution path. | |||||
| Learning rate of the covariance evolution path. | |||||
| Learning rate of the rank- covariance update. |
Table 2: Numerical parameters exposed by the ConfigSpace interface of ModCMA under the default module configuration. Here is the search-space dimension, is the initial offspring population size, is the number of selected parents, and is computed from the normalized positive recombination weights. The bounds for and are practical numerical limits rather than theoretical constraints; extreme values, particularly for , are generally better handled through appropriate scaling or normalization of the search space.
Figure 1: Running time in seconds vs. search space dimensionality . The left figure shows the total time to perform 1 000 generations of the algorithm, using the default population size of . The median time over 15 independent runs is shown. The right figure shows the average time per matrix update.
Figure 2: Expected Running Time (ERT) divided by the dimensionality for the benchmark functions from the BBOB suite, for six different values of . All matrix adaptation methods are compared with the official Python CMA-ES implementation Pycma . The ERT is computed over 100 independent runs, each with a maximum budget of , to achieve a target value of . Lower values are better.
Figure 3: Expected running time (ERT) of default and tuned ModCMA configurations on the 24 BBOB functions in dimension , compared to the best-performing solver per function from the bbob2009 portfolio. Lower values indicate better performance. The best (lowest) value per function is shown in bold. For some functions, configurations fail to reach the target within the evaluation budget, resulting in large or infinite ERT values.
Figure 4: Global contribution of module options based on SHAP values. Each bar shows the median SHAP value of a configuration option, aggregated over all evaluated configurations from the SMAC runs. Options are grouped by module (separated by vertical lines), with module names shown as annotations. Negative values indicate options that tend to improve performance, while positive values indicate options that tend to degrade performance, according to the surrogate model.
Figure 5: Per-function SHAP analysis for the Matrix Adaptation module. Rows correspond to the available matrix adaptation strategies, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Expected running time (ERT) of the default and tuned ModCMA configurations on the 24 BBOB functions in dimension , together with the individual algorithms from the bbob2009 benchmark portfolio. Lower values indicate better performance. The best (lowest) value per function is shown in bold. For some functions, algorithms fail to reach the target within the evaluation budget, resulting in large or infinite ERT values.
Figure 7: Per-function SHAP analysis for the Active Update module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 8: Per-function SHAP analysis for the Elitism module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 9: Per-function SHAP analysis for the Mirrored Sampling module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 10: Per-function SHAP analysis for the Orthogonal Sampling module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 11: Per-function SHAP analysis for the Repelling module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 12: Per-function SHAP analysis for the Restart Strategy module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 13: Per-function SHAP analysis for the Sample Transformation module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 14: Per-function SHAP analysis for the Base Sampler module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 15: Per-function SHAP analysis for the Sequential Selection module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 16: Per-function SHAP analysis for the Step-Size Adaptation module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 17: Per-function SHAP analysis for the Threshold Convergence module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.
Figure 18: Per-function SHAP analysis for the Recombination Weights module. Rows correspond to the available module options, and columns to the 24 BBOB functions. Each cell shows the median SHAP value over all evaluated configurations, indicating whether a given option tends to improve (negative values) or degrade (positive values) performance for that function.