Evolution Strategies

Evolution strategies (ES) optimize by evaluating perturbed candidates and using their rewards or ranks to estimate which parameter changes are useful, without backpropagation. Methods such as CMA-ES and natural evolution strategies adapt mutation scales or covariance, while low-rank and cooperative variants reduce the cost of searching large spaces. They are used for black-box engineering and reinforcement learning, as well as language-model fine-tuning, prompt search, and agent training where rewards may be delayed or gradients unavailable. A central tension is balancing broad exploration and diverse solutions against evaluation cost, noisy feedback, and stability; matching gradient-based methods on task scores does not necessarily mean similar effects on other capabilities or knowledge.