Evolutionary Optimization

Follow this topic Ask about this topic

Evolutionary optimization studies how populations of candidate solutions can improve through selection, mutation, recombination, and other forms of variation, especially when gradients are unavailable or objectives are expensive. Common approaches include genetic algorithms, differential evolution, CMA-ES and evolution strategies, as well as multi-objective methods that preserve trade-offs and diversity. Researchers apply them to engineering design, robotics, scheduling, scientific discovery, and increasingly to evolving code, prompts, and language-model behavior; newer methods add learned surrogates, adaptive resource allocation, and large-language-model-generated operators. A central challenge is balancing exploration, solution quality, and evaluation cost, while ensuring improvements generalize beyond the benchmarks and search budgets used to find them.

Latest papers

295 papers33 papers in the last four weeks, up 50% on the four weeks before. 0.2% of all new papers.

Apr 3, 2026

Evolutionary Search for Automated Design of Uncertainty Quantification Methods

Uncertainty quantification (UQ) methods for large language models are predominantly designed by hand based on domain knowledge and heuristics, limiting their scalability and generality. We apply LLM-powered evolutionary search to automatically discover unsupervised UQ methods represented as Python programs. On the task of atomic claim verification, our evolved methods outperform strong manually-designed baselines, achieving up to 6.7% relative ROC-AUC improvement across 9 datasets while generalizing robustly out-of-distribution. Qualitative analysis reveals that different LLMs employ qualitatively distinct evolutionary strategies: Claude models consistently design high-feature-count linear estimators, while Gpt-oss-120B gravitates toward simpler and more interpretable positional weighting schemes. Surprisingly, only Sonnet 4.5 and Opus 4.5 reliably leverage increased method complexity to improve performance -- Opus 4.6 shows an unexpected regression relative to its predecessor. Overall, our results indicate that LLM-powered evolutionary search is a promising paradigm for automated, interpretable hallucination detector design.

Mikhail Seleznyov, Daniil Korbut, Viktor Moskvoretskii +3

AIRI · Skoltech · Amazon +2

Uncertainty Quantification LLM Uncertainty Estimation LLM Hallucination Detection Large Language Model-Guided Optimization Evolutionary Optimization

Apr 3, 2026

Accelerating Black-Box Bilevel Optimization with Rank-Based Upper-Level Value Function Approximation

Bilevel optimization is a field of significant theoretical and practical interest, yet solving such optimization problems remains challenging. Evolutionary methods have been employed to address these problems in the black-box setting; however, they incur high computational cost due to the nested nature of bilevel optimization. Although previous methods have attempted to reduce this cost through various heuristic techniques, such approaches limit versatility on challenging optimization landscapes, such as those with multimodality and significant interaction between upper- and lower-level decision variables. In this study, we propose an efficient framework that exploits the invariance of rank-based evolutionary algorithms to monotonic transformations, thereby reducing the computational burden of the lower-level optimization loop. Specifically, our method directly approximates the rankings of the upper-level value function, bypassing the need to run the lower-level optimizer until convergence for each upper-level iteration. We apply this framework to the setting where both levels are continuous, adopting CMA-ES as the optimizer. We demonstrate that our method achieves competitive performance on standard bilevel optimization benchmarks and can solve problems that are intractable with previously proposed methods, particularly those with multimodality and strong inter-variable interactions.

Marc Ong, Youhei Akimoto

University of Tsukuba Tsukuba, Ibaraki, JAPAN · University of Tsukuba & RIKEN AIP Tsukuba, Ibaraki, JAPAN

Bilevel Optimization Black-Box Optimization Evolutionary Optimization Surrogate-Assisted Optimization

Apr 1, 2026

CliffSearch: Structured Agentic Co-Evolution over Theory and Code for Scientific Algorithm Discovery

Scientific algorithm discovery is iterative: hypotheses are proposed, implemented, stress-tested, and revised. Current LLM-guided search systems accelerate proposal generation, but often under-represent scientific structure by optimizing code-only artifacts with weak correctness/originality gating. We present CliffSearch, an agentic evolutionary framework in which the core evolution operators (pair selection, crossover, mutation, and review) are implemented as LLM agents, and the loop is designed around three principles: (1) each node is a structured scientific artifact, instantiated in either theory+code or code_only mode, (2) reviewer judgments of correctness and originality are first-class selection gates alongside optimization of the benchmark metric of interest, and (3) mutation is split into exploration and correction pathways with distinct objectives. Exploration mutation imports ideas from adjacent scientific domains to increase novelty, while correction mutation performs targeted evidence-guided repair using reviewer signals over theory, code, benchmark results, and runtime errors. We illustrate the framework on three benchmark-grounded studies: transformer hyper-connection evolution, optimizer discovery on a fixed nanoGPT stack, and a smaller native-optimizer ablation. Across these settings, the same loop supports explicit metric direction, reproducible persistence, and reviewer-gated comparison of discoveries under controlled search conditions. The result is a discovery workflow that prioritizes scientific interpretability and correctness while optimizing task metrics under controlled novelty constraints, rather than maximizing candidate throughput alone. Full run artifacts, interactive visualizations, and exported best nodes for the reported studies are available at https://cliffsearch.ai .

Youssef Mroueh, Carlos Fonseca, Brian Belgodere +1

IBM Research

LLMs for Evolutionary Optimization Automated Algorithm Discovery AI Agents for Scientific Discovery Large Language Model-Guided Optimization Evolutionary Optimization

Apr 1, 2026

Cooperative Coevolution versus Monolithic Evolutionary Search for Semi-Supervised Tabular Classification

This paper studies semi-supervised tabular classification in the extreme low-label regime using lightweight base learners. The paper proposes a cooperative coevolutionary method (CC-SSL) that evolves (i) two feature-subset views and (ii) a pseudo-labeling policy, and compares it to a matched monolithic evolutionary baseline (EA-SSL) and three lightweight SSL baselines. Experiments on 25 OpenML datasets with labeled fractions {1%,5%,10%} evaluate test MacroF1 and accuracy, together with evolutionary and pseudo-label diagnostics. CC-SSL and EA-SSL achieve higher median test MacroF1 than the lightweight baselines, with the largest separations at 1% labeled data. Most CC-SSL vs. EA-SSL comparisons are statistical draws on final test performance. EA-SSL shows higher best-so-far fitness and higher diversity during search, while time-to-target is comparable and generations-to-target favors EA-SSL in several multiclass settings. Pseudo-label volume, ProbeDrop, and validation optimism show no significant differences between CC-SSL and EA-SSL under the shared protocol.

Jamal Toutouh

ITIS, Universidad de Málaga Spain

Semi-Supervised Learning Tabular ML Evolutionary Optimization Pseudo-Labeling

Mar 31, 2026

Associative Constructive Evolution: Enhancing Metaheuristics through Hebbian-Learned Generative Guidance

Metaheuristic algorithms such as Particle Swarm Optimization (PSO) and Evolutionary Algorithms (EA) excel at exploring solution spaces but lack mechanisms to accumulate and reuse procedural knowledge from successful search trajectories. This paper proposes Associative Constructive Evolution (ACE), a framework that enhances metaheuristics through learned generative guidance. ACE introduces a Generative Construction Automaton (GCA) -- a probabilistic model over operation sequences -- coupled with the base metaheuristic in a synergistic loop: the metaheuristic explores and provides trajectory samples, while the GCA consolidates successful patterns and guides future exploration. Three mechanisms realize this cooperation: Hebbian weight consolidation that strengthens associations between co-successful operations, guided sampling that biases search toward learned high-quality regions, and symbolic abstraction that extracts frequent patterns into reusable macro-operations. Experiments integrating ACE with EA and PSO on molecular design and maze navigation demonstrate consistent improvements. ACE-PSO achieves a 27.5% increase in success rate while reducing convergence time by 49.6%. In molecular design, ACE-EA improves fitness by 10.1% with 126 chemically interpretable macro-operations automatically discovered.

Shanxian Lin, Yuichi Nagata, Haichuan Yang

Graduate School of Technology, Industrial and Social Sciences, Tokushima University, Tokushima, Japan

Molecule Design Metaheuristic Optimization Evolutionary Optimization Particle Swarm Optimization

Mar 30, 2026

Evolutionary Discovery of Reinforcement Learning Algorithms via Large Language Models

Reinforcement learning algorithms are defined by their learning update rules, which are typically hand-designed and fixed. We present an evolutionary framework for discovering reinforcement learning algorithms by searching directly over executable update rules that implement complete training procedures. The approach builds on REvolve, an evolutionary system that uses large language models as generative variation operators, and extends it from reward-function discovery to algorithm discovery. To promote the emergence of nonstandard learning rules, the search excludes canonical mechanisms such as actor--critic structures, temporal-difference losses, and value bootstrapping. Because reinforcement learning algorithms are highly sensitive to internal scalar parameters, we introduce a post-evolution refinement stage in which a large language model proposes feasible hyperparameter ranges for each evolved update rule. Evaluated end-to-end by full training runs on multiple Gymnasium benchmarks, the discovered algorithms achieve competitive performance relative to established baselines, including SAC, PPO, DQN, and A2C.

Alkis Sygkounas, Amy Loutfi, Andreas Persson

Machine Perception and Interaction Lab, Örebro University, Sweden

Automated Algorithm Discovery Evolutionary RL LLM-Guided RL Large Language Model-Guided Optimization Evolutionary Optimization

Mar 30, 2026

Evolutionary Algorithms for Generating Graphs Matching Desired Laplacian Spectra

Graphs with diverse structural characteristics play a central role in modelling and optimization tasks. The ability to generate different types of graphs that exhibit shared properties is likewise essential for algorithm selection and configuration. However, constructing graphs that preserve high-level properties across a broad range of graph classes remains a challenging problem. We present a novel evolutionary approach to evolve graphs based on the Laplacian graph spectra descriptor. This descriptor can be used as part of a fitness function to evaluate graphs according to their desired high-level properties. Our evolutionary algorithm evolves graphs towards this descriptor in order to obtain graphs having properties that are consistent with it but are different from each other in terms of non-spectral graph metrics, such as path length, clustering coefficient and betweenness centrality. Our experimental results show that our approach is successful for different classes of graphs and a wide range of Laplacian graph spectra.

Hendrik Richter, Frank Neumann

HTWK Leipzig University of Applied Sciences, Faculty of Engineering, D–04251 Leipzig, Germany · Optimisation and Logistics, Adelaide University, Adelaide, SA 5005, Australia

Graph Generation Evolutionary Optimization

Mar 30, 2026

A Learning-Based Cooperative Coevolution Framework for Heterogeneous Large-Scale Global Optimization

Cooperative Coevolution (CC) effectively addresses Large-Scale Global Optimization (LSGO) via decomposition but struggles with the emerging class of Heterogeneous LSGO (H-LSGO) problems arising from real-world applications, where subproblems exhibit diverse dimensions and distinct landscapes. The prevailing CC paradigm, relying on a fixed low-dimensional optimizer, often fails to navigate this heterogeneity. To address this limitation, we propose the Learning-Based Heterogeneous Cooperative Coevolution Framework (LH-CC). By formulating the optimization process as a Markov Decision Process, LH-CC employs a meta-agent to adaptively select the most suitable optimizer for each subproblem. We also introduce a flexible benchmark suite to generate diverse H-LSGO problem instances. Extensive experiments on 3000-dimensional problems with complex coupling relationships demonstrate that LH-CC achieves superior solution quality and computational efficiency compared to state-of-the-art baselines. Furthermore, the framework exhibits robust generalization across varying problem instances, optimization horizons, and optimizers. Our findings reveal that dynamic optimizer selection is a pivotal strategy for solving complex H-LSGO problems.

Wenjie Qiu, Zixin Wang, Hongyu Fang +2

South China University of Technology, Guangzhou, Guangdong, China

Global Optimization Evolutionary Optimization Learning to Optimize

Mar 28, 2026

AutoMS: Multi-Agent Evolutionary Search for Cross-Physics Inverse Microstructure Design

Designing microstructures with coupled cross-physics objectives is a fundamental challenge where traditional topology optimization is often computationally prohibitive and deep generative models frequently suffer from physical hallucinations. We introduce AutoMS, a multi-agent neuro-symbolic framework that reformulates inverse design as an LLM-driven evolutionary search. AutoMS leverages LLMs as semantic navigators to decompose complex requirements and coordinate agent workflows, while a novel Simulation-Aware Evolutionary Search (SAES) mechanism handles low-level numerical optimization via local gradient approximation and directed parameter updates. This architecture achieves a state-of-the-art 83.8% success rate on 17 diverse cross-physics tasks, significantly outperforming both traditional evolutionary algorithms and existing agentic baselines. By decoupling open-ended semantic orchestration from simulation-grounded numerical search, AutoMS provides a robust pathway for navigating complex physical landscapes that remain intractable for standard generative or purely linguistic approaches.

Zhenyuan Zhao, Yu Xing, Tianyang Xue +3

Shandong University, Qingdao, China · The University of Hong Kong, Hong Kong, China

Inverse Design Large Language Model-Guided Optimization Evolutionary Optimization Materials Science Multi-Agent System Optimization

Mar 28, 2026

RDEx-MOP: Indicator-Guided Reconstructed Differential Evolution for Fixed-Budget Multiobjective Optimization

Multiobjective optimisation in the CEC 2025 MOP track is evaluated not only by final IGD values but also by how quickly an algorithm reaches the target region under a fixed evaluation budget. This report documents RDEx-MOP, the reconstructed differential evolution variant used in the IEEE CEC 2025 numerical optimisation competition (C06 special session) bound-constrained multiobjective track. RDEx-MOP integrates indicator-based environmental selection, a niche-maintained Pareto-candidate set, and complementary differential evolution operators for exploration and exploitation. We evaluate RDEx-MOP on the official CEC 2025 MOP benchmark using the released checkpoint traces and the median-target U-score framework. Experimental results show that RDEx-MOP achieves the highest total score and the best average rank among all released comparison algorithms, including the earlier RDEx baseline.

Sichen Tao, Yifei Yang, Ruihan Zhao +3

Department of Engineering, University of Toyama, Toyama-shi 930-8555, Japan · Cyberscience Center, Tohoku University, Sendai-shi 980-8578, Japan · Faculty of Science and Technology, Hirosaki University, Hirosaki-shi 036-8560, Japan +5

Multi-Objective Evolutionary Optimization Evolutionary Optimization Differential Evolution

Mar 27, 2026

SwarmCoDe: A Scalable Co-Design Framework for Heterogeneous Robot Swarms via Dynamic Speciation

Robot swarm co-design requires optimizing not only individual robot designs and behaviors, but also the heterogeneous swarm structure: how many robot types should exist, how they specialize and cooperate, and in what proportions they are deployed. However, these structural choices are often specified in advance rather than optimized. We present SwarmCoDe, a Cooperative Co-Evolutionary Algorithm (CCEA) that incorporates swarm decomposition into the optimization process without requiring the number of species to be specified in advance. SwarmCoDe combines dynamic speciation with evolved genetic tags, partner selectivity, and relative dominance to jointly optimize robot hardware, task-planning behavior, cooperative relationships, and deployed swarm composition. Across replicated simulated foraging experiments, SwarmCoDe discovers task-dependent swarm decompositions and adapts them when task requirements change, in contrast to fixed-K CCEAs which are sensitive to their pre-defined decomposition. We further show that fabrication budgets can change the preferred swarm composition and robot designs, and that SwarmCoDe can adapt its co-design under these resource constraints. Finally, SwarmCoDe co-designs 200-agent swarms using only 50 evolutionary individuals, demonstrating deployment at four times the evolutionary population size. Together, these results demonstrate that the heterogeneous structure of a robot swarm can be incorporated as a design variable and optimized alongside the robots themselves.

Andrew Wilhelm, Josie Hughes

Department of Mechanical Engineering, École Polytechnique Fédérale de Lausanne

Swarm Robotics Evolutionary Robotics Robot Morphology-Control Co-Design Evolutionary Optimization

Mar 26, 2026

EvoForest: A Novel Machine-Learning Paradigm via Open-Ended Evolution of Computational Graphs

Modern machine learning is still largely organized around a single recipe: choose a parameterized model family and optimize its weights. Although highly successful, this paradigm is too narrow for many structured prediction problems, where the main bottleneck is not parameter fitting but discovering what should be computed from the data. Success often depends on identifying the right transformations, statistics, invariances, interaction structures, temporal summaries, gates, or nonlinear compositions, especially when objectives are non-differentiable, evaluation is cross-validation-based, interpretability matters, or continual adaptation is required. We present EvoForest, a hybrid neuro-symbolic system for end-to-end open-ended evolution of computation. Rather than merely generating features, EvoForest jointly evolves reusable computational structure, callable function families, and trainable low-dimensional continuous components inside a shared directed acyclic graph. Intermediate nodes store alternative implementations, callable nodes encode reusable transformation families such as projections, gates, and activations, output nodes define candidate predictive computations, and persistent global parameters can be refined by gradient descent. For each graph configuration, EvoForest evaluates the discovered computation and uses a lightweight Ridge-based readout to score the resulting representation against a non-differentiable cross-validation target. The evaluator also produces structured feedback that guides future LLM-driven mutations. In the 2025 ADIA Lab Structural Break Challenge, EvoForest reached 94.13% ROC-AUC after 600 evolution steps, exceeding the publicly reported winning score of 90.14% under the same evaluation protocol.

Kamer Ali Yuksel, Hassan Sawaf

aiXplain, Inc., San Jose, CA, USA

Neuro-Symbolic AI Neuroevolution Evolutionary Optimization Genetic Programming

Mar 25, 2026

AVO: Agentic Variation Operators for Autonomous Evolutionary Search

Agentic Variation Operators (AVO) are a new family of evolutionary variation operators that replace the fixed mutation, crossover, and hand-designed heuristics of classical evolutionary search with autonomous coding agents. Rather than confining a language model to candidate generation within a prescribed pipeline, AVO instantiates variation as a self-directed agent loop that can consult the current lineage, a domain-specific knowledge base, and execution feedback to propose, repair, critique, and verify implementation edits. We evaluate AVO on attention, among the most aggressively optimized kernel targets in AI, on NVIDIA Blackwell (B200) GPUs. Over 7 days of continuous autonomous evolution on multi-head attention, AVO discovers kernels that outperform cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations. The discovered optimizations transfer readily to grouped-query attention, requiring only 30 minutes of additional autonomous adaptation and yielding gains of up to 7.0% over cuDNN and 9.3% over FlashAttention-4. Together, these results show that agentic variation operators move beyond prior LLM-in-the-loop evolutionary pipelines by elevating the agent from candidate generator to variation operator, and can discover performance-critical micro-architectural optimizations that produce kernels surpassing state-of-the-art expert-engineered attention implementations on today's most advanced GPU hardware.

Terry Chen, Zhifan Ye, Bing Xu +20

NVIDIA

Evolutionary Optimization GPU Kernel Optimization Large Language Model-Guided Optimization Efficient Attention

Mar 25, 2026

A Longitudinal Analysis of the CEC Single-Objective Competitions (2010-2024) and Implications for Variational Quantum Optimization

This paper provides a historical analysis of the IEEE CEC Single Objective Optimization competition results (2010-2024). We analyze how benchmark functions shaped winning algorithms, identifying the 2014 introduction of dense rotation matrices as a key performance filter. This design choice introduced parameter non-separability, reduced effectiveness of coordinate-dependent methods (PSO, GA), and established the dominance of Differential Evolution variants capable of preserving the rotational invariance of their difference vectors, specifically L-SHADE. Post-2020 analysis reveals a shift towards high complexity hybrid optimizers that combine different mechanisms (e.g., Eigenvector Crossover, Societal Sharing, Reinforcement Learning) to maximize ranking stability. We conclude by identifying structural similarities between these modern benchmarks and Variational Quantum Algorithm landscapes, suggesting that evolved CEC solvers possess the specific adaptive capabilities required for quantum control.

Vojtěch Novák, Tomáš Bezděk, Ivan Zelinka +2

Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VSB - Technical University of Ostrava, Ostrava, Czech Republic · IT4Innovations National Supercomputing Center, VSB - Technical University of Ostrava, 708 00 Ostrava, Czech Republic · Department of Informatics and Statistics, Marine Research Institute, Klaipeda University, Lithuania +3

Evolutionary Optimization Differential Evolution

Mar 21, 2026

Elite Lanes: Evolutionary Generation of Realistic Small-Scale Road Networks

We present a comparative study of methods for generating realistic, constrained small- to medium-scale road networks with built-in redundancy. In this research, we evaluate the proposed Evolutionary Algorithm (EA) with connectivity and redundancy constraints against the Wave Function Collapse (WFC) method - commonly used in procedural terrain generation for games - and swarm algorithms: Particle Swarm (PSO) and Gray Wolf (GWO). Our focus is on producing realistic, redundant road networks suitable for vision, localization and navigation problems. We evaluate metrics: connectivity, cycles, intersections, dead ends, graph cut-edges while enforcing physical plausibility. We propose an EA and its extended version with elitism via MAP-Elites method. We detail the implementation, constraints, metrics and provide both visual and quantitative comparisons with baselines. Results highlight how fitness function design choices affect the structural characteristics of generated networks and highlight the impact of specific constraints in practical applications. Our contribution is a method for creating realistic synthetic datasets from sparse tile definitions derived from real-world data. We demonstrate a practical application by generating realistic maps using a laboratory-collected tileset from a Duckietown city model. Our approach performs coherent geometric transformations on metadata, in this work exemplified by semantic segmentation masks of the generated road networks.

Artur Morys-Magiera, Marek Długosz, Paweł Skruch

AGH University of Krakow, Poland

Graph Generation Synthetic Data Generation for Robotics Evolutionary Optimization

Mar 21, 2026

Decoupling Numerical and Structural Parameters: An Empirical Study on Adaptive Genetic Algorithms via Deep Reinforcement Learning for the Large-Scale TSP

Proper parameter configuration is a prerequisite for the success of Evolutionary Algorithms (EAs). While various adaptive strategies have been proposed, it remains an open question whether all control dimensions contribute equally to algorithmic scalability. To investigate this, we categorize control variables into numerical parameters (e.g., crossover and mutation rates) and structural parameters (e.g., population size and operator switching), hypothesizing that they play distinct roles. This paper presents an empirical study utilizing a dual-level Deep Reinforcement Learning (DRL) framework to decouple and analyze the impact of these two dimensions on the Traveling Salesman Problem (TSP). We employ a Recurrent PPO agent to dynamically regulate these parameters, treating the DRL model as a probe to reveal evolutionary dynamics. Experimental results confirm the effectiveness of this approach: the learned policies outperform static baselines, reducing the optimality gap by approximately 45% on the largest tested instance (rl5915). Building on this validated framework, our ablation analysis reveals a fundamental insight: while numerical tuning offers local refinement, structural plasticity is the decisive factor in preventing stagnation and facilitating escape from local optima. These findings suggest that future automated algorithm design should prioritize dynamic structural reconfiguration over fine-grained probability adjustment. To facilitate reproducibility, the source code is available at https://github.com/StarDream1314/DRLGA-TSP

Hongyu Wang, Yuhan Jing, Yibing Shi +3

School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, China · School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, China · Frontier Institute of Science and Technology, Xi’an Jiaotong University, Xi’an, China

Evolutionary Optimization Genetic Algorithms RL for Combinatorial Optimization Traveling Salesperson Problem

Mar 20, 2026

FormalEvolve: Neuro-Symbolic Evolutionary Search for Diverse Autoformalization

Autoformalization aims to produce formal statements that compile and faithfully preserve the intended meaning of informal mathematics. Yet standard single-output evaluation collapses this many-to-many structure into a single prediction. For downstream proving, this granularity is too coarse: a formal statement is not merely a faithful translation endpoint, but also a prover-facing interface whose structure can alter proof search under a fixed budget. We therefore recast autoformalization as budgeted test-time search: FormalEvolve maintains a compilation-feasible archive for reuse and returns a deduplicated, semantically accepted repertoire for evaluation and downstream proving. It expands the archive with LLM-driven mutation, crossover, bounded patch repair, and symbolic abstract syntax tree (AST) rewrites for structural diversity. Under a generator-call budget of T=100 with a fixed LLM semantic judge, FormalEvolve reaches SH@100 of 58.0% on CombiBench and 84.9% on ProofNet, improving over all no-archive controls while reducing the cross-problem concentration of semantic successes. Under a fixed B=64 prover budget, these repertoires improve theorem-complete proving over the matched no-archive control. Additional stronger-base statement-generation experiments show that archive-search gains persist with stronger seed and repair models.

Haijian Lu, Wei Wang, Jing Liu

School of Artificial Intelligence, Xidian University, Xi’an, China · Beijing Institute for General Artificial Intelligence, Beijing, China

Autoformalization Inference-Time Search Evolutionary Optimization

Mar 20, 2026

Helix: A Dual-Helix Co-Evolutionary Multi-Agent System for Prompt Optimization and Question Reformulation

Automated prompt optimization (APO) aims to improve large language model performance by refining prompt instructions. However, existing methods are largely constrained by fixed prompt templates, limited search spaces, or single-sided optimization that treats user questions as immutable inputs. In practice, question formulation and prompt design are inherently interdependent: clearer question structures facilitate focused reasoning and task understanding, while effective prompts reveal better ways to organize and restate queries. Ignoring this coupling fundamentally limits the effectiveness and adaptability of current APO approaches. We propose a unified multi-agent system (Helix) that jointly optimizes question reformulation and prompt instructions through a structured three-stage co-evolutionary framework. Helix integrates (1) planner-guided decomposition that breaks optimization into coupled question-prompt objectives, (2) dual-track co-evolution where specialized agents iteratively refine and critique each other to produce complementary improvements, and (3) strategy-driven question generation that instantiates high-quality reformulations for robust inference. Extensive experiments on 12 benchmarks against 6 strong baselines demonstrate the effectiveness of Helix, achieving up to 3.95% performance improvements across tasks with favorable optimization efficiency.

Kewen Zhu, Liping Yi, Zhiming Zhao +2

College of Intelligence and Computing, Tianjin University, Tianjin, China

Automatic Prompt Optimization Query Reformulation Multi-Agent LLM Systems Evolutionary Optimization

Mar 19, 2026

POET: Power-Oriented Evolutionary Tuning for LLM-Based RTL PPA Optimization

Applying large language models (LLMs) to RTL code optimization for improved power, performance, and area (PPA) faces two key challenges: ensuring functional correctness of optimized designs despite LLM hallucination, and systematically prioritizing power reduction within the multi-objective PPA trade-off space. We propose POET (Power-Oriented Evolutionary Tuning), a framework that addresses both challenges. For functional correctness, POET introduces a differential-testing-based testbench generation pipeline that treats the original design as a functional oracle, using deterministic simulation to produce golden references and eliminating LLM hallucination from the verification process. For PPA optimization, POET employs an LLM-driven evolutionary mechanism with non-dominated sorting, power-first intra-level ranking, and proportional survivor selection to steer the search toward the low-power region of the Pareto front without manual weight tuning. Evaluated on the RTL-OPT benchmark across 40 diverse RTL designs, POET achieves 100% functional correctness, the best power on all 40 designs, and competitive area and delay improvements.

Heng Ping, Peiyu Zhang, Zhenkun Wang +5

University of Southern California, United States

Large Language Model-Guided Optimization Evolutionary Optimization

Mar 16, 2026

INSTRUMENTAL: Automatic Synthesizer Parameter Recovery from Audio via Evolutionary Optimization

Existing audio-to-MIDI tools extract notes but discard the timbral characteristics that define an instrument's identity. We present Instrumental, a system that recovers continuous synthesizer parameters from audio by coupling a differentiable 28-parameter subtractive synthesizer with CMA-ES, a derivative-free evolutionary optimizer. We optimize a composite perceptual loss combining mel-scaled STFT, spectral centroid, and MFCC divergence, achieving a matching loss of 2.09 on real recorded audio. We systematically evaluate eight hypotheses for improving convergence and find that only parametric EQ boosting yields meaningful improvement. Our results show that CMA-ES outperforms gradient descent on this non-convex landscape, that more parameters do not monotonically improve matching, and that spectral analysis initialization accelerates convergence over random starts.

Philipp Bogdan

Parameter Estimation Evolutionary Optimization

Mar 15, 2026

Data Darwinism Part II: DataEvolve -- AI can Autonomously Evolve Pretraining Data Curation

Data Darwinism (Part I) established a ten-level hierarchy for data processing, showing that stronger processing can unlock greater data value. However, that work relied on manually designed strategies for a single category. Modern pretraining corpora comprise hundreds of heterogeneous categories spanning domains and content types, each demanding specialized treatment. At this scale, manual strategy design becomes prohibitive. This raises a key question: can strategies evolve in an automated way? We introduce DataEvolve, a framework that enables strategies to evolve through iterative optimization rather than manual design. For each data category, DataEvolve operates in a closed evolutionary loop: it identifies quality issues, generates candidate strategies, executes them on sampled data, evaluates results, and refines approaches across generations. The process accumulates knowledge through an experience pool of discovered issues and a strategy pool tracking performance across iterations. Applied to 8 categories spanning 672B tokens from Nemotron-CC, DataEvolve produces Darwin-CC, a 504B-token dataset with strategies evolved through 30 iterations per category. Training 3B models on 500B tokens, Darwin-CC outperforms raw data (+3.96 points) and achieves a 44.13 average score across 18 benchmarks, surpassing DCLM, Ultra-FineWeb, and FineWeb-Edu, with strong gains on knowledge-intensive tasks such as MMLU. Analysis shows evolved strategies converge on cleaning-focused approaches: targeted noise removal and format normalization with domain-aware preservation, echoing the L4 (Generative Refinement) principles from Part I. Ablation studies confirm iterative evolution is essential: optimized strategies outperform suboptimal ones by 2.93 points, establishing evolutionary strategy design as feasible and necessary for pretraining-scale data curation.

Tiantian Mi, Dongming Shan, Zhen Huang +6

SII · FDU · GAIR +2

Training Data Curation Language Model Pretraining Evolutionary Optimization

Mar 12, 2026

KernelFoundry: Hardware-aware evolutionary GPU kernel optimization

GPU kernel optimization challenges LLMs beyond standard coding tasks, as it requires an understanding of hardware architecture, parallel computing optimization strategies, and profiling outputs. However, most existing approaches leveraging LLMs for kernel generation apply standard prompting and feedback loops, considering hardware only through profiling feedback. We introduce KernelFoundry, an evolutionary framework that efficiently explores the space of GPU kernels through (1) MAP-Elites quality diversity search with kernel-specific behavioral dimensions to sustain exploration; (2) meta-prompt evolution that co-evolves prompts with kernels to uncover task-specific optimization strategies, and (3) a template-based parameter optimization approach to tune kernels to inputs and hardware. We evaluate this framework on Kernel-Bench, robust-kbench and custom tasks, generating SYCL kernels as a cross-platform GPU programming paradigm, and CUDA kernels for comparison to prior work. Our approach consistently outperforms the baseline methods and achieves an average speedup of 2.3 on KernelBench for SYCL. Moreover, KernelFoundry is implemented as a distributed framework with remote access to diverse hardware, allowing quick benchmarking and featuring a flexible user input layer to support kernel generation for a wide range of real use cases beyond benchmarking.

Nina Wiedemann, Quentin Leboutet, Michael Paulitsch +2

Intel Corporation

GPU Kernel Optimization GPU Acceleration Evolutionary Optimization Large Language Model-Guided Optimization LLM-Based Program Synthesis GPU Kernel Generation+1

Mar 9, 2026

Advancing Automated Algorithm Design via Evolutionary Stagewise Design with LLMs

With the rapid advancement of human science and technology, problems in industrial scenarios are becoming increasingly challenging, bringing significant challenges to traditional algorithm design. Automated algorithm design with LLMs emerges as a promising solution, but the currently adopted black-box modeling deprives LLMs of any awareness of the intrinsic mechanism of the target problem, leading to hallucinated designs. In this paper, we introduce Evolutionary Stagewise Algorithm Design (EvoStage), a novel evolutionary paradigm that bridges the gap between the rigorous demands of industrial-scale algorithm design and the LLM-based algorithm design methods. Drawing inspiration from CoT, EvoStage decomposes the algorithm design process into sequential, manageable stages and integrates real-time intermediate feedback to iteratively refine algorithm design directions. To further reduce the algorithm design space and avoid falling into local optima, we introduce a multi-agent system and a "global-local perspective" mechanism. We apply EvoStage to the design of two types of common optimizers: designing parameter configuration schedules of the Adam optimizer for chip placement, and designing acquisition functions of Bayesian optimization for black-box optimization. Experimental results across open-source benchmarks demonstrate that EvoStage outperforms human-expert designs and existing LLM-based methods within only a couple of evolution steps, even achieving the historically state-of-the-art half-perimeter wire-length results on every tested chip case. Furthermore, when deployed on a commercial-grade 3D chip placement tool, EvoStage significantly surpasses the original performance metrics, achieving record-breaking efficiency. We hope EvoStage can significantly advance automated algorithm design in the real world, helping elevate human productivity.

Chen Lu, Ke Xue, Chengrui Gao +5

National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · Huawei Noah’s Ark Lab, China

Automated Algorithm Discovery Automated Heuristic Design Evolutionary Optimization Electronic Design Automation Bayesian Optimization

Mar 8, 2026

A Primer on Evolutionary Optimization Frameworks for Near-Field Multi-Source Localization

This paper introduces evolutionary optimization as a grid-free training-free continuous-domain search mechanism for near-field multi-source localization, addressing the major limitations of grid-based subspace methods such as MUSIC and data-driven deep learning approaches. To this end, we develop two complementary evolutionary localization frameworks that operate directly on the continuous spherical-wave signal model and support arbitrary array geometries without requiring labeled data, discretized angle-range grids, or architectural constraints. The first framework, termed NEar-field MultimOdal DE (NEMO-DE) associates each individual in the evolutionary population to a single source and optimizes a residual least-squares objective in a sequential manner, updating the data residual and enforcing spatial separation to estimate multiple source locations. To overcome the limitation of NEMO-DE under large power imbalances among the sources, we propose the second framework, named NEar-field Eigen-subspace Fitting DE (NEEF-DE), which jointly encodes all source locations and minimizes a subspace-fitting criterion that aligns a model-based array response subspace with the received signal subspace. The proposed formulations are not intrinsically tied to a specific optimizer; however, this work adopts differential evolution (DE) as a representative evolutionary search strategy because of its simple implementation, small number of control parameters, and strong empirical performance in continuous nonconvex optimization problems. Numerical results show that the proposed frameworks provide competitive accuracy compared with MUSIC-type baselines while avoiding pre-defined grid construction and labeled training data. This work establishes evolutionary computation as a powerful and flexible paradigm for model-based near-field localization, paving the way for future innovations in this domain.

Seyed Jalaleddin Mousavirad, Parisa Ramezani, Mattias O'Nils +1

Department of Computer and Electrical Engineering, Mid Sweden University, Sundsvall, Sweden · the school of electrical and data engineering, University of Technology Sydney, Sydney, Australia · the Department of Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden

Wireless Localization Direction-of-Arrival Estimation Evolutionary Optimization

Mar 8, 2026

CDEoH: Category-Driven Automatic Algorithm Design With Large Language Models

With the rapid advancement of large language models (LLMs), LLM-based heuristic search methods have demonstrated strong capabilities in automated algorithm generation. However, their evolutionary processes often suffer from instability and premature convergence. Existing approaches mainly address this issue through prompt engineering or by jointly evolving thought and code, while largely overlooking the critical role of algorithmic category diversity in maintaining evolutionary stability. To this end, we propose Category Driven Automatic Algorithm Design with Large Language Models (CDEoH), which explicitly models algorithm categories and jointly balances performance and category diversity in population management, enabling parallel exploration across multiple algorithmic paradigms. Extensive experiments on representative combinatorial optimization problems across multiple scales demonstrate that CDEoH effectively mitigates convergence toward a single evolutionary direction, significantly enhancing evolutionary stability and achieving consistently superior average performance across tasks and scales.

Yu-Nian Wang, Shen-Huan Lyu, Ning Chen +3

Key Laboratory of Water Big Data Technology of Ministry of Water Resources, College of Computer Science and Software Engineering, Hohai University, Nanjing, China · Department of Computer Science, City University of Hong Kong, Hong Kong, China · State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China

Automated Heuristic Design Automated Algorithm Discovery Large Language Model-Guided Optimization Evolutionary Optimization Combinatorial Optimization

Mar 7, 2026

Large Language Model-Driven Full-Component Evolution of Adaptive Large Neighborhood Search

Adaptive Large Neighborhood Search (ALNS) is a prominent metaheuristic and a widely adopted approach for production and logistics optimization. However, it has long relied on hand-crafted components built on expert experience, which makes development slow and costly to adapt to new problems. This paper proposes a closed-loop, large-language-model-driven evolutionary framework that decouples ALNS and automatically rebuilds all of its components. We break ALNS into seven key modules: destroy, repair, operator selection, weight update, initial solution construction, acceptance rule, and destroy-rate control, and evolve each module through a dedicated task. By incorporating the Multi-dimensional Archive of Phenotypic Elites mechanism, the framework maintains a multi-dimensional elite archive to simultaneously drive the evolution of solution quality and strategic diversity. In addition, we design multiple mechanisms, including parallel and sequential multi-module evolution as well as single-expert-driven and multi-expert-driven evolution, to systematically evaluate the impact of different evolutionary paradigms on algorithm generation performance. Evaluations on Traveling Salesman Problem and Capacitated Vehicle Routing Problem benchmarks demonstrate that evolved algorithms consistently outperform optimized classic ALNS baselines under both fixed-iteration and fixed-time limits. The framework also shows a degree of generalizability and cross-problem transferability. Code analysis also uncovers several counterintuitive yet meaningful design patterns that emerged naturally during evolution, offering practical and theoretical insights for future ALNS design. Finally, comparisons across multiple language models highlight clear differences in their ability to support evolutionary algorithm design, helping guide model selection for real-world engineering use.

Shaohua Yu, Tianyu Chen, Linyan Liu +1

School of Intelligent Science and Technology, Nanjing University of Science and Technology, Nanjing 210094, China · School of Mechanical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China · Université Paris-Saclay, CentraleSupélec, Laboratoire Génie Industriel, 91190, Gif-sur-Yvette, France +1

Automated Heuristic Design Large Neighborhood Search Large Language Model-Guided Optimization Evolutionary Optimization

Mar 5, 2026

EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selection

LLM-based agents depend on effective tool-use policies to solve complex tasks, yet optimizing these policies remains challenging due to delayed supervision and the difficulty of credit assignment in long-horizon trajectories. Existing optimization approaches tend to be either monolithic, which are prone to entangling behaviors, or single-aspect, which ignore cross-module error propagation. To address these limitations, we propose EvoTool, a self-evolving framework that optimizes a modular tool-use policy via a gradient-free evolutionary paradigm. EvoTool decomposes agent's tool-use policy into four modules, including Planner, Selector, Caller, and Synthesizer, and iteratively improves them in a self-improving loop through three novel mechanisms. Trajectory-Grounded Blame Attribution uses diagnostic traces to localize failures to a specific module. Feedback-Guided Targeted Mutation then edits only that module via natural-language critique. Diversity-Aware Population Selection preserves complementary candidates to ensure solution diversity. Across four benchmarks, EvoTool outperforms strong baselines by over 5 points on both GPT-4.1 and Qwen3-8B, while achieving superior efficiency and transferability. The code will be released once paper is accepted.

Shuo Yang, Soyeon Caren Han, Xueqi Ma +3

School of Computing and Information Systems, The University of Melbourne

Policy Optimization LLM Agent Self-Improvement Self-Evolving Agents Evolutionary Optimization RL for Tool Use

Mar 5, 2026

ReVEL: Multi-Turn Reflective LLM-Guided Heuristic Evolution via Structured Performance Feedback

Designing effective heuristics for NP-hard combinatorial optimization problems remains challenging and often requires substantial domain expertise. Recent LLM-guided evolutionary methods have shown promise for automated heuristic generation, but most existing approaches refine heuristics independently or through limited pairwise feedback. We propose ReVEL: Multi-Turn Reflective LLM-Guided Heuristic Evolution via Structured Performance Feedback, a framework for group-wise multi-turn heuristic refinement. ReVEL organizes heuristics into behavior-aware reflective groups, including similarity-driven groups for localized refinement and diversity-driven groups for exploratory search. Within each group, the LLM performs iterative multi-turn refinement using accumulated performance feedback, enabling related heuristics to be jointly analyzed and progressively improved across evolutionary iterations. Experiments on standard combinatorial optimization benchmarks show that ReVEL generally improves optimization performance over existing LLM-guided evolutionary baselines across multiple settings and LLM backbones. Additional analyses suggest that behavior-aware grouping contributes to more consistent refinement trajectories during iterative heuristic evolution.

Cuong Van Duc, Minh Nguyen Dinh Tuan, Tam Vu Duc +4

VinMotion · Hanoi University of Science and Technology · VinUniversity +2

Automated Heuristic Design Large Language Model-Guided Optimization Evolutionary Optimization Combinatorial Optimization LLM Self-Refinement

Mar 4, 2026

Empirical Evaluation of No Free Lunch Violations in Permutation-Based Optimization

The No Free Lunch (NFL) theorem guarantees equal average performance only under uniform sampling of a function space closed under permutation (c.u.p.). We ask when this averaging ceases to reflect what benchmarking actually reports. We study an iterative-search setting with sampling without replacement, where algorithms differ only in evaluation order. Binary objectives allow exhaustive evaluation in the fully enumerable case, and efficiency is defined by the first time the global minimum is reached. We then construct two additional benchmarks by algebraically recombining the same baseline functions through sums and differences. Function-algorithm relations are examined via correlation structure, hierarchical clustering, delta heatmaps, and PCA. A one-way ANOVA with Tukey contrasts confirms that algebraic reformulations induce statistically meaningful shifts in performance patterns. The uniformly sampled baseline remains consistent with the global NFL symmetry. In contrast, the algebraically modified benchmarks yield stable re-rankings and coherent clusters of functions and sampling policies. Composite objectives can also exhibit non-additive search effort despite being built from simpler components. Monte Carlo experiments indicate that order effects persist in larger spaces and depend on function class. Taken together, the results show how objective reformulation and benchmark design can generate structured local departures from NFL intuition. They motivate algorithm choice that is aware of both the problem class and the objective representation. This message applies to evolutionary computation as well as to statistical procedures based on relabeling, resampling, and permutation tests.

Grzegorz Sroka

Department of Nonlinear Analysis, Rzeszów University of Technology, Powstańców Warszawy 12, 35-959 Rzeszów, Poland

Black-Box Optimization Benchmark Design Evolutionary Optimization

Feb 27, 2026

All Mutation Rates $c/n$ for the $(1+1)$ Evolutionary Algorithm

For every real number

c \geq 1

and for all

\varepsilon > 0

, there is a fitness function

f : \{0,1\}^n \to \mathbb{R}

for which the optimal mutation rate for the

(1+1)

evolutionary algorithm on

f

, denoted

p_n

, satisfies

p_n \approx c/n

in that

|np_n - c| < \varepsilon

. In other words, the set of all

c \geq 1

for which the mutation rate

c/n

is optimal for the

(1+1)

EA is dense in the interval

[1, \infty)

. To show this, a fitness function is introduced which is called HillPathJump.

Andrew James Kelley

Evolutionary Optimization Runtime Analysis of Evolutionary Algorithms