AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization

Mert Cemri; Shubham Agrawal; Akshat Gupta; Shu Liu; Audrey Cheng; Qiuyang Mang; Ashwin Naren; Lutfi Eren Erdogan

cs.NEFeb 23, 2026

AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization

Authors: Mert Cemri, Shubham Agrawal, Akshat Gupta, Shu Liu, Audrey Cheng, Qiuyang Mang, Ashwin Naren, Lutfi Eren Erdogan,

Organizations: University of California, Berkeley · Bespoke Labs

Abstract

The paradigm of automated program generation is shifting from one-shot generation to inference-time search, where Large Language Models (LLMs) function as semantic mutation operators within evolutionary loops. While effective, these systems are currently governed by static schedules that fail to account for the non-stationary dynamics of the search process. This rigidity results in substantial computational waste, as resources are indiscriminately allocated to stagnating populations while promising frontiers remain under-exploited. We introduce AdaEvolve, a framework that reformulates LLM-driven evolution as a hierarchical adaptive optimization problem. AdaEvolve uses an "accumulated improvement signal" to unify decisions across three levels: Local Adaptation, which dynamically modulates the exploration intensity within a population of solution candidates; Global Adaptation, which routes the global resource budget via bandit-based scheduling across different solution candidate populations; and Meta-Guidance which generates novel solution tactics based on the previously generated solutions and their corresponding improvements when the progress stalls. We demonstrate that AdaEvolve consistently outperforms the open-sourced baselines across 185 different open-ended optimization problems including combinatorial, systems optimization and algorithm design problems.

Figures & tables

Figure 1: AdaEvolve overview. Left: Standard LLM-guided search relies on fixed optimization policies, with static schedules, uniform resource allocation, and rigid prompts. Center: AdaEvolve introduces hierarchical adaptivity by dynamically modulating exploration intensity, reallocating compute across populations of programs (islands), and generating meta-level guidance from a unified improvement signal. Right: AdaEvolve overcomes stagnation, reaching to a best-known score of 2.636 on the Circle Packing (N = 26), surpassing the Human SOTA (2.634) and AlphaEvolve (2.635).

Figure 2: Comparison of the evolutionary algorithms on Circle Packing (Square

n=26

) and Heilbronn Triangles (

n=11

) problems using GPT-5 backbone for all of them.

n

is a parameter of the optimization problems we explain in Table 7 .

Figure 3: GPT-5 best configurations comparisons for the Circle Packing experiment.

Figure 4: Score distribution comparison across methods on the Frontier-CS benchmark.

Ablation Setting	Circle Packing	Signal Processing
AdaEvolve	$\mathbf{2.6294\pm 0.003}$	$\mathbf{0.7178\pm 0.019}$
w/o Local Adaptation	$2.5906\pm 0.048$	$0.6807\pm 0.021$
w/o Adaptive Island Selection	$2.6180\pm 0.005$	$0.619\pm 0.0541$
w/o Meta-Guidance	$2.5213\pm 0.028$	$0.5476\pm 0.011$
Fixed 2 Islands	$2.6187\pm 0.007$	$0.5512\pm 0.024$
Fixed 5 Islands	$2.5891\pm 0.018$	$0.6085\pm 0.081$

Table 4: Ablation results on circle packing and signal processing tasks. We report the mean and standard deviation of 3 runs.

Panel of Figure 5 — Figure 5: AdaEvolve adapts search behavior across tasks. (Left) Signal Processing: exploration transitions to refinement as improvement signals accumulate. (Right) Circle Packing: the evolved strategy breaks stagnation and drives near-optimal layouts.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Use case	Meta-Guidance Solution Tactics	Approach type
Math / equation systems	Solve nonlinear systems using a trust-region root finder	scipy.optimize.root
	Exploit structure by directly solving linear systems	scipy.linalg.solve
	Isolate scalar roots with bracketed search	scipy.optimize.brentq
	Iterate toward equilibrium with damped fixed-point updates	Fixed-point iteration
	Generate symbolic expressions before numerical evaluation	sympy.lambdify + solver
Continuous optimization	Perform constrained minimization under geometric constraints	scipy.optimize.minimize (SLSQP)

Appendix

Table 5: Example tactics produced by the tactic generator across domains. Each tactic pairs a high-level solution strategy with a representative computational approach.

Problem	Objective (exact task definition)
Circle Packing (Square) $N{=}26$	Pack $N$ disjoint circles inside a unit square so as to maximize the sum of their radii .
Circle Packing (Rectangle) $N{=}21$	Pack $N$ disjoint circles inside a rectangle of perimeter $4$ so as to maximize the sum of their radii .
Heilbronn Problem (Triangle) $N{=}11$	Place $N$ points on or inside a triangle of unit area so that the minimum area of any triangle formed by the points is maximized .
Heilbronn Problem (Convex Region) $N{=}13$	Place $N$ points on or inside a convex region of unit area so that the minimum area of any triangle formed by the points is maximized .
MinMax Distance $N{=}3$	For given $N$ and dimension $d$ , find $N$ points in $\mathbb{R}^{d}$ that maximize the ratio between the minimum and maximum pairwise distances .
Signal Processing	Optimize a continuous signal processing objective under noisy evaluation, where candidate programs transform an input signal and are scored by an external evaluator.

Appendix

Table 7: Mathematical optimization benchmarks used in AdaEvolve. Each task is defined exactly as in Appendix B of the AlphaEvolve paper.

Backbone	OpenEvolve	AdaEvolve
GPT-5	42%	49%
Gemini-3-Pro	44%	50%

Appendix

Table 8: AdaEvolve performance on ARC-AGI-2 benchmarks. Values denote final accuracy under a matched inference budget.

Large Language Model-Guided Optimization

Large Language Model-Guided Optimization uses language models to propose, steer, and refine solutions to hard search and design problems.

See all 587 papers

EvoX: Meta-Evolution for Automated Discovery
Feb 26, 2026Shu Liu, Shubham Agarwal, Monishwaran Maheswaran +14Evolutionary Optimization LLMs for Evolutionary Optimization
UC Berkeley · Stanford University · Bespoke Labs
TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution
Apr 12, 2026Yang Yang, Zining Zhong, Jindong Li +5Large Language Model-Guided Optimization Evolutionary Optimization
Artificial Intelligence Thrust, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.
Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery
May 21, 2026Halil Alperen Gozeten, Xuechen Zhang, Emrullah Ildiz +3Evolutionary Optimization Evolutionary Multitask Optimization
University of Michigan - Ann Arbor · University of California San Diego

Feb 26, 2026J/K move · Enter open · S save

EvoX: Meta-Evolution for Automated Discovery

Shu Liu, Shubham Agarwal, Monishwaran Maheswaran, Mert Cemri+13

UC Berkeley · Stanford University · Bespoke Labs

Evolutionary Optimization LLMs for Evolutionary Optimization Large Language Model-Guided Optimization Automated Algorithm Discovery

Open paper AskPDF arXiv

Recent work such as AlphaEvolve has shown that combining LLM-driven optimization with evolutionary search can effectively improve programs, prompts, and algorithms across domains. In this paradigm, previously evaluated solutions are reused to guide the model toward new candidate solutions. Crucially, the effectiveness of this evolution process depends on the search strategy: how prior solutions are selected and varied to generate new candidates. However, most existing methods rely on fixed search strategies with predefined knobs (e.g., explore-exploit ratios) that remain static throughout execution. While effective in some settings, these approaches often fail to adapt across tasks, or even within the same task as the search space changes over time. We introduce EvoX, an adaptive evolution method that optimizes its own evolution process. EvoX jointly evolves candidate solutions and the search strategies used to generate them, continuously updating how prior solutions are selected and varied based on progress. This enables the system to dynamically shift between different search strategies during the optimization process. Across nearly 200 real-world optimization tasks, EvoX outperforms existing AI-driven evolutionary methods including AlphaEvolve, OpenEvolve, GEPA, and ShinkaEvolve on the majority of tasks.

Abstract

Large Language Model-Guided Optimization

Explore similar work

EvoX: Meta-Evolution for Automated Discovery

TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution

Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery