TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution

Yang Yang; Zining Zhong; Jindong Li; Jiemin Wu; Kaishen Yuan; Wenshuo Chen; Menglin Yang; Yutao Yue

cs.NEApr 12, 2026

TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution

Authors: Yang Yang, Zining Zhong, Jindong Li, Jiemin Wu, Kaishen Yuan, Wenshuo Chen, Menglin Yang, Yutao Yue

Organizations: Artificial Intelligence Thrust, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.

Abstract

LLM-driven program evolution can discover high-quality programs, but high API cost and run-to-run variance hinder reliable progress. We propose TurboEvolve, a multi-island evolutionary framework that improves sample efficiency and stability under matched API-cost budgets. Inspired by multiple-offspring strategies in evolutionary algorithms, TurboEvolve introduces Verbalized Sampling, which elicits K diverse candidates in one LLM call while using self-assigned probabilities only as generation signals, together with an online scheduler that adapts K to expand exploration under stagnation and reduce redundant generation during steady progress. To exploit existing solution pools, TurboEvolve further introduces island-aware initialization, which clusters seed programs into mode-specific subpopulations and applies controlled cross-cluster mixing with elitist preservation to balance diversity, sharing, and refinement. We also curate a reusable cross-task solution-pool dataset to support warm-start initialization studies and matched-resource comparisons. Across mathematical discovery and GPU-kernel optimization tasks, TurboEvolve achieves stronger cost-quality trade-offs and more stable performance than controlled baselines, while reaching competitive reported scores.

Figures & tables

Figure 1: Comparison between AlphaEvolve and TurboEvolve . AlphaEvolve uses single-program initialization and one descendant per iteration, whereas TurboEvolve adds island-aware initialization, Verbalized Sampling for multi-candidate generation, and adaptive $K$ scheduling.

Method	Autocorr_ First $\downarrow$	Autocorr_ Second $\uparrow$	Uncer- tainty $\downarrow$	Circle Pack_26 $\uparrow$	Erdos_ Overlap $\downarrow$	Matmul_ SwishGN $\uparrow$	Conv3d_ Mish $\uparrow$	Conv2d_ BN $\uparrow$
	Gemini-2.0-Flash/Pro
AlphaEvolve	1.505294	0.896280	0.321587	2.635863	0.380923	–	–	–
	Gemini-2.5-Flash
LoongFlow	1.513806	0.910976	0.358684	2.570675	0.381078	1.852214	2.207348	1.198789
PartEvo	1.510948	0.925310	0.352101	2.377295	0.381039	1.356195	1.432049	1.031080
OpenEvolve	1.509724	0.934569	0.358149	2.630972	0.381420	1.894889	2.210669	1.301967

Table 1: Best-over-runs scores under matched API-cost budgets. Each controlled configuration is run five times. Arrows indicate optimization direction; bold and underline mark the best and second-best controlled results within each backend block. AlphaEvolve values are reported references with a different backend and are excluded from rankings.

Figure 2: Run-to-run consistency under Gemini-2.5-Flash. Bars show the mean of final per-run best scores over five runs, and error bars show standard deviation.

Figure 3: Backbone-paired efficiency at equal evaluated-program horizons: 800 programs for mathematical tasks and 100 for KernelBench tasks. (A) Endpoint score, LLM calls, and API cost. OE: OpenEvolve ; TE: TurboEvolve . Bold marks the better score or lower resource use within each pair. (B) Representative CirclePack_26 trajectories. Columns: Gemini-2.5-Flash and Gemini-3-Flash; rows: best score and generated programs versus API cost.

Table 5

Figure 4: Within-event prefix replay on CirclePack_26 (

K=7

). Left: probability that a retained prefix contains a valid improvement. Right: best score change

\max_{j\leq m}\Delta_{j}

relative to the parent; negative values indicate regression. Curves group events by inspiration-score quintile; lines show medians across runs and bands show IQR.

Variant	$T$	Candidate counts $K$	Erdos_Overlap $\downarrow$	CirclePack_26 $\uparrow$
Default	3	$\{1,3,5,7\}$	$0.380927\pm 0.000041$	$2.635114\pm 0.001690$
Wider range	3	$\{1,3,5,7,9\}$	$0.380985\pm 0.000069$	$2.634228\pm 0.002474$
Slower update	5	$\{1,3,5,7\}$	$0.380944\pm 0.000042$	$2.632477\pm 0.002243$

Table 4: Sensitivity to scheduler range and update window. Entries are mean

\pm

standard deviation; arrows indicate the optimization direction.

Figure 5: Adaptive

K

ablation with Gemini-2.5-Flash. Thresholds count programs needed to reach each score.

Figure 6: Warm-start initialization on Uncertainty . TurboEvolve with different seed-pool allocations: random, kmeans, and kmeans+elite. Left: bottom 80% pool (top 20% removed). Right: full pool. Curves show best-so-far objective across runs vs. evolved programs.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Run-to-run consistency under Gemini-3-Flash. Bars show the mean of final per-run best scores over five runs, and error bars show standard deviation.

Panel of Figure 8 — Figure 8: Additional backbone-paired efficiency trajectories: Mathematical tasks. Each task panel shows best-so-far score and generated programs versus API cost for OpenEvolve and TurboEvolve . Columns within each panel correspond to Gemini-2.5-Flash and Gemini-3-Flash.

Panel of Figure 9 — Figure 9: Additional backbone-paired efficiency trajectories: KernelBench tasks. Each task panel shows best-so-far score and generated programs versus API cost for OpenEvolve and TurboEvolve . Columns within each panel correspond to Gemini-2.5-Flash and Gemini-3-Flash.

Figure 10: Rank-wise quality and diversity under large

K

. We profile candidates within the same LLM call by their rank in the returned VS order. We report validity rate, valid-only score improvement

\Delta

, and a diversity proxy (archive cell-distance to the primary parent), aggregated at the run level with median and IQR. For panels that compare rank groups, we summarize ranks into head (1–2), mid (3–5), and tail (6–7) using the same

K{=}7

events.

Figure 11: Distance– $K$ interaction under adaptive scheduling. Heatmaps report (A) $P(\mathrm{improve})$ and (B) $\mathbb{E}[\Delta_{\mathrm{best}}]$ over pre-generation distance deciles and $K\in\{1,3,5,7\}$ . Events are first aggregated within each run on the $(\text{distance},K)$ grid and then summarized across runs; cell opacity reflects the number of events in each bin.

Figure 12: QD footprint over evaluation budget. We report archive coverage (number of occupied cells) and cell quality (mean best score per occupied cell) as functions of evaluation budget for TurboEvolve . Each run is interpolated onto a shared budget grid; lines show the median across runs and bands indicate IQR.

Method	Input: 3,626,247	Output: 393,339	Total: 4,019,586
TurboEvolve	$\mathbf{0.329957}\mathbf{\pm 0.002858}$	$\mathbf{0.329957}\mathbf{\pm 0.002858}$	$\mathbf{0.329957}\mathbf{\pm 0.002858}$
OpenEvolve	$0.348475\pm 0.012751$	$0.348368\pm 0.012869$	$0.348475\pm 0.012751$
Independent- $K$	$0.353319\pm 0.002742$	$0.350119\pm 0.006296$	$0.353319\pm 0.002742$
PartEvo	$0.350246\pm 0.024267$	$0.385045\pm 0.038397$	$0.361850\pm 0.022181$
LoongFlow	$0.374063\pm 0.007187$	$0.374063\pm 0.007187$	$0.374063\pm 0.007187$

Appendix

Table 5: Matched-token controls on Uncertainty with Gemini-3-Flash. Entries are mean

\pm

standard deviation; lower objective values are better. Each column applies a separate token budget.

Matched resource	TurboEvolve	Independent- $K$
Input tokens	800	$169.3\pm 15.9$
Output tokens	800	$204.0\pm 22.9$
Total tokens	800	$173.3\pm 14.3$

Appendix

Table 6: Evaluated-program counts under the token budgets of Table I.1 .

Method	Objective	Evaluated programs
Erdos_Overlap ( $\downarrow$ )
TurboEvolve	$\mathbf{0.380951390}$ $\mathbf{\pm 0.000001475}$	$798.5$ $\pm 0.5$
OpenEvolve	$0.383684278$ $\pm 0.002500354$	$241.0$ $\pm 32.9$
LoongFlow	$0.382213364$ $\pm 0.001534239$	$109.0$ $\pm 13.1$
PartEvo	$0.381014212$ $\pm 0.000042971$	$196.3$ $\pm 4.5$
Conv2d_BN ( $\uparrow$ )

Appendix

Table 7: Local DeepSeek-V4-Flash results on four H200 GPUs. Entries are mean

\pm

standard deviation. Bold marks the strongest mean objective within each task; these are descriptive comparisons, not significance tests.

Method	Active GPU-h	Reserved GPU-h
Erdos_Overlap
TurboEvolve	$8.225$ $\pm 0.164$	$33.314$ $\pm 0.485$
OpenEvolve	$5.899$ $\pm 0.599$	$16.199$ $\pm 3.813$
LoongFlow	$6.179$ $\pm 0.404$	$23.395$ $\pm 12.828$
PartEvo	$10.300$ $\pm 0.963$	$17.852$ $\pm 0.096$
Conv2d_BN

Appendix

Table 8: Local-inference GPU resource measurements (mean

\pm

standard deviation). Active and reserved GPU hours are reported separately. Total resource use and resource use per evaluated program answer different questions; the methods evaluate different numbers of programs.

Matched resource	TurboEvolve	OpenEvolve	LoongFlow	PartEvo
Input tokens	$\mathbf{0.338952}$ $\mathbf{\pm 0.013404}$	$0.361739$ $\pm 0.017780$	$0.374063$ $\pm 0.007187$	$0.367575$ $\pm 0.018286$
Output tokens	$\mathbf{0.338211}$ $\mathbf{\pm 0.013874}$	$0.360487$ $\pm 0.019143$	$0.374063$ $\pm 0.007187$	$0.361777$ $\pm 0.022264$
Cached-input usage	$\mathbf{0.345516}$ $\mathbf{\pm 0.024338}$	$0.361187$ $\pm 0.018641$	$0.379927$ $\pm 0.007187$	$0.368904$ $\pm 0.015781$
API cost	$\mathbf{0.338211}$ $\mathbf{\pm 0.013874}$	$0.360849$ $\pm 0.018612$	$0.374063$ $\pm 0.007187$	$0.361777$ $\pm 0.022264$
Evaluator compute	$\mathbf{0.338662}$ $\mathbf{\pm 0.013573}$	$0.360504$ $\pm 0.019125$	$0.372746$ $\pm 0.009043$	$0.367575$ $\pm 0.018286$
Active wall time	$\mathbf{0.338662}$ $\mathbf{\pm 0.013573}$	$0.360504$ $\pm 0.019125$	$0.374063$ $\pm 0.007187$	$0.361777$ $\pm 0.022264$

Appendix

Table 9: Representative resource-matched quality on Uncertainty with Gemini-3-Flash. Entries are mean

\pm

standard deviation; lower is better. Each row independently matches one resource. This table is distinct from the token-budget experiment in Table I.1 .

Measure	TurboEvolve	OpenEvolve	LoongFlow	PartEvo
API cost ($)	$1.9598\pm 0.3079$	$1.9501\pm 0.0050$	$1.8559\pm 0.1055$	$1.9555\pm 0.0038$
Evaluated programs	$797.4\pm 2.1$	$107.0\pm 15.9$	$72.6\pm 30.3$	$233.6\pm 18.0$
Evaluator compute (h)	$0.63\pm 0.16$	$0.18\pm 0.08$	$1.03\pm 0.69$	$0.20\pm 0.05$
Active wall time (h)	$1.72\pm 0.71$	$0.90\pm 0.35$	$3.72\pm 0.83$	$1.61\pm 0.31$
Evaluator s/program	$2.87\pm 0.71$	$6.01\pm 2.09$	$49.33\pm 21.03$	$3.07\pm 0.62$
Active wall s/program	$7.78\pm 3.21$	$29.93\pm 9.02$	$211.65\pm 80.56$	$24.62\pm 3.53$

Appendix

Table 10: Evaluator and active-wall-time breakdown on CirclePack_26 with Gemini-3-Flash (mean

\pm

standard deviation). Per-program measurements are reported separately from aggregate totals.

Large Language Model-Guided Optimization

Large Language Model-Guided Optimization uses language models to propose, steer, and refine solutions to hard search and design problems.

See all 587 papers

AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization
Feb 23, 2026Mert Cemri, Shubham Agrawal, Akshat Gupta +9Large Language Model-Guided Optimization LLMs for Evolutionary Optimization
University of California, Berkeley · Bespoke Labs
EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution
Oct 8, 2026Yanning Dai, Yuhui Wang, Nanbo Li +2Large Language Model-Guided Optimization Self-Evolving Agents
Center of Excellence for Generative AI, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia. · Sakana AI, Tokyo, Japan.
Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery
May 21, 2026Halil Alperen Gozeten, Xuechen Zhang, Emrullah Ildiz +3Evolutionary Optimization Evolutionary Multitask Optimization
University of Michigan - Ann Arbor · University of California San Diego

Feb 23, 2026J/K move · Enter open · S save

AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization

Mert Cemri, Shubham Agrawal, Akshat Gupta, Shu Liu+8

University of California, Berkeley · Bespoke Labs

Large Language Model-Guided Optimization LLMs for Evolutionary Optimization Evolutionary Optimization LLM-Based Program Synthesis Zeroth-Order Optimization

Open paper AskPDF arXiv

The paradigm of automated program generation is shifting from one-shot generation to inference-time search, where Large Language Models (LLMs) function as semantic mutation operators within evolutionary loops. While effective, these systems are currently governed by static schedules that fail to account for the non-stationary dynamics of the search process. This rigidity results in substantial computational waste, as resources are indiscriminately allocated to stagnating populations while promising frontiers remain under-exploited. We introduce AdaEvolve, a framework that reformulates LLM-driven evolution as a hierarchical adaptive optimization problem. AdaEvolve uses an "accumulated improvement signal" to unify decisions across three levels: Local Adaptation, which dynamically modulates the exploration intensity within a population of solution candidates; Global Adaptation, which routes the global resource budget via bandit-based scheduling across different solution candidate populations; and Meta-Guidance which generates novel solution tactics based on the previously generated solutions and their corresponding improvements when the progress stalls. We demonstrate that AdaEvolve consistently outperforms the open-sourced baselines across 185 different open-ended optimization problems including combinatorial, systems optimization and algorithm design problems.

Abstract

Large Language Model-Guided Optimization

Explore similar work

AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization

EvoAlloc: A Self-Evolving Resource Allocation Agent for Efficient Program Evolution

Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery