Holistic Diversity Management in LLM-Guided Evolutionary Search

Seockbean Song; Youngsik Yoon; Sungjae Lee; Siwei Wang; Wei Chen; Jungseul Ok

cs.CLOct 28, 2026

Holistic Diversity Management in LLM-Guided Evolutionary Search

Authors: Seockbean Song, Youngsik Yoon, Sungjae Lee, Siwei Wang, Wei Chen, Jungseul Ok

Organizations: Graduate School of AI, POSTECH · Department of CSE, POSTECH · Microsoft Research Asia

Abstract

LLM-guided evolutionary search iteratively selects existing programs, uses LLMs as mutation operators to generate a new program, and evaluates it to guide subsequent search. To preserve diversity and reduce the risk of premature convergence, it often adopts the island model, in which groups of programs, called islands, evolve separately. However, this structure alone does not guarantee distinct search directions, as islands may start from similar programs or converge toward similar solutions during evolution. This suggests that diversity must be considered throughout the search process, from initializing islands with distinct programs to developing island trajectories and sharing discoveries across islands. To this end, we propose HoliDivEvolve , a holistic approach to establishing and effectively utilizing island diversity throughout evolutionary search. Specifically, we introduce diverse initialization to establish different initial programs, trajectory adaptive island selection to allocate computation based on current score, recent growth, and the remaining search budget, and growth adaptive migration to share alternative ideas across islands. Experiments across diverse benchmarks show that HoliDivEvolve achieves competitive performance on most evaluated tasks. We further conduct controlled analyses to examine how our design choices affect search behavior and final performance.

Figures & tables

Figure 1: Overview of HoliDivEvolve . We consider three components for diversity management: diverse initialization, trajectory adaptive island selection, and growth adaptive migration.

		Mathematical Optimization			Systems Optimization
Backbone	Algorithm	Heilbronn Triangle $\uparrow$	Circle Packing (Square) $\uparrow$	Signal Processing $\uparrow$	PRISM $\uparrow$	EPLB $\uparrow$	CloudCast $\downarrow$
GPT-5	OpenEvolve	$0.028\pm.006$	$2.531\pm.018$	$0.569\pm.047$	$\underline{26.23}\pm.00$	$0.127\pm.000$	$926.9\pm 171.0$
	ShinkaEvolve	$0.032\pm.012$	$2.464\pm.083$	$0.485\pm.044$	$\mathbf{26.26}\pm.00$	$0.118\pm.010$	$954.8\pm 125.0$
	AdaEvolve	$\underline{0.033}\pm.001$	$\mathbf{2.629}\pm.001$	$\underline{0.698}\pm.017$	$\leq 26.26$ †	$\underline{0.134}\pm.010$	$\underline{662.3}\pm.1$
	Ours	$\mathbf{0.034}\pm.002$	$\underline{2.622}\pm.008$	$\mathbf{0.722}\pm.013$	$\mathbf{26.26}\pm.00$	$\mathbf{0.146}\pm.000$	$\mathbf{643.9}\pm 36.6$
Qwen3-32B	OpenEvolve	$0.012\pm.011$	$1.970\pm.073$	$0.531\pm.029$	$23.55\pm.63$	$0.127\pm.000$	$1020.2\pm 25.9$

Table 1: Performance comparison across mathematical and systems optimization benchmarks using GPT-5 and Qwen3-32B (mean

\pm

SD). Bold indicates the best observed mean within each backbone, and underline indicates the second-best distinct mean, with ties determined at displayed precision. GPT-5 baselines are from Tables 1–2 of Cemri et al. (2026) (three runs). All methods use a total budget of 100 program generations: Ours allocates 20 generations to diverse initialization and 80 rounds to evolutionary search, while the baselines use all 100 rounds for search with the same initial program. Arrows indicate the preferred direction.

Metric $\downarrow$	Same	Ours (Max–min)
Best program similarity	0.183	0.062
Population proximity	0.424	0.086
Exact code overlap	0.183	0.000
Descendant proximity	0.122	0.085

Table 2: Cross-island redundancy after 20 evolutionary rounds in Signal Processing under same start and max–min initialization. Lower values indicate less overlap across islands. Best program similarity compares the current best programs across islands. Population proximity measures similarity between island populations. Exact-code overlap measures shared normalized programs; and Descendant proximity excludes the initial programs before measuring population similarity. Detailed definitions are provided in Appendix A.4 .

Initialization	Mean $\pm$ SD $\uparrow$
Same	0.700 $\pm$ 0.026
Random	0.682 $\pm$ 0.020
Ours (Max–min)	0.714 $\pm$ 0.022

Table 3: Comparison of initialization strategies on Signal Processing using Qwen3.8-27B over five random seeds. Results report the mean

\pm

sample standard deviation of the final best score.

Selection policy	Final score $\uparrow$	Selection-count SD
Round-robin	0.628 $\pm$ 0.031	0.00
Uniform	0.645 $\pm$ 0.036	5.49
UCB	0.633 $\pm$ 0.036	3.48
Ours	0.659 $\pm$ 0.044	17.33

Table 4: Comparison of island-selection policies using Qwen3.8-27B. Final score reports mean

\pm

sample standard deviation. Selection-count SD is the standard deviation of the selection counts across the four islands within each run, averaged across runs.

Panel of Figure 2 — Figure 2: Qualitative example of migration on Signal Processing. (a) The recipient parent and migrant exhibit complementary strengths, and the resulting child improves both accuracy and smoothness over the parent. (b) Migration occurs at iteration 25, followed by the discovery of an improved child at iteration 41.

Diverse init.	Adaptive selection	Adaptive migration	Final score $\uparrow$
			0.698 $\pm$ 0.032
✓			0.703 $\pm$ 0.047
	✓	✓	0.700 $\pm$ 0.026
✓	✓	✓	0.714 $\pm$ 0.022

Table 5: Component ablation on Signal Processing over five random seeds. Checkmarks indicate the use of diverse initialization, trajectory adaptive island selection, and growth adaptive migration. Without these components, we use the same initial program, round-robin selection, and no migration, respectively. Results report mean

\pm

sample SD.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Example best-so-far score trajectories on Signal Processing using Qwen3-32B. Ours uses 80 evolutionary rounds after 20 initial program generations, while the baselines use 100 evolutionary rounds.

Figure 4: Example solutions for Circle Packing with

n=26

using Qwen3-32B. Each panel shows a circle arrangement in the unit square, with its corresponding score in parentheses. Higher scores indicate better solutions.

Evolutionary Optimization

Evolutionary optimization searches for strong solutions through population-based variation, selection, and adaptation.

See all 295 papers

TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution
Apr 12, 2026Yang Yang, Zining Zhong, Jindong Li +5Large Language Model-Guided Optimization Evolutionary Optimization
Artificial Intelligence Thrust, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.
LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Search
May 10, 2026Temoor TanveerEvolutionary Optimization Large Language Model-Guided Optimization
Independent Researcher
SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution
Apr 27, 2026Sichun Luo, Yi Huang, Haochen Luo +7Automated Algorithm Discovery Evolutionary Optimization
1The University of Hong Kong · 2JIUTIAN Research, China Mobile · 3City University of Hong Kong

Apr 12, 2026J/K move · Enter open · S save

TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution

Yang Yang, Zining Zhong, Jindong Li, Jiemin Wu+4

Artificial Intelligence Thrust, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.

Large Language Model-Guided Optimization Evolutionary Optimization LLMs for Evolutionary Optimization

Open paper AskPDF arXiv

LLM-driven program evolution can discover high-quality programs, but high API cost and run-to-run variance hinder reliable progress. We propose TurboEvolve, a multi-island evolutionary framework that improves sample efficiency and stability under matched API-cost budgets. Inspired by multiple-offspring strategies in evolutionary algorithms, TurboEvolve introduces Verbalized Sampling, which elicits K diverse candidates in one LLM call while using self-assigned probabilities only as generation signals, together with an online scheduler that adapts K to expand exploration under stagnation and reduce redundant generation during steady progress. To exploit existing solution pools, TurboEvolve further introduces island-aware initialization, which clusters seed programs into mode-specific subpopulations and applies controlled cross-cluster mixing with elitist preservation to balance diversity, sharing, and refinement. We also curate a reusable cross-task solution-pool dataset to support warm-start initialization studies and matched-resource comparisons. Across mathematical discovery and GPU-kernel optimization tasks, TurboEvolve achieves stronger cost-quality trade-offs and more stable performance than controlled baselines, while reaching competitive reported scores.

Abstract

Evolutionary Optimization

Explore similar work

TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution

LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Search

SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution