ATLAS: Scaffold-Free Algorithm Synthesis by LLMs via Embedding-Guided Quality-Diversity Search

Danial Yazdani; Mohammad Nabi Omidvar; Yuan Sun; Maksud Ibrahimov; Xiaodong Li

cs.AIAug 16, 2026

ATLAS: Scaffold-Free Algorithm Synthesis by LLMs via Embedding-Guided Quality-Diversity Search

Authors: Danial Yazdani, Mohammad Nabi Omidvar, Yuan Sun, Maksud Ibrahimov, Xiaodong Li

Organizations: School of Computing Technologies, RMIT University, Melbourne, VIC, Australia · School of Computing and Leeds University Business School, University of Leeds, Leeds, UK · La Trobe Business School, La Trobe University, Melbourne, VIC, Australia · Effective AI, Melbourne, VIC, Australia

Abstract

Most LLM-based automated algorithm design methods optimize a designated component within a human-specified scaffold, fixing overall organization and component interactions. We present ATLAS, an embedding-guided quality-diversity framework for scaffold-free full-algorithm synthesis in combinatorial optimization. The problem specification supplies objectives and constraints; a minimal I/O interface fixes only instance and solution formats; the LLM chooses and restructures components, interactions, and control flow. This freedom enlarges the search space, risking invalid candidates and premature convergence to one design region. ATLAS independently detects execution, interface, and feasibility failures, recomputes objectives, and applies error-conditioned repair; similarity-based archive management preserves algorithms across embedding-space regions to counter premature convergence. Its three-layer search refines the best design, gives other regions dedicated refinement opportunities, and performs cross-region synthesis to recombine components and their interactions. Across four NP-hard problems, ATLAS outperforms several state-of-the-art component-synthesis methods and a matched full-synthesis baseline while remaining competitive with strong human-designed algorithms. One ATLAS run retains several algorithms with comparable performance from distinct embedding-space regions rather than a single design. Code inspection finds that these multi-component designs differ in their primary construction or global-search backbone. Our results suggest that embedding-guided quality-diversity search can make the enlarged full-algorithm design space practically searchable. Source code and exact executable prompts are available at https://github.com/Danial-Yazdani/ATLAS .

Figures & tables

Fig. 1 : Component-based vs. scaffold-free full-algorithm synthesis. ATLAS synthesizes a complete optimization algorithm that receives problem instances and returns solutions through a fixed external I/O interface, without a user-provided internal algorithmic scaffold; component methods synthesize a designated function within user-provided control flow.

Fig. 2 : Overview of ATLAS. (Top-Left) Initialization generates diverse candidates via Create and Diverge operators. (Bottom-Left) Candidate Processing evaluates, repairs, and embeds generated algorithms. (Bottom-Center) Archive Management performs deduplication and capacity-constrained pruning. (Top-Center) Archive Clustering organizes algorithms by similarity under the selected embedding into the elite region

\mathcal{G}^{*}

(gold stars) and semantic clusters (colored circles); cluster representatives are shown as squares. (Right) Three-Layer Search allocates resources hierarchically: Layer 1 refines the best algorithm (

a^{*}

) and its neighborhood; Layer 2 refines cluster representatives in isolation; Layer 3 performs cross-cluster discovery. Candidates cycle through processing and archive integration to enable systematic refinement across multiple represented regions.

Parameter	Value	Parameter	Value
Archive capacity	$N_{\max}=100$	Layer 1 size	$\|\mathcal{G}^{*}\|=5$
Initialization	50 algorithms	Clustering	K-Medoids, adaptive $K$
Multi-ref operators (refs)		Single-ref operator probs
Combine	$m_{\text{comb}}=2$	Improve , Tune	$p=0.5$
Diverge	$m_{\text{div}}=3$	Simplify	$p=0.2$
Embedding model	mGTE-large-en-v1.5 [ 58 ]	LLM	GPT-5-mini (T=1.0, LOW)

TABLE I : ATLAS configuration. Complete details and rationale in Appendix B-E3 .

Group	Method	FSS $(n50m10)$	CVRP $(n50)$	CVRPTW $(n50)$	QAP $(n50)$
Domain-specific	NEH	2.809%	–	–	–
	IG-TB (Taillard acc.)	0.000%	–	–	–
	Iterative Beam Search	1.787%	–	–	–
	OR-Tools (GLS)	–	1.430%	0.121%	–
	PyVRP (ILS)	–	0.000%	0.000%	–
	VROOM	–	0.095%	0.060%	–

TABLE II : Comparison with baselines on the default benchmark settings using relative mean gaps. Values are relative mean gaps (%) to the statistically selected human-designed reference in each setting; lower is better. The reference in each column is the human-designed, domain-specific method with the lowest mean within the statistically best-performing human-designed group and therefore has gap 0.000%. Statistical tests use the paired raw objectives, not these gaps. Boldface identifies the statistically best-performing group across all methods under the paired Wilcoxon signed-rank test: the method with the lowest mean and any method not significantly different from it. Light-gray cell shading identifies the best LLM-based method in each setting.

Group	Method	FSS $(n25m5)$	FSS $(n100m20)$	CVRP $(n25)$	CVRP $(n100)$
Domain-specific	NEH	1.143%	4.133%	–	–
	IG-TB (Taillard acc.)	0.000%	0.000%	–	–
	Iterative Beam Search	2.321%	2.865%	–	–
	OR-Tools (GLS)	–	–	0.162%	4.930%
	PyVRP (ILS)	–	–	0.000%	0.000%
	VROOM	–	–	0.000%	0.975%

TABLE III : Generalization beyond the default benchmark settings using relative mean gaps (re-run from scratch). Values are relative mean gaps (%) to the statistically selected human-designed reference for every problem size. Lower is better. Boldface identifies the statistically best-performing group across all methods under the paired Wilcoxon signed-rank test: the method with the lowest mean and any method not significantly different from it. Light-gray cell shading identifies the best LLM-based method in each setting.

Problem	Training-selected representatives, ordered by descriptive test gap
FSS $(n50m10)$	Priority-Seeded Prefix-Guided LNS with Lightweight Anneal (0.331%); Priority-seeded NEH with Guided Neighborhood Search (0.458%); Priority-Spectral Hybrid with Tabu-Repair Local Search (0.472%); Prefix-Aware Hybrid Genetic with Annealed Polish (0.478%); Sampled Pairwise Greedy with Bounded Cache and Adaptive Iterated Refinement (0.527%); Hybrid NEH-Regret Genetic Search with Aggressive Complexity Reduction (0.714%); Guided Greedy Adaptive Cache with Pheromone and Block Relocation (0.965%); Bottleneck-Guided Greedy with ILS and Simulated Annealing (1.239%)
CVRP $(n50)$	Hybrid Multi-Start Clustered-Repair Large Neighborhood Search (0.000%); Multi-Start Regret Insertion with Focused Large Neighborhood Search (0.095%); Spectral-Seeding Regret-LNS with Focused Savings (0.191%); Hybrid Demand-Seeding with Localized Destroy-and-Repair (3.622%); Clustered Demand-Aware Insertion with Focused LNS (3.622%); Adaptive Clustered Chain-Savings Solver (3.908%)
CVRPTW $(n50)$	Large-Neighborhood Search with Time-Aware Regret Repair (0.000%); Ruin-and-Recreate Vehicle Routing with Time Windows (0.000%); Hybrid Label-Constructive and Ruin-Repair Routing (0.000%); Hybrid Adaptive Ruin-and-Repair with Temporal Beam Seeding (0.060%); Hybrid Guided Repair with Selective Savings Merge (0.302%); Permutation-Guided Large Neighborhood Search (2.171%); Radial and Time-Priority Insertion with Neighbor Guidance (3.317%)
QAP $(n50)$	Hierarchical Spectral-Memetic Solver with Block Neighborhoods and Cycle Diversification (0.440%); Soft-Probabilistic Beam with Block Reassignment and Tabu Polishing (0.461%); Adaptive Spectral-Pheromone Memetic Solver (0.496%); Spectral-Guided Adaptive Large-Neighborhood Search with Local Intensification (0.525%); Spectral-Softassign Hybrid with Mixed Local Search (0.813%)

TABLE IV : Training-selected archive representatives with descriptive test performance. All representatives are drawn from the final archive of a single synthesis run: the run, among the

R{=}5

synthesis runs, whose deployed algorithm achieved the best training objective. Representative identities are fixed from training objectives as the training-best members of final embedding-space clusters. Values in parentheses are test-set relative mean gaps (%) to the statistically selected human-designed, domain-specific reference; lower is better. Ordering by these gaps is descriptive only and affects neither selection nor statistical tests.

Configuration	CVRP $(n50)$ gap (%) $\downarrow$	FSS $(n50m10)$ gap (%) $\downarrow$
Variant A: All layers	0.094	0.347
Variant B: Layer 1 only	3.620	0.814
Variant C: Layers 2+3	0.668	0.474
Variant D: Layer 3 only	2.095	0.672

TABLE V : Relative mean gaps for the layer ablation. Percent gap to the statistically selected human-designed reference in each setting (PyVRP for CVRP and IG-TB for FSS); lower is better. Each gap is computed from the configuration’s mean raw objective and the corresponding reference mean. Boldface identifies the statistically best-performing group among the ablation configurations from paired tests on raw per-instance objectives; the displayed gaps are descriptive and are not test inputs. Each run is represented by its training-selected algorithm; the test set is used once for evaluation without fallback, and failed runs are retained. Detailed setup and analysis are provided in Appendix C-D .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Component-Based Single-Algorithm Synthesis (Appendix A-B1 )
Method	Scope	Primary Mechanism	Organizing Space	Preservation Logic
FunSearch [ 36 ]	Component	Islands + best-shot prompting	Output signatures (eval scores)	Cluster-biased sampling + island resets
EoH [ 25 ]	Component	Operator-based evolution	Fitness (scalar performance)	Fitness-based selection; no explicit diversity objective
ReEvo [ 57 ]	Component	Verbal gradients + reflection	Fitness + meta-objective constraints	Successful candidates under meta-objective constraints
HSEvo [ 9 ]	Component	Harmony Search + reflection	Embeddings (diversity indices) + fitness	Embedding diversity indices (SWDI/CDI) modulate search
MCTS-AHD [ 61 ]	Component	MCTS with LLM actions	Genealogy (tree) + UCT scores	Tree retention with UCT-based revisiting

Appendix

TABLE A.1: Comparative analysis of LLM-based algorithm synthesis methods. Scope distinguishes component optimization from full-algorithm synthesis. Organizing space indicates where diversity/selection pressure is applied. Preservation logic states the operational rule by which candidates persist in the search state; these mechanisms are reported descriptively without imposing a hierarchy among them.

Problem setting	Reference method	Raw mean objective
FSS $(n50m10)$	IG-TB (Taillard acc.) [ 38 , 45 , 12 ]	3 343.00
CVRP $(n50)$	PyVRP (ILS) [ 52 , 53 ]	10.49
CVRPTW $(n50)$	PyVRP (ILS) [ 52 , 53 ]	16.58
QAP $(n50)$	BMA [ 2 ]	5 704 446
FSS $(n25m5)$	IG-TB (Taillard acc.) [ 38 , 45 , 12 ]	1 599.84
FSS $(n100m20)$	IG-TB (Taillard acc.) [ 38 , 45 , 12 ]	6 808.45

Appendix

TABLE B.1 : Raw reference means used for percentage-gap reporting. For each problem and size, paired tests on raw per-instance objectives identify the statistically best-performing group of human-designed, domain-specific methods; the member of that group with the lowest mean supplies the reported reference. These are empirical reference means on the study instances, not optima or best-known-solution claims. The CVRP

n25

methods tie at the reported precision.

	CVRP $(n50)$ gap (%) $\downarrow$			FSS $(n50m10)$ gap (%) $\downarrow$
$\boldsymbol{n_{\text{intensify}}}$	2	3	4	2	3	4
Relative mean gap (%)	0.572	0.094	0.095	0.401	0.347	0.351

Appendix

TABLE B.2 : Relative mean gaps for the

n_{\mathrm{intensify}}

sensitivity analysis. Percent gap to the strongest human-designed reference in each setting (PyVRP for CVRP and IG-TB for FSS); lower is better. Values are computed from the underlying raw objective means. The setting

n_{\mathrm{intensify}}=3

was selected using training-set performance only; the test-set gaps are reported only for interpretability.

Order	Class	Identifying evidence	Correction objective
1	Memory limit	Memory-limit flag or MemoryError	Reduce large materialized structures, unbounded containers, redundant copies, or deep recursion
2	Partial failure	Some training instances succeed and others fail	Preserve the core approach while handling edge cases and removing instance-dependent failures
3	Timeout	Timeout marker in the aggregate error	Reduce computational complexity, redundant work, and excessive iteration within the existing approach
4	Constraint violation	A feasibility keyword in the evaluator message (Table B.4 )	Correct construction or modification logic so returned solutions satisfy the problem constraints
5	Runtime error	Exception, error, or traceback markers after excluding earlier classes	Locate and correct the source-level crash and add relevant defensive checks
6	Framework violation	Remaining unmatched validation or interface failure	Correct the entry point, signature, return type, output structure, or other interface requirement

Appendix

TABLE B.3 : Failure classes and correction objectives used by Repair .

Category	Keywords
Capacity and resources	exceeds capacity; capacity constraint; over capacity
Completeness	not all customers visited; missing jobs; missing items; missing customers
Uniqueness	visited multiple times; duplicate; appears multiple times
Permutation and sequence	not a valid permutation; job sequence length; invalid permutation; sequence length
General validity	constraint violation; invalid solution; feasibility; infeasible; out of range; invalid index; invalid customer; invalid item

Appendix

TABLE B.4 : Keyword groups used to identify constraint-violation failures. Matches are case-insensitive.

Model	Access	Params	Dim.	Context
General-Purpose Models
text-embedding-3-large (OpenAI) [ 34 ]	Proprietary	–	3072	8K
text-embedding-3-small (OpenAI) [ 34 ]	Proprietary	–	1536	8K
Qwen3-Embedding-0.6B [ 60 ]	Open	0.6B	1024	32K
Qwen3-Embedding-4B [ 60 ]	Open	4.0B	2560	32K
mGTE-large-en-v1.5 [ 58 ]	Open	434M	1024	8K

Appendix

TABLE C.1 : Embedding Model Specifications. Models are grouped into general-purpose encoders and code-specialized encoders.

General-Purpose Models
	Text-Only		Code-Only		Concatenation (Early Fusion)
Model Architecture	NNA (%) $\uparrow$	CCI $\uparrow$	NNA (%) $\uparrow$	CCI $\uparrow$	NNA (%) $\uparrow$	CCI $\uparrow$
text-embedding-3-small (OpenAI)	94.62	0.40	73.32	0.23	91.33	0.32
text-embedding-3-large (OpenAI)	91.44	0.35	65.38	0.14	86.62	0.25
Qwen3-Embedding-0.6B	88.77	0.29	75.81	0.22	91.80	0.31
Qwen3-Embedding-4B	89.53	0.29	77.09	0.21	95.33	0.39

Appendix

TABLE C.2 : Single-Encoder Strategy Results. Nearest Neighbor Accuracy (NNA) and Class Cohesion Index (CCI) for text-only, code-only, and concatenation strategies across all models. General-purpose models are evaluated on all three strategies, while code-specialized models are evaluated only in the code-only setting. Bold indicates the best NNA performance within each strategy. Higher is better for both metrics.

Panel of Figure 15 — Fig. C.1 : Dual-View (Late Fusion) Sensitivity Analysis. We sweep the mixing parameter $\alpha$ from 0.0 (code-only) to 1.0 (text-only).

Persona Name	Persona Text
default	You are an expert computer scientist specializing in algorithm design for combinatorial optimization problems.
Single-Dimension Personas
simplicity_focused	You are an algorithm designer who prioritizes elegant, minimal solutions.
complexity_embracing	You are an algorithm designer who builds sophisticated, feature-rich solutions.
modernist	You are an algorithm designer who uses cutting-edge techniques and recent innovations.
traditionalist	You are an algorithm designer who relies on proven, standard approaches.

Appendix

TABLE C.3 : Persona library used in the initialization-strategy analysis. The default persona is used in ATLAS; the remaining personas are used to evaluate prompt-level variation during initialization.

CVRP $(n50)$
	Baseline			+Random Persona			+Divergence (ATLAS)
Model	Valid $\uparrow$	MNND $\uparrow$	MPD $\uparrow$	Valid $\uparrow$	MNND $\uparrow$	MPD $\uparrow$	Valid $\uparrow$	MNND $\uparrow$	MPD $\uparrow$
GPT-4.1-Mini	0.96	0.015	0.035	0.98	0.023	0.045	0.93	0.058	0.151
GPT-5-mini (low)	0.99	0.037	0.103	0.98	0.034	0.097	0.98	0.062	0.159
Claude Haiku 4.5	0.86	0.029	0.077	0.89	0.027	0.068	0.90	0.064	0.147
Llama 4 Maverick	0.85	0.020	0.095	0.87	0.019	0.094	0.84	0.039	0.123

Appendix

TABLE C.4 : Initialization-strategy analysis. Results are averaged over 5 runs of 50 algorithms each. Valid denotes the fraction of generated algorithms that pass validation. MNND and MPD measure local and global diversity in embedding space, respectively; higher is better for all three metrics. Best results are shown in bold .

Guidance Strategy	CVRP $(n50)$ gap (%) $\downarrow$	FSS $(n50m10)$ gap (%) $\downarrow$
Variant A: Semantic Guidance	0.570	0.391
Variant B: Fitness-Based Guidance	2.601	0.715
Variant C: Random Guidance	3.143	0.884

Appendix

TABLE C.5 : Relative mean gaps for the semantic search-guidance ablation. Percent gap to the strongest human-designed reference in each setting (PyVRP for CVRP and IG-TB for FSS); lower is better. Values are computed from the underlying raw objective means. Boldface mirrors the paired raw-objective analysis and is not based on a test of the displayed gaps.

Reasoning effort	Relative mean gap (%) $\downarrow$		Total tokens per run (k)
Reasoning effort	CVRP $(n50)$	FSS $(n50m10)$	CVRP $(n50)$	FSS $(n50m10)$
Minimal	1.815	0.597	6 146 (135)	4 258 (178)
Low (default)	0.094	0.347	6 335 (144)	4 491 (246)
Medium	0.092	0.301	8 126 (161)	5 865 (272)
High	0.000	0.292	10 928 (241)	8 247 (294)

Appendix

TABLE C.6 : Reasoning-effort sensitivity and cost analysis for GPT-5-mini . Test performance is reported as the relative mean gap (%) to the strongest human-designed reference in each setting (PyVRP for CVRP and IG-TB for FSS); lower is better. Gaps are computed from the underlying raw objective means. Token usage is the mean total tokens consumed per run, in thousands, with standard errors in parentheses; each run uses 500 evaluated operator executions. Boldface mirrors the paired raw-objective analysis and is not based on a test of the displayed gaps.

Underlying LLM	FSS $(n50m10)$ gap (%) $\downarrow$
GPT-5-nano (low)	0.557
GPT-5-mini (low)	0.347
GPT-5 (low)	0.284
Claude Haiku 4.5 (default)	0.540

Appendix

TABLE C.7 : Relative mean gaps for the underlying-LLM comparison on FSS. Percent gap to IG-TB, the strongest human-designed reference for this setting; lower is better. Values are computed from the underlying raw objective means. Boldface mirrors the conclusion of the paired raw-objective analysis and is not based on a test of the displayed gaps.

LLM	Training-selected representatives, ordered by descriptive test gap
GPT-5-nano (low)	Permutation Based Flow Shop Heuristic with Cascaded Insertion and Local Search (0.474%); Hybrid Seeded Cascaded Insertion with Global Lookahead (0.501%); Fixed Permutation Flow Shop Seeded Local Search (0.527%); Diversified Guided Cascaded Beam Search for Permutation Flow Shop (0.753%); Hybrid Prefix Guided Insertion with Global Lookahead and Prefix Preserving Local Search (1.333%); Genetic Algorithm based on permutation population with crossover and mutation (1.372%); Greedy NEH-inspired permutation for flow shop makespan minimization (2.809%); Recursive Skyline Block Sequencing (3.319%); Hybrid Pheromone Guided Time Slice Sequencing (3.383%); Pheromone Guided Permutation Flow Shop Scheduler (3.483%)
GPT-5-mini (low)	Priority-Seeded Prefix-Guided LNS with Lightweight Anneal (0.331%); Priority-seeded NEH with Guided Neighborhood Search (0.458%); Priority-Spectral Hybrid with Tabu-Repair Local Search (0.472%); Prefix-Aware Hybrid Genetic with Annealed Polish (0.478%); Sampled Pairwise Greedy with Bounded Cache and Adaptive Iterated Refinement (0.527%); Hybrid NEH-Regret Genetic Search with Aggressive Complexity Reduction (0.714%); Guided Greedy Adaptive Cache with Pheromone and Block Relocation (0.965%); Bottleneck-Guided Greedy with ILS and Simulated Annealing (1.239%)
GPT-5 (low)	Pairwise backbone scheduling with regret beam construction and consensus edge relinking (0.277%); Precedence corridor scheduling with shockwave rebuild and edge voted relinking (0.280%); Vote guided anchor beam scheduling with cached insertion scoring and focused reconstruction (0.304%); Multi-sequence NEH flow shop heuristic with CDS, Palmer, and elite reinsertion search (0.391%); Consensus rollout scheduling with hot segment repair and block guided relinking (0.454%); Pairwise precedence learning with exact insertion and conflict focused segment reconstruction (0.460%); Tournament backbone search with window dynamic programming (0.800%); Cross-entropy random-key optimization with block permutation polishing (2.746%)
Claude Haiku 4.5	Hybrid Adaptive Simulated Annealing with Dynamic Critical Path Guidance (0.484%); Simulated Annealing with Critical Path Optimization for Flow Shop Scheduling (0.571%); Simulated Annealing with Greedy Initialization for Flow Shop Scheduling (0.579%); Hybrid Memetic Algorithm with Adaptive Dual-Strategy Exploration (0.809%); Hybrid Adaptive Metaheuristic with Critical Path Learning (1.016%); Hybrid Initialization and Local Search for Flow Shop Scheduling (1.237%); Hybrid Memetic Strategy with Dual-Phase Evolution and Adaptive Intensity Balancing (1.537%); Genetic Algorithm with Dual Crossover Operators and Adaptive Multi-Strategy Mutations (1.558%); Constraint-Based Graph Routing with Bottleneck-Aware Optimization (2.181%)

Appendix

TABLE C.8 : Training-selected FSS archive representatives by LLM. Representative identities are fixed from training objectives as the training-best members of final semantic clusters. Within each LLM, the displayed set is ordered by test-set relative mean gap only for interpretation. Values in parentheses are test-set relative mean gaps (%) to IG-TB, the strongest human-designed FSS reference; lower is better. This descriptive ordering does not affect representative membership, the training-selected deployable algorithm, the headline LLM comparison, or statistical tests.

Automated Algorithm Discovery

Automated algorithm discovery develops methods that generate and test executable algorithms for optimization, science, software, and machine learning.

See all 170 papers

Jun 29, 2026

AlgoSkill: Learning to Design Algorithms by Scheduling Human-Like Skills

Designing an algorithm from a natural-language problem statement requires identifying the problem structure, reading constraints, choosing a suitable paradigm, checking correctness, and refining complexity. Existing large language model (LLM) methods often rely on direct generation or generic self-refinement, leaving these steps implicit. We propose AlgoSkill, which models algorithm design as sequential decision-making over a typed library of algorithmic skills, including abstraction, constraint analysis, state design, data-structure selection, proof checking, counterexample construction, and complexity refinement. A learned scheduler proposes skills from the current design state, while a Monte Carlo Tree Search (MCTS) controller explores skill sequences using verification feedback from compilation, testing, stress testing, and complexity analysis. Experiments on competitive programming and combinatorial optimization benchmarks show that AlgoSkill improves over direct LLM generation, chain-of-thought prompting, self-refinement, and MCTS without typed skills. Ablations show that typed skills, verification-based repair, and search-based scheduling each contribute to performance. These results support treating automatic algorithm design as verification-guided skill scheduling rather than one-shot code generation.

Xinyuan Song, Zekun Cai, Liang Zhao

Emory University, Atlanta, GA, USA · The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan

Automated Algorithm Discovery LLM-Based Program Synthesis Algorithmic Reasoning

Ask PDF

Apr 27, 2026

A2DEPT: Large Language Model-Driven Automated Algorithm Design via Evolutionary Program Trees

Designing heuristics for combinatorial optimization problems (COPs) is a fundamental yet challenging task that traditionally requires extensive domain expertise. Recently, Large Language Model (LLM)-based Automated Heuristic Design (AHD) has shown promise in autonomously generating heuristic components with minimal human intervention. However, most existing LLM-based AHD methods enforce fixed algorithmic templates to ensure executability, which confines the search to component-level tuning and limits system-level algorithmic expressiveness. To enable open-ended solver synthesis beyond rigid templates, we propose Automated Algorithm Design via Evolutionary Program Trees (A2DEPT), which treats LLMs as system-level algorithm architects. A2DEPT explores the vast program space via a tree-structured evolutionary search with hybrid selection and hierarchical operators, enabling iterative refinement of complete algorithms. To make open-ended generation practical, we enforce executability with a lightweight program-maintenance loop that performs feedback-driven repair. In experiments, A2DEPT consistently outperforms representative LLM-based baselines on both standard and highly constrained benchmarks. On the standard benchmarks, it reduces the mean normalized optimality gap by 9.8% relative to the strongest competing AHD baseline.

Bin Chen, Shouliang Zhu, Beidan Liu +4

The Institute of Intelligent Computing, University of Electronic Science and Technology of China, Chengdu 611731, China · College of Systems Engineering, National University of Defense Technology, Changsha 410073, China · Academy of Military Medical Sciences, Beijing 100071, China

Automated Algorithm Discovery Automated Heuristic Design Metaheuristic Optimization Large Language Model-Guided Optimization Genetic Programming LLM-Based Program Synthesis+1

Ask PDF

Mar 23, 2026

Early Discoveries of Algorithmist I: Promise of Provable Algorithm Synthesis at Scale

Designing algorithms with provable guarantees that also work well in practice remains difficult, requiring both mathematical reasoning and careful implementation. Existing approaches that bridge worst-case theory and empirical performance, such as beyond-worst-case analysis and data-driven algorithm selection, typically assume prior distributional knowledge or restrict attention to a fixed pool of algorithms. Recent progress in LLMs suggests a new possibility: provable algorithm synthesis on the fly. To study this, we built Algorithmist, an autonomous researcher agent on top of GitHub Copilot that runs a multi-agent research-and-review loop, with separate stages for idea generation, algorithm and proof development, proof-guided implementation, and review of proofs, code, and their alignment. We evaluate Algorithmist on research-level tasks in private data analysis and clustering. When asked to design practical methods that jointly satisfy privacy, approximation, and interpretability requirements, it produced provably sound and empirically effective algorithms, together with research-style writeups and audited implementations. It also found improved algorithms in some settings, explained principled barriers in others, and uncovered a subtle proof bug in prior published work. More broadly, our results suggest a new paradigm in which LLM systems generate research-paper-quality algorithmic artifacts tailored to each dataset and deployment setting. They also point to a proof-first code-synthesis paradigm, in which code is developed alongside a structured natural-language proof intermediate representation and kept aligned with it throughout synthesis.

Janardhan Kulkarni

Automated Algorithm Discovery Program Synthesis AI Agents for Scientific Discovery

Ask PDF

Abstract

Automated Algorithm Discovery

Explore similar work

AlgoSkill: Learning to Design Algorithms by Scheduling Human-Like Skills

A2DEPT: Large Language Model-Driven Automated Algorithm Design via Evolutionary Program Trees

Early Discoveries of Algorithmist I: Promise of Provable Algorithm Synthesis at Scale