Large Language Models (LLMs) are increasingly used for automated algorithm design. However the computational cost of evaluating the generated algorithms can be excessive. We consider the common LLM-driven automated algorithm design (LLM4AD) setting in which a candidate algorithm is evaluated by aggregating its performance over a shared set of training instances. This instance-wise structure raises a natural question: must every candidate be evaluated on the entire instance set before deciding whether it remains competitive? Taking inspiration from algorithm configuration, we introduce AdaEva, a drop-in adaptive partial-evaluation framework that progressively evaluates candidates on larger subsets of the same instance pool and eliminates unpromising candidates as evidence accumulates. Importantly, AdaEva leaves the underlying LLM4AD procedure and per-instance evaluator unchanged and requires no prior knowledge about instance difficulty. We instantiate this idea using successive halving (AdaEva-S) and statistical racing (AdaEva-R), and evaluate both mechanisms across three representative LLM4AD frameworks, multiple LLM backbones, and optimization domains spanning combinatorial and continuous black-box optimization. Under matched evaluation budgets, AdaEva more reliably balances evaluation effort across candidates than fixed partial-evaluation strategies, yielding strong search efficiency and anytime performance together with improved held-out generalization across the evaluated settings.
Figures & tables
Figure 1: LLM-driven algorithm design as a loop. At each generation, a prompt built from the problem specification and selected parents is passed to an LLM, which generates candidate algorithms; these are evaluated on problem instances, and the best are selected and fed back to guide the next generation. Standard frameworks evaluate every candidate on every instance (left). AdaEva replaces only the evaluation stage with adaptive partial evaluation (right): candidates are evaluated progressively, those that fall behind are eliminated early, and the saved budget is concentrated on the candidates that remain competitive. Prompt construction, generation, and selection are unchanged.
Framework
Problem
Search Pool
Held-out Set
M
N
T
B
EoH
OBP
5,000 items, capacity 100
1k/5k/10k items, capacities 100/500
20
20
20
10,000
TSP
100-city, uniform coordinates
29 TSPLIB instances
10
10
10
6,400
FSSP
50 jobs, 2–20 machines
Taillard benchmark
10
10
10
6,400
HiFo-Prompt
OBP
5,000 items, capacity 100
1k/5k/10k items, capacities 100/500
10
50
10
12,500
TSP
100-city, uniform coordinates
29 TSPLIB instances
10
50
5
16,000
FSSP
50 jobs, 2–20 machines
Taillard benchmark
10
50
5
16,000
Table 1: Benchmark and search settings including the search and held-out instance sets, population size M , offspring generated per generation N , number of generations T , and evaluation budget B .
Framework
Benchmark
LLM
Fixed-1
Fixed-25%
Fixed-50%
Full
AdaEva-S
AdaEva-R
EoH
FSSP
Devstral
4.967
3.167
3.767
3.933
3.133
2.033
Qwen
4.767
4.033
2.667
4.267
2.867
2.400
OBP
Devstral
2.717
1.817
3.067
5.083
3.667
4.650
Qwen
1.983
4.300
3.567
3.867
3.400
3.883
TSP
Devstral
4.333
2.967
3.133
6.000
2.900
1.667
Qwen
3.933
2.767
2.933
5.833
3.367
2.167
Table 2: Held-out generalization performance of each evaluation strategy across LLM4AD frameworks, benchmarks, and LLM backbones. Mean ranks (lower is better) are computed across held-out instance group, then averaged across checkpoints. Bold and underlined denote the best and second-best methods in each row, respectively. Detailed per-checkpoint and third LLM backbone (Gemini) results are provided in Appendix C .
Figure 2: Held-out generalization trajectories of evaluation strategies across three frameworks. Mean ranks (lower is better) are computed within each held-out instance group, then averaged across the corresponding benchmarks, and LLM backbones. Each panel shows performance at increasing fractions of the search budget (20%–100%).
Figure 3: Search performance under increasing evaluation budget with the Devstral backbone, on 3 settings chosen to span the range of behaviours observed. Incumbents are selected during search from partial evaluations and then re-evaluated on the complete pool for reporting; the y-axis gives that value, as optimality gap for OBP, gap for TSP, and mean AOCC for BBOB; lower is better in the first two panels and higher is better in the third. Curves show the mean over 5 runs, with shaded regions indicating variability across seeds. The remaining 4 settings, and results with the Qwen2.5-Coder and Gemini-3.1 backbones are available in Appendix C .
Property
AdaEva-R Depth
Benchmark
Kendall’s W
Across CV
Within CV
Mean Depth
Stop at First Block
OBP
1.0000
0.3403
0.0016
7.04±0.72
92.8%±3.5%
TSP
0.0312
0.0001
0.0348
12.49±1.98
73.8%±5.6%
FSSP
0.6133
0.0088
0.1216
19.25±1.02
46.1%±4.4%
BBOB
0.6567
0.6809
1.1235
22.79±2.94
61.9%±4.4%
Table 3: Benchmark properties and the evaluation depth reached by AdaEva-R. Kendall’s W and the Across- and Within-candidate coefficients of variation (CVs) are computed from a fixed initial set of candidates evaluated on every instance. Evaluation depth is reported under EoH for the CO benchmarks and under LLaMEA for BBOB. Definitions of these metrics are provided in Section C.1 .
Strategy
Configuration
EoH/OBP
EoH/TSP
EoH/FSSP
LLaMEA/BBOB
AdaEva-S
n0=10
+1.41
−0.17
−0.02
−2.01
η=1.5
+0.25
+0.26
−0.02
−7.74
AdaEva-R
Tstats=t -test
+2.39
+0.04
−0.06
+14.93
Refill = Off
−0.39
+0.03
−0.01
+19.48
Stochastic = Off
+0.41
+0.09
+0.03
+21.16
Prompt = Partial Evaluation
–
–
–
+14.99
Table 4: Selected sensitivity results. Each value is the percent change in incumbent quality relative to the default of the same instantiation, computed within matched seeds, taking the median across seeds at each of five budget checkpoints and then averaging across checkpoints. Positive values indicate worse performance than the default. The complete sweep is reported in Appendix E .
Figure 4: Relative number of candidate algorithms evaluated during the search, under different evaluation strategies across EoH, HiFo-Prompt, and LLaMEA (using the Devstral backbone). The values are normalized by the corresponding full-evaluation baseline, so larger ratios indicate that more candidate algorithms can be sampled within the same evaluation budget; error bars denote variability across five runs.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Search performance under increasing evaluation budget with the Devstral backbone for HiFo-Prompt. Each panel compares AdaEva-S and AdaEva-R with Fixed- K evaluation at K=1 , 25% , and 50% of the pool, and with full-pool evaluation ( K=∣I∣ ). Incumbents are selected during search from partial evaluations and re-evaluated on the complete pool for reporting. The y-axis shows this full-pool performance, measured as optimality gap for OBP and TSP and makespan for FSSP, with lower values better. Curves show mean performance across five independent search seeds, with shaded regions indicating variability across seeds.
Figure 6: Search performance under increasing evaluation budget with the Qwen2.5-Coder backbone for EoH (top) and HiFo-Prompt (bottom). Each panel compares AdaEva-S and AdaEva-R with Fixed- K evaluation at K=1 , 25% , and 50% of the pool, and with full-pool evaluation ( K=∣I∣ ). Incumbents are selected during search from partial evaluations and re-evaluated on the complete pool for reporting. The y-axis shows this full-pool performance, measured as optimality gap for OBP and TSP and makespan for FSSP, with lower values better. Curves show mean performance across five independent search seeds, with shaded regions indicating variability across seeds.
Figure 7: Search performance under increasing evaluation budget with the Qwen2.5-Coder and Gemini-3.1-Flash-Lite backbones. The LLaMEA/BBOB panels and the EoH/TSP panel compare AdaEva-S and AdaEva-R with Fixed- K evaluation at K=25% and 50% of the pool, and with full-pool evaluation ( K=∣I∣ ). Incumbents are selected during search from partial evaluations and re-evaluated on the complete pool for reporting. The y-axis shows mean AOCC for BBOB and optimality gap for TSP; higher AOCC and lower gap indicate better performance. Curves show mean performance across five independent search seeds, with shaded regions indicating variability across seeds.
Search Budget Consumed
Overall
Strategy
20%
40%
60%
80%
100%
Rank ↓
Best % ↑
Top-2 % ↑
Fixed-1
3.528
3.389
3.792
4.000
4.208
3.783
15.0
25.0
Fixed-25%
2.583
3.306
3.556
3.264
3.167
3.175
11.7
35.0
Fixed-50%
3.611
3.458
2.958
2.972
2.944
3.189
10.6
27.2
Full
4.639
4.931
4.944
4.778
4.861
4.831
3.3
8.3
AdaEva-S
3.944
3.431
3.097
3.014
2.625
3.222
8.9
27.8
Appendix
Table 5: EoH anytime held-out performance, reported as mean rank (lower is better) at each search-budget checkpoint. Results are macro-averaged over the three CO benchmarks and two LLM backbones (Devstral and Qwen2.5-Coder). Bold and underline denote the best and runner-up strategies, respectively, in each column; higher values are better for the Best % and Top-2 % columns.
Search Budget Consumed
Overall
Strategy
20%
40%
60%
80%
100%
Rank ↓
Best % ↑
Top-2 % ↑
Fixed-1
4.472
4.639
4.556
4.639
4.681
4.597
6.7
12.8
Fixed-25%
2.889
2.708
2.806
3.014
2.986
2.881
16.7
32.8
Fixed-50%
3.569
3.167
3.403
3.333
3.569
3.408
6.7
18.9
Full
4.889
5.097
5.111
5.000
4.569
4.933
1.7
5.0
AdaEva-S
2.472
2.625
2.625
2.389
2.681
2.558
19.4
47.8
Appendix
Table 6: HiFo-Prompt anytime held-out performance, reported as mean rank (lower is better) at each search-budget checkpoint. Results are macro-averaged over the three CO benchmarks and two LLM backbones (Devstral and Qwen2.5-Coder). Bold and underline denote the best and runner-up strategies, respectively, in each column; higher values are better for the Best % and Top-2 % columns.
Search Budget Consumed
Overall
Strategy
20%
40%
60%
80%
100%
Rank ↓
Best % ↑
Top-2 % ↑
(1+1)-LLaMEA
3.917
5.000
4.083
4.750
4.833
4.517
0.0
3.3
(10+10)-LLaMEA (Full)
3.917
3.333
3.333
3.250
3.500
3.467
0.0
26.7
Fixed-1
7.000
7.000
7.000
6.833
6.833
6.933
0.0
0.0
Fixed-25%
4.500
3.667
4.500
4.583
4.750
4.400
0.0
13.3
Fixed-50%
5.083
5.000
5.083
4.333
4.583
4.817
3.3
10.0
Appendix
Table 7: LLaMEA anytime held-out performance, reported as mean rank (lower is better) at each search-budget checkpoint. Results are macro-averaged over the 24 BBOB functions and two LLM backbones (Devstral and Qwen2.5-Coder), with the original single-solution (1+1) -LLaMEA configuration included as an additional reference. Bold and underline denote the best and runner-up strategies, respectively, in each column; higher values are better for the Best % and Top-2 % columns.
C=100 (in-distribution)
C=500 (capacity-shifted)
Budget
Method
1k
5k
10k
1k
5k
10k
20%
Fixed-1
3.59
1.09
0.56
6.31
1.83
1.23
[2.59, 4.33]
[0.69, 1.39]
[0.39, 1.01]
[4.82, 7.05]
[1.58, 2.43]
[0.93, 1.46]
Fixed-25%
3.84
2.23
1.83
4.57
3.13
2.60
[2.34, 4.58]
[1.78, 2.63]
[1.56, 2.05]
[3.08, 5.56]
[2.68, 3.28]
[2.30, 2.93]
Fixed-50%
4.83
2.18
1.36
12.02
5.61
4.19
Appendix
Table 8: Anytime held-out performance of EoH on Online Bin Packing using the Devstral backbone. Cells report the incumbent’s objective value at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), for in-distribution capacity C=100 and capacity-shifted C=500 across three instance sizes (1k, 5k, and 10k). Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Lower values are better. (Part 1)
C=100 (in-distribution)
C=500 (capacity-shifted)
Budget
Method
1k
5k
10k
1k
5k
10k
60%
Fixed-1
4.08
1.19
0.54
12.77
3.97
2.33
[3.09, 4.83]
[0.94, 1.69]
[0.46, 1.06]
[11.77, 13.51]
[3.67, 4.67]
[2.15, 2.60]
Fixed-25%
3.34
1.09
0.59
10.53
3.42
2.10
[2.10, 3.84]
[0.64, 1.49]
[0.41, 0.88]
[9.04, 11.53]
[2.98, 3.87]
[1.78, 2.43]
Fixed-50%
4.08
1.19
0.86
9.54
3.87
2.18
Appendix
Table 9: Anytime held-out performance of EoH on Online Bin Packing using the Devstral backbone. Cells report the incumbent’s objective value at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), for in-distribution capacity C=100 and capacity-shifted C=500 across three instance sizes (1k, 5k, and 10k). Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Lower values are better. (Part 2)
≤100 (in-distribution)
101 – 150
>150 (extrapolation)
Budget
Method
12 instances
11 instances
6 instances
20%
Fixed-1
0.034
0.168
0.694
[0.008, 0.406]
[0.015, 0.370]
[0.159, 0.906]
Fixed-25%
0.028
0.318
0.521
[0.008, 0.403]
[0.015, 0.499]
[0.124, 0.988]
Fixed-50%
0.244
0.744
0.873
Appendix
Table 10: Anytime held-out performance of EoH on TSPLIB using the Devstral backbone. Cells report the incumbent’s objective value at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), for three instance-size groups: ≤100 cities (12 in-distribution instances), 101 – 150 cities (11 instances), and >150 cities (6 extrapolation instances). Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Lower values are better. (Part 1)
≤100 (in-distribution)
101 – 150
>150 (extrapolation)
Budget
Method
12 instances
11 instances
6 instances
60%
Fixed-1
0.059
0.203
0.533
[0.008, 0.403]
[0.015, 0.381]
[0.132, 0.685]
Fixed-25%
0.024
0.244
0.486
[0.004, 0.403]
[0.077, 0.526]
[0.102, 0.705]
Fixed-50%
0.012
0.085
0.399
Appendix
Table 11: Anytime held-out performance of EoH on TSPLIB using the Devstral backbone. Cells report the incumbent’s objective value at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), for three instance-size groups: ≤100 cities (12 in-distribution instances), 101 – 150 cities (11 instances), and >150 cities (6 extrapolation instances). Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Lower values are better. (Part 2)
Budget
Method
m=5 (30)
m=10 (40)
m=20 (40)
20%
Fixed-1
0.00
0.51
1.49
[0.00, 0.07]
[0.29, 0.81]
[1.01, 1.86]
Fixed-25%
0.00
0.32
1.34
[0.00, 0.00]
[0.22, 0.57]
[0.73, 1.57]
Fixed-50%
0.00
0.43
1.39
[0.00, 0.04]
[0.24, 0.66]
[0.80, 1.62]
Appendix
Table 12: Anytime held-out performance of EoH on the Taillard flow-shop scheduling benchmark using the Devstral backbone. Cells report the incumbent’s objective value at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), for three problem-size groups: m=5 (30 instances), m=10 (40 instances), and m=20 (40 instances). Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Lower values are better. (Part 1)
Budget
Method
m=5 (30)
m=10 (40)
m=20 (40)
100%
Fixed-1
0.00
0.43
1.43
[0.00, 0.07]
[0.25, 0.66]
[0.84, 1.83]
Fixed-25%
0.00
0.41
1.22
[0.00, 0.00]
[0.18, 0.58]
[0.71, 1.56]
Fixed-50%
0.00
0.32
1.23
[0.00, 0.00]
[0.21, 0.61]
[0.57, 1.60]
Appendix
Table 13: Anytime held-out performance of EoH on the Taillard flow-shop scheduling benchmark using the Devstral backbone. Cells report the incumbent’s objective value at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), for three problem-size groups: m=5 (30 instances), m=10 (40 instances), and m=20 (40 instances). Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Lower values are better. The lower block reports the percentage of instances whose normalized gap is at or below the upper bound (gap ≤0 ). (Part 2)
Budget
Method
d=5
d=10
d=20
All
20%
(1+1)-LLaMEA
0.505
0.458
0.429
0.464
[0.477, 0.517]
[0.435, 0.463]
[0.401, 0.435]
[0.438, 0.472]
(10+10)-LLaMEA
0.526
0.471
0.428
0.474
[0.517, 0.543]
[0.452, 0.474]
[0.389, 0.431]
[0.462, 0.475]
Fixed-1
0.486
0.433
0.402
0.439
[0.458, 0.521]
[0.424, 0.459]
[0.400, 0.418]
[0.428, 0.464]
Appendix
Table 14: Anytime held-out performance of LLaMEA on BBOB using the Devstral backbone. Cells report AOCC at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), evaluated on held-out instances across all 24 BBOB functions and dimensions 5 , 10 , and 20 . All denotes the mean across the three dimensions. Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Higher AOCC is better. (Part 1)
Budget
Method
d=5
d=10
d=20
All
60%
(1+1)-LLaMEA
0.513
0.463
0.430
0.469
[0.476, 0.530]
[0.436, 0.470]
[0.400, 0.438]
[0.437, 0.480]
(10+10)-LLaMEA
0.552
0.494
0.433
0.488
[0.539, 0.557]
[0.476, 0.496]
[0.408, 0.437]
[0.483, 0.501]
Fixed-1
0.472
0.426
0.395
0.430
[0.441, 0.513]
[0.409, 0.460]
[0.391, 0.417]
[0.414, 0.463]
Appendix
Table 15: Anytime held-out performance of LLaMEA on BBOB using the Devstral backbone. Cells report AOCC at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), evaluated on held-out instances across all 24 BBOB functions and dimensions 5 , 10 , and 20 . All denotes the mean across the three dimensions. Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Higher AOCC is better. (Part 2)
Budget
Method
d=5
d=10
d=20
All
20%
Fixed-1
0.499
0.425
0.365
0.435
[0.000, 0.515]
[0.000, 0.432]
[0.000, 0.385]
[0.000, 0.439]
Fixed-25%
0.545
0.474
0.431
0.477
[0.526, 0.568]
[0.453, 0.488]
[0.377, 0.434]
[0.473, 0.497]
Fixed-50%
0.544
0.451
0.404
0.473
[0.537, 0.624]
[0.451, 0.460]
[0.389, 0.415]
[0.462, 0.483]
Appendix
Table 16: Anytime held-out performance of LLaMEA on BBOB using the Qwen2.5-Coder backbone. Cells report AOCC at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), evaluated on held-out instances across all 24 BBOB functions and dimensions 5 , 10 , and 20 . All denotes the mean across the three dimensions. Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Higher AOCC is better. (Part 1)
Budget
Method
d=5
d=10
d=20
All
60%
Fixed-1
0.499
0.425
0.365
0.435
[0.000, 0.515]
[0.000, 0.432]
[0.000, 0.385]
[0.000, 0.439]
Fixed-25%
0.560
0.456
0.445
0.476
[0.433, 0.561]
[0.341, 0.496]
[0.428, 0.455]
[0.400, 0.504]
Fixed-50%
0.557
0.465
0.409
0.487
[0.419, 0.596]
[0.453, 0.485]
[0.376, 0.437]
[0.411, 0.493]
Appendix
Table 17: Anytime held-out performance of LLaMEA on BBOB using the Qwen2.5-Coder backbone. Cells report AOCC at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), evaluated on held-out instances across all 24 BBOB functions and dimensions 5 , 10 , and 20 . All denotes the mean across the three dimensions. Main entries are medians across five search seeds, with interquartile ranges shown in brackets. Higher AOCC is better. (Part 2)
≤100 (in-distribution)
101 – 150
>150 (extrapolation)
Budget
Method
12 instances
11 instances
6 instances
20%
Fixed-1
0.024
0.287
0.493
[0.004, 0.403]
[0.015, 0.404]
[0.107, 0.694]
Fixed-25%
0.020
0.037
0.338
[0.004, 0.403]
[0.002, 0.174]
[0.057, 0.635]
Fixed-50%
0.172
0.372
0.597
Appendix
Table 18: Anytime held-out performance of EoH on TSPLIB using the Gemini-3.1-Flash-Lite backbone. Cells report the median optimality gap across five search seeds at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), with the interquartile range shown beneath. Results are grouped by instance size: ≤100 (12 instances), 101 – 150 (11 instances), and >150 (6 extrapolation instances). Lower values are better, and the best median within each budget block and instance-size group is shown in bold . (Part 1)
≤100 (in-distribution)
101 – 150
>150 (extrapolation)
Budget
Method
12 instances
11 instances
6 instances
60%
Fixed-1
0.052
0.247
0.400
[0.008, 0.403]
[0.015, 0.338]
[0.101, 0.655]
Fixed-25%
0.024
0.028
0.333
[0.004, 0.403]
[0.002, 0.236]
[0.060, 0.542]
Fixed-50%
0.103
0.107
0.623
Appendix
Table 19: Anytime held-out performance of EoH on TSPLIB using the Gemini-3.1-Flash-Lite backbone. Cells report the median optimality gap across five search seeds at five search-budget checkpoints ( 20% , 40% , 60% , 80% , and 100% ), with the interquartile range shown beneath. Results are grouped by instance size: ≤100 (12 instances), 101 – 150 (11 instances), and >150 (6 extrapolation instances). Lower values are better, and the best median within each budget block and instance-size group is shown in bold . (Part 2)
Parameter
Value
EoH/OBP
EoH/TSP
EoH/FSSP
LLaMEA/BBOB
n0 (AdaEva-R)
1
−0.22
+0.02
0.00
+21.37
5
0†
0†
0†
0†
10
+0.84
+0.05
−0.03
+32.29
Δ
1
0†
0†
0†
0†
3
+1.05
+0.02
−0.01
+28.92
n0 (AdaEva-S)
1
−0.03
−0.08
+0.02
+2.83
Appendix
Table 20: Configuration sensitivity of AdaEva. Each cell is the percent change in the incumbent’s full-pool quality under the ablated configuration relative to the default within the same seed, averaged across budget checkpoints ( Δ(c) ); positive values indicate worse performance than the default. One factor is varied at a time, so each configuration differs from its default in exactly one parameter. n0 is varied separately for AdaEva-S and AdaEva-R, while η is specific to AdaEva-S and the racing parameters are specific to AdaEva-R. The prompt rows correspond to the LLaMEA feedback ablation. Defaults are marked † , and – denotes a configuration with no usable run.
The paradigm of automated program generation is shifting from one-shot generation to inference-time search, where Large Language Models (LLMs) function as semantic mutation operators within evolutionary loops. While effective, these systems are currently governed by static schedules that fail to account for the non-stationary dynamics of the search process. This rigidity results in substantial computational waste, as resources are indiscriminately allocated to stagnating populations while promising frontiers remain under-exploited. We introduce AdaEvolve, a framework that reformulates LLM-driven evolution as a hierarchical adaptive optimization problem. AdaEvolve uses an "accumulated improvement signal" to unify decisions across three levels: Local Adaptation, which dynamically modulates the exploration intensity within a population of solution candidates; Global Adaptation, which routes the global resource budget via bandit-based scheduling across different solution candidate populations; and Meta-Guidance which generates novel solution tactics based on the previously generated solutions and their corresponding improvements when the progress stalls. We demonstrate that AdaEvolve consistently outperforms the open-sourced baselines across 185 different open-ended optimization problems including combinatorial, systems optimization and algorithm design problems.
Large language models (LLMs) enable automated algorithm design (AAD) through reasoning and code synthesis. However, most existing LLM-based AAD methods separate algorithm design from target optimization, deploying a fixed design even as the optimization state evolves. Conventional adaptive optimizers can respond to such changes, but their adjustments remain confined to predefined parameters, operators, or strategies. To address these limitations, we introduce online LLM-based AAD, a novel optimization paradigm that treats the algorithm itself as a state-dependent decision variable. At each stage, LLM agents synthesize an algorithm with new behavior logic from the current optimization state. Executing the generated algorithm advances the search and provides feedback for subsequent designs, coupling algorithm design with target optimization without requiring a separate offline algorithm pretraining stage. To implement this paradigm, we propose OnDesign, a multi-agent framework that reconciles competing design perspectives to synthesize executable algorithms and uses execution feedback to refine how runtime evidence is interpreted for subsequent designs. We evaluate OnDesign across two mainstream black-box optimization paradigms on three scenarios: Bayesian optimization, evolutionary continuous optimization, and evolutionary mixed-variable optimization. Extensive experiments on six benchmark suites and one engineering problem across multiple problem dimensions demonstrate superior overall performance over conventional optimizers and offline LLM-based AAD methods.
Zhiyao Zhang, Yichen Li, Xingyu Wu +2
Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University, Hong Kong SAR 999077, China · College of Computer Science, Chongqing University, Chongqing 400044, China
Large Language Model (LLM)-based automated algorithm design typically evolves algorithms as complete, indivisible programs. While this whole-program perspective simplifies the search space, it fundamentally couples the useful local logic to its host program. Consequently, valuable code snippets vanish when the overall program is discarded, making it highly difficult to assess the contribution of individual algorithmic components.To address this, we propose Primitive-Aware Code Evolution (PACE), which decouples local logic from complete programs by representing it as persistent units called Executable Algorithmic Primitives (EAPs). To enable code-level transfer, PACE maintains a dynamic set of EAPs. Algorithm evolution is driven by primitive-aware operators that structurally guarantee the retention and cross-program transfer of these components. To evaluate them effectively, PACE leverages Thompson sampling based on parent-relative performance improvements, guiding primitive selection from the set without requiring extra evaluation datasets. Experiments on four tasks demonstrate that PACE effectively discovers competitive algorithms while structurally preserving valuable algorithmic components.
Zhuoliang Xie, Ruihao Zheng, Xiang Xu +2
1Southern University of Science and Technology · 2Shenzhen University