Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace, can take hours, so a search can afford only a handful of true evaluations. We present DuoDiscover, a search system that jointly discovers what to optimize and how to evaluate progress efficiently, through three mechanisms. First, each worker in a fleet of coding agents co-discovers an optimization target and a fast, tailored proxy evaluator, which we call a yardstick. The agent then hill-climbs the yardstick rather than the true evaluation. Second, since search can exploit gaps between a yardstick and the true objective, fidelity tests continually recheck each yardstick's ranking and invalidate any that drift. Third, the fleet is bottom-up, with no central decomposition of the problem. A lightweight orchestrator spawns and terminates agents, briefs running ones, and seeds new agents with others' confirmed work. DuoDiscover enables autonomous optimization from software to chip architecture. We demonstrate DuoDiscover on two repository-level benchmarks and four GPU and TPU kernel suites covering hundreds of kernels. On SWE-fficiency, it doubles the average speedup of human experts. On SOL-Exec, it ranks first on 121 of 235 kernels and in the top three on 212. On the heavily optimized vLLM and SQLite engines, DuoDiscover achieves 1.92× and 1.38× gains over the base code. On a CPU microarchitecture optimization task with evaluations of up to 28 hours, DuoDiscover surpasses both AlphaEvolve-based ArchAgent v2 and the best human design, with 1.65× ArchAgent v2's gain in 16.7 days versus more than 40 days, advancing the frontier of autonomous chip design.
Figures & tables
Figure 2: Overview of DuoDiscover. Left: a fleet of worker agents, each on its own copy of the codebase, coordinated by an orchestrator that terminates agents (a), spawns new ones, here one combining A and C (b), and briefs running ones (c). Right: inside one agent, a fast loop co-discovers a target and yardstick and climbs it on cheap scores, while a slow loop confirms gains with true evaluations and runs the fidelity test that validates or invalidates the yardstick.
Repositories
GPU Kernels
TPU Kernels
Method
FormulaCode
SWE-fficiency
SOL-Exec
FA4
JAXBench
TPU-Inference
( × )
( × )
(% SOL)
(%)
( × )
( × )
Claude Code
1.778
12.81
67.31
7.05
3.82
2.38
OpenEvolve
1.161
3.76
52.36
1.48
2.66
1.14
PerfAgent
1.774
12.27
64.10
6.86
4.08
2.14
KDA
–
–
70.22
8.93
–
–
Table 1: Benchmark results under an 8-hour, four-agent budget (24 hours on the long FormulaCode tasks, 10 hours on TPU-Inference), higher is better. Numbers are speedups over the base code, except FA4 (% over Flash Attention 4), JAXBench (over XLA), TPU-Inference (over tuned Pallas kernels), and SOL-Exec, the mean % of speed-of-light over 235 leaderboard kernels. Dashes mark benchmarks a method does not target.
Figure 3: Gain over the base code across search time. Thick lines: geometric mean over the objectives (DuoDiscover blue, Claude Code red). Dashed lines: individual DuoDiscover objectives.
FullBW
LimitBW
Multi-Core
Overall
BertiGO
+5.16%
+2.57%
+2.85%
+3.52%
ArchAgent v2
+5.61%
+4.56%
+1.33%
+3.82%
DuoDiscover
+9.08%
+5.93%
+3.93%
+6.31%
Table 2: Official DPC4 scores, geometric-mean performance gain over the reference core (610 traces for FullBW and LimitBW, 484 mixes for Multi-Core). BertiGO (the best human design) and ArchAgent v2 are as reported in the ArchAgent v2 paper ( Gonzalez et al., 2026 ) .
True evaluation
Yardstick
Faster by
Validated
Invalidated
τ
SWE-fficiency tasks
15 min
40 s
22 ×
5.3 of 9.2
41%
0.66
SOL-Exec kernels
14 min
16 s
53 ×
11 of 15
45%
0.76
Table 3: Yardstick speed and fidelity across SWE-fficiency tasks and SOL-Exec kernels. Here Validated : targets whose yardstick passes the fidelity test. Invalidated : percentage invalidated later for drift. τ : median rank correlation with the true evaluation.
Figure 4: Best-so-far SOL (mean) over 50 kernels vs. (a) time, (b) tokens, and (c) true evaluations. DuoDiscover attains Claude Code’s score (69.12%) in 2.8 hours, 0.93M tokens and 13 true evaluations.
Table 7
SWE-fficiency
SOL-Exec
( × )
(% SOL)
Claude Code/Opus 5
12.85
69.12
+DuoDiscover
14.93
80.80
Antigravity/Gemini 3.8 Flash
9.72
58.79
+DuoDiscover
12.66
73.26
Table 6: Generalization across coding-agent backbones under the 8-hour, 4-agent budget.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Meaning
Defined in
x , X
a program state (a version of the code); the set of all states
§ 2.1
x0
the initial program
§ 2.1
f
the true evaluation, f:X→R∪{⊥} , higher is better; ⊥ means the state fails the task’s correctness check
§ 2.1
cf
the cost of one call to f , measured as its running time
§ 2.1
N
the number of agents in a run
§ 2.1
T , XT
a target (a region of the code); the states reachable by editing only inside T
§ 2.2
Appendix
Table 7: Notation of § 2 , in the order it is introduced.
Benchmark
Task
Score
Compares against
n
Budget
Eval time
FormulaCode
repo runtime
speedup ( × )
base code
25
8 h / 24 h
7 min - 6.8 h
SWE-fficiency
repo runtime
speedup ( × )
base code
100
8 h
15 min (3 to 93)
SOL-Exec
GPU kernel
% of SOL
hardware peak
235
8 h
14 min (2 to 60)
JAXBench
kernel
speedup ( × )
XLA baseline
17
8 h
17 min (7 to 32)
TPU-Inference
kernel
speedup ( × )
optimized Pallas kernel
7
10 h
3 min (1 to 8)
FA4
kernel
% over FA4
Flash Attention 4
1
8 h
4 min
Appendix
Table 8: The benchmarks, with their task type, score, comparison baseline, task count n , budget, and true-evaluation time. On each benchmark, every method we run shares the same hardware, budget, agent count (four), and concurrency limit (four concurrent true evaluations on the benchmarks, one on the case studies). A true evaluation is one sequential run of the full workload, so adding machines cannot shorten it. The data-prefetcher study is the exception. It uses three agents, runs its traces in parallel, and takes its baselines as reported rather than rerun.
Baseline
Benchmarks
Setup
Claude Code
all
Four independent agents on Opus 5, same prompt as DuoDiscover, each on its own copy of the code. Every candidate scored by a true evaluation. Best of four reported.
OpenEvolve
all
On FormulaCode and SWE-fficiency we run OpenEvolve-Agentic, with the mutation operator replaced by a coding agent so it can edit whole codebases. On the kernel suites, unmodified OpenEvolve. Both run on Opus 5 under the same budget and true-evaluation setup as the other baselines. True evaluation as fitness, best of four independent runs reported.
PerfAgent
all
Our reimplementation on Opus 5, four agents. Each profiles with the task’s profiler, edits the hottest region, re-profiles. Every candidate scored by a true evaluation.
KDA
SOL-Exec, FA4
Run by us, 8-hour budget, same hardware. Same model and agents as DuoDiscover, run as KDA kernel agents.
AVO
FA4
As reported in the AVO paper.
cuDNN
FA4
NVIDIA’s attention kernels, no search. As reported in the AVO paper.
Appendix
Table 9: Summary of baseline methods, benchmark coverage, and execution configurations.
Task
OpenEvolve
Claude Code
PerfAgent
DuoDiscover
TileDB-Py_2206
1.175
1.627
1.461
1.730
uxarray_1249
1.116
3.207
4.093
6.076
networkx_6879
1.430
3.349
2.310
3.554
networkx_7181
1.056
1.652
1.566
2.627
networkx_7249
1.420
2.137
1.984
3.666
networkx_7595
1.015
1.607
2.237
2.616
Appendix
Table 10: Speedup comparison across 25 tasks for FormulaCode. Entries report geometric mean speedups (higher is better). The best result in each row is bold. Mean Speedup is the arithmetic mean of the task scores, and Wins counts first-place finishes.
Figure 5: Per-kernel finishing position of the top five SOL-Exec entries, with mean SOL at the bar.
Method
Speedup over FA4 (%)
FA4
0.00
cuDNN
1.30
AVO
4.11
KDA
8.93
Claude Code
7.05
PerfAgent
6.86
Appendix
Table 11: Geometric-mean speedup over Flash Attention 4 across the 24 FA4 configurations. cuDNN and AVO are as reported in the AVO paper ( Chen et al., 2026a ) . The rest are our runs.
Figure 6: Speedup over Flash Attention 4 on each of the 24 FA4 configurations.
Figure 7: Speedup over the optimized production Pallas reference (horizontal line at 1.00 × ) on every evaluated configuration of each TPU-Inference kernel, for DuoDiscover, Claude Code, PerfAgent, and OpenEvolve.
Figure 8: Gain over the base code as the search runs. Thick solid lines are the geometric mean over the objectives at each true evaluation, DuoDiscover in blue and Claude Code in red. The main text shows the same curves without event markers (Figure 3 ). Thin dashed lines are the individual objectives behind that mean. Markers on the thick line are the search events in the legend. For Claude Code only confirmed gains are marked.
Measure
Tuned unmodified vLLM
DuoDiscover
Change
p99 decode latency (inter-token latency), real workload
141 to 337 ms
22 to 26 ms
6 to 13 × lower
GPU power at equal throughput
4594 to 4968 W
3038 to 3237 W
34% lower
Tokens per joule, fixed low clock
baseline
1.314 × baseline
31.4% higher
Light-client first-token latency under a heavy client (vs. first come first served)
9654 ms
1114 ms
8.7 × lower
p99 decode latency under 8 concurrent long prefills
221.6 to 507.7 ms (baseline cap)
24.3 ms
9 to 21 × lower
Peak throughput
baseline
unchanged
within 1%
Appendix
Table 12: Each mechanism of the vLLM solution measured after the run on 8 × B200 serving GLM-5.2-NVFP4, under the fixed-rate conditions given in the text. The first-token row is against first-come-first-served scheduling, the throughput row is the full solution.
Model
Batch
Unmodified
DuoDiscover
Speedup
GLM-5.2-NVFP4 (753B, 40B active)
1
6.75
7.9
1.17 ×
GLM-5.2-NVFP4 (753B, 40B active)
8
10.10
17.8
1.76 ×
Appendix
Table 13: FreeToken on one L40S, decode tokens per second against the unmodified build, measured by the true evaluation, three runs each.
Figure 9: Gain over the base code as the search runs, for SQLite and zstd, each a single objective, the zstd axis an instruction-count ratio. Lines and markers as in Figure 8 .
Figure 21
Figure 11: SOL score against agent count N , 8-hour runs on 50 SOL-Exec kernels.
Figure 12: Yardstick quality by kind on 15 of the 50 SOL-Exec kernels. (a) Yardstick time against true-evaluation time, one point per target, dashed lines at 1, 10 and 100 × faster. (b) Rank correlation between yardstick and true evaluation, one point per truly evaluated candidate, its rank by the yardstick against its rank by the true evaluation, pooled over the candidates of each kind, with Kendall τ per kind.
SOL-Exec kernels
SWE-fficiency tasks
Yardstick kind
Scores a candidate on
Share
Faster by
Validated
Share
Faster by
Validated
Reduced inputs
smaller or low-precision inputs
33%
63 ×
75%
42%
20 ×
67%
Profiling harness
an isolated hot loop
42%
28 ×
81%
29%
11 ×
58%
Filtered test subsuite
tests on the changed code
14%
78 ×
69%
24%
31 ×
51%
Static cost model
instruction or memory counts
11%
89 ×
32%
5%
52 ×
18%
Appendix
Table 15: The four kinds of yardstick agents build on the 50 SOL-Exec kernels and the 20 SWE-fficiency tasks. Weighted by share, the kernel columns give the 53 × and 11 of 15 of Table 3 , the repository columns the 22 × and 5.3 of 9.2.
SWE-fficiency (20 tasks)
SOL-Exec (50 kernels)
Claude Code
DuoDiscover
ratio
Claude Code
DuoDiscover
ratio
Targets pursued
2 to 4
9.2
–
3 to 7
15
–
Yardsticks validated
–
5.3
–
–
11
–
Time per candidate check
15 min
40 s
22 ×↑
14 min
16 s
53 ×↑
Candidates scored
72
358
5.0 ×↑
76
285
3.8 ×↑
True evaluations
72
59
0.8 ×↓
76
54
0.7 ×↓
Appendix
Table 16: Search per task, means over 20 SWE-fficiency tasks and 50 SOL-Exec kernels, times are medians. Ratio is DuoDiscover over Claude Code, arrow up for more or faster, down for fewer true evaluations. Confirmed gains include the combined edits. Gains per true evaluation counts only edits found on a yardstick.
DuoDiscover
Claude Code
Token class
$ per M tokens
tokens
cost
tokens
cost
generated
25
4.4M
$110
3.6M
$90
cache writes
10
13.6M
$136
11.2M
$112
cache reads
0.50
258.7M
$129
213.2M
$107
total
276.7M
$375
228.0M
$309
Appendix
Table 17: Tokens and API cost of one 8-hour, four-agent run on one SOL-Exec kernel, mean over 50 kernels, at Opus 5 list prices with cache writes billed at the one-hour rate. GPU time for true evaluations is not included.
Condition
What happens
Confirmed score
No reliable yardstick
search runs on true evaluations alone until the orchestrator moves the agent on
equals Claude Code’s search, the method’s floor
Unreliable yardstick
rejected before validation, 5 to 9% of true evaluations spent
never below the baseline, search resumes on a rebuilt yardstick
Noisy true evaluation
the noise is measured and folded into the test
no false gain from timing jitter
α from 0.01 to 0.10
the fidelity test is stricter or looser
within a 1.4% band, lowest at 0.01
Three seeds, 50 kernels
each kernel rerun three times
within ± 1.12% of the mean, 78% of gains shared, Claude Code ± 3.27%
Three seeds, 20 repository tasks
each task rerun three times
within ± 1.8% of the mean, 81% of gains shared, Claude Code ± 5.21%
Appendix
Table 18: DuoDiscover under missing, unreliable or noisy yardsticks, and its score sensitivity to α and seed, on the 50 SOL-Exec kernels and 20 SWE-fficiency tasks.
Autonomous research agents use language models and tools to search literature, design and run experiments, analyze results, and support scientific discovery.
Frameworks that use large language models for scientific discovery typically rely on a fixed, human-designed algorithm that decides what the model sees at each step, leaving the model only the role of proposer. The model knows nothing of the search beyond what it is shown. As models grow more capable, a question arises: does a search strategy chosen by a human before the run scale better than promoting the model from proposer to planner and letting it own the search? The Bitter Lesson suggests that choosing the strategy in advance is the kind of hand-designed structure that general methods eventually outscale. We introduce AgentDiscover, in which a coding agent plans the search using its context as working memory, runs experiments, and records every attempt in a database of ideas, candidates, and their relations. This database serves as the agent's long-term memory and is structured so that the selection rules of classical algorithms such as MAP-Elites and Monte Carlo tree search each reduce to a single query, which the agent is free to use, combine, or replace. A server maintains the database and steers the agent after every submission, keeping it on course over long runs. In our experiments, AgentDiscover is more cost-efficient than existing frameworks, reaching better scores at lower cost. On tasks in kernel engineering, biology, algorithm design, and mathematics, AgentDiscover outperforms prior discovery frameworks. Its programs would have placed first among human competitors in seven past AtCoder heuristic contests, and on eleven mathematical and systems optimization tasks it matches or exceeds every baseline that uses the same model. Our code is available at https://github.com/mhdfb/AgentDiscover.
Long-running coding agents such as autoresearch can persistently discover optimizations for open-ended problems. However, they tend to converge onto a single high-level approach, then proceed with low-level edits while missing other superior approaches to the problem. We hypothesize two harness-level design choices contribute to this behavior: accumulating context in a single long-running agent and only exposing a single program state to edit. We introduce SwarmResearch, an orchestrator-subagent harness in which a Shepherd Agent uses global context to steer a population of Search Agents, each operating with local context in their respective git branch. On open-ended optimization tasks, SwarmResearch discovers better or comparable solutions to state-of-the-art LLM-guided evolution and multi-agent techniques on 13/15 tasks, driven by higher-level exploration. Compared with fixed scaling of serial and parallel agents, SwarmResearch's orchestrator-guided scaling discovers better-performing solutions by adapting parallelism at different search depths.
Large language models have advanced automated algorithm discovery by synthesizing executable code, but existing frameworks trap them in rigid search pipelines with pre-defined control flows. This limitation restricts adaptive reasoning, blocks cross-paradigm transfer, and overlooks richer execution feedback. To bridge this gap, we introduce an end-to-end framework, AlgoEvo, a unified agentic architecture that transforms automated algorithm discovery into an interactive, knowledge-accumulating process. An autonomous agent dynamically inspects, diagnoses, and edits code based on runtime feedback. A design skill hub decouples paradigm-specific knowledge from the core discovery engine, allowing a unified workflow to seamlessly handle single-heuristic, multi-objective, and multi-component design. Meanwhile, a hierarchical experience bank organizes search trajectories into a task-level tree to guide exploration and consolidates cross-task patterns into reusable skills. Across six representative benchmark tasks, AlgoEvo reaches state-of-the-art performance with as little as 7% of the evaluation budget and reduced token consumption, demonstrating strong intra-task accumulation, cross-task transfer, and the ability to reproduce or exceed the strongest existing methods through flexible skill activation.
Junhao Qiu, Qinglong Hu, Ji Cheng +3
Department of Computer Science, City University of Hong Kong · Huawei Noah’s Ark Lab · Institute of Advanced Intelligence and Computing, A*STAR