DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck

Shubham Agarwal; Akshat Gupta; Mike Heddes; Dilara Soylu; Rui Meng; Patrick Tser Jern Kon; Jiefeng Chen; Mert Cemri

cs.CLOct 12, 2026

DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck

Authors: Shubham Agarwal, Akshat Gupta, Mike Heddes, Dilara Soylu, Rui Meng, Patrick Tser Jern Kon, Jiefeng Chen, Mert Cemri,

Organizations: Google · UC Berkeley · Stanford University

Abstract

Evaluation has been central to advances in AI, and it is equally critical to autonomous discovery. Yet scaling autonomous discovery to real-world engineering challenges faces a fundamental bottleneck: the cost of evaluating progress. In real settings, a single true evaluation, such as a full benchmark or serving trace, can take hours, so a search can afford only a handful of true evaluations. We present DuoDiscover, a search system that jointly discovers what to optimize and how to evaluate progress efficiently, through three mechanisms. First, each worker in a fleet of coding agents co-discovers an optimization target and a fast, tailored proxy evaluator, which we call a yardstick. The agent then hill-climbs the yardstick rather than the true evaluation. Second, since search can exploit gaps between a yardstick and the true objective, fidelity tests continually recheck each yardstick's ranking and invalidate any that drift. Third, the fleet is bottom-up, with no central decomposition of the problem. A lightweight orchestrator spawns and terminates agents, briefs running ones, and seeds new agents with others' confirmed work. DuoDiscover enables autonomous optimization from software to chip architecture. We demonstrate DuoDiscover on two repository-level benchmarks and four GPU and TPU kernel suites covering hundreds of kernels. On SWE-fficiency, it doubles the average speedup of human experts. On SOL-Exec, it ranks first on 121 of 235 kernels and in the top three on 212. On the heavily optimized vLLM and SQLite engines, DuoDiscover achieves 1.92 $\times$ and 1.38 $\times$ gains over the base code. On a CPU microarchitecture optimization task with evaluations of up to 28 hours, DuoDiscover surpasses both AlphaEvolve-based ArchAgent v2 and the best human design, with 1.65 $\times$ ArchAgent v2's gain in 16.7 days versus more than 40 days, advancing the frontier of autonomous chip design.

Figures & tables

Figure 2: Overview of DuoDiscover. Left: a fleet of worker agents, each on its own copy of the codebase, coordinated by an orchestrator that terminates agents (a), spawns new ones, here one combining A and C (b), and briefs running ones (c). Right: inside one agent, a fast loop co-discovers a target and yardstick and climbs it on cheap scores, while a slow loop confirms gains with true evaluations and runs the fidelity test that validates or invalidates the yardstick.

	Repositories		GPU Kernels		TPU Kernels
Method	FormulaCode	SWE-fficiency	SOL-Exec	FA4	JAXBench	TPU-Inference
	( $\times$ )	( $\times$ )	(% SOL)	(%)	( $\times$ )	( $\times$ )
Claude Code	1.778	12.81	67.31	7.05	3.82	2.38
OpenEvolve	1.161	3.76	52.36	1.48	2.66	1.14
PerfAgent	1.774	12.27	64.10	6.86	4.08	2.14
KDA	–	–	70.22	8.93	–	–

Table 1: Benchmark results under an 8-hour, four-agent budget (24 hours on the long FormulaCode tasks, 10 hours on TPU-Inference), higher is better. Numbers are speedups over the base code, except FA4 (% over Flash Attention 4), JAXBench (over XLA), TPU-Inference (over tuned Pallas kernels), and SOL-Exec, the mean % of speed-of-light over 235 leaderboard kernels. Dashes mark benchmarks a method does not target.

Figure 3: Gain over the base code across search time. Thick lines: geometric mean over the objectives (DuoDiscover blue, Claude Code red). Dashed lines: individual DuoDiscover objectives.

	FullBW	LimitBW	Multi-Core	Overall
BertiGO	+5.16%	+2.57%	+2.85%	+3.52%
ArchAgent v2	+5.61%	+4.56%	+1.33%	+3.82%
DuoDiscover	+9.08%	+5.93%	+3.93%	+6.31%

Table 2: Official DPC4 scores, geometric-mean performance gain over the reference core (610 traces for FullBW and LimitBW, 484 mixes for Multi-Core). BertiGO (the best human design) and ArchAgent v2 are as reported in the ArchAgent v2 paper ( Gonzalez et al., 2026 ) .

	True evaluation	Yardstick	Faster by	Validated	Invalidated	$\tau$
SWE-fficiency tasks	15 min	40 s	22 $\times$	5.3 of 9.2	41%	0.66
SOL-Exec kernels	14 min	16 s	53 $\times$	11 of 15	45%	0.76

Table 3: Yardstick speed and fidelity across SWE-fficiency tasks and SOL-Exec kernels. Here Validated : targets whose yardstick passes the fidelity test. Invalidated : percentage invalidated later for drift.

\tau

: median rank correlation with the true evaluation.

Figure 4: Best-so-far SOL (mean) over 50 kernels vs. (a) time, (b) tokens, and (c) true evaluations. DuoDiscover attains Claude Code’s score (69.12%) in 2.8 hours, 0.93M tokens and 13 true evaluations.

Table 7

	SWE-fficiency	SOL-Exec
	( $\times$ )	(% SOL)
Claude Code/Opus 5	12.85	69.12
+DuoDiscover	14.93	80.80
Antigravity/Gemini 3.8 Flash	9.72	58.79
+DuoDiscover	12.66	73.26

Table 6: Generalization across coding-agent backbones under the 8-hour, 4-agent budget.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning	Defined in
$x$ , $\mathcal{X}$	a program state (a version of the code); the set of all states	§ 2.1
$x_{0}$	the initial program	§ 2.1
$f$	the true evaluation, $f:\mathcal{X}\to\mathbb{R}\cup\{\bot\}$ , higher is better; $\bot$ means the state fails the task’s correctness check	§ 2.1
$c_{f}$	the cost of one call to $f$ , measured as its running time	§ 2.1
$N$	the number of agents in a run	§ 2.1
$T$ , $\mathcal{X}_{T}$	a target (a region of the code); the states reachable by editing only inside $T$	§ 2.2

Appendix

Table 7: Notation of § 2 , in the order it is introduced.

Benchmark	Task	Score	Compares against	$n$	Budget	Eval time
FormulaCode	repo runtime	speedup ( $\times$ )	base code	25	8 h / 24 h	7 min - 6.8 h
SWE-fficiency	repo runtime	speedup ( $\times$ )	base code	100	8 h	15 min (3 to 93)
SOL-Exec	GPU kernel	% of SOL	hardware peak	235	8 h	14 min (2 to 60)
JAXBench	kernel	speedup ( $\times$ )	XLA baseline	17	8 h	17 min (7 to 32)
TPU-Inference	kernel	speedup ( $\times$ )	optimized Pallas kernel	7	10 h	3 min (1 to 8)
FA4	kernel	% over FA4	Flash Attention 4	1	8 h	4 min

Appendix

Table 8: The benchmarks, with their task type, score, comparison baseline, task count

n

, budget, and true-evaluation time. On each benchmark, every method we run shares the same hardware, budget, agent count (four), and concurrency limit (four concurrent true evaluations on the benchmarks, one on the case studies). A true evaluation is one sequential run of the full workload, so adding machines cannot shorten it. The data-prefetcher study is the exception. It uses three agents, runs its traces in parallel, and takes its baselines as reported rather than rerun.

Baseline	Benchmarks	Setup
Claude Code	all	Four independent agents on Opus 5, same prompt as DuoDiscover, each on its own copy of the code. Every candidate scored by a true evaluation. Best of four reported.
OpenEvolve	all	On FormulaCode and SWE-fficiency we run OpenEvolve-Agentic, with the mutation operator replaced by a coding agent so it can edit whole codebases. On the kernel suites, unmodified OpenEvolve. Both run on Opus 5 under the same budget and true-evaluation setup as the other baselines. True evaluation as fitness, best of four independent runs reported.
PerfAgent	all	Our reimplementation on Opus 5, four agents. Each profiles with the task’s profiler, edits the hottest region, re-profiles. Every candidate scored by a true evaluation.
KDA	SOL-Exec, FA4	Run by us, 8-hour budget, same hardware. Same model and agents as DuoDiscover, run as KDA kernel agents.
AVO	FA4	As reported in the AVO paper.
cuDNN	FA4	NVIDIA’s attention kernels, no search. As reported in the AVO paper.

Appendix

Table 9: Summary of baseline methods, benchmark coverage, and execution configurations.

Task	OpenEvolve	Claude Code	PerfAgent	DuoDiscover
TileDB-Py_2206	1.175	1.627	1.461	1.730
uxarray_1249	1.116	3.207	4.093	6.076
networkx_6879	1.430	3.349	2.310	3.554
networkx_7181	1.056	1.652	1.566	2.627
networkx_7249	1.420	2.137	1.984	3.666
networkx_7595	1.015	1.607	2.237	2.616

Appendix

Table 10: Speedup comparison across 25 tasks for FormulaCode. Entries report geometric mean speedups (higher is better). The best result in each row is bold. Mean Speedup is the arithmetic mean of the task scores, and Wins counts first-place finishes.

Figure 5: Per-kernel finishing position of the top five SOL-Exec entries, with mean SOL at the bar.

Method	Speedup over FA4 (%)
FA4	0.00
cuDNN	1.30
AVO	4.11
KDA	8.93
Claude Code	7.05
PerfAgent	6.86

Appendix

Table 11: Geometric-mean speedup over Flash Attention 4 across the 24 FA4 configurations. cuDNN and AVO are as reported in the AVO paper ( Chen et al., 2026a ) . The rest are our runs.

Figure 6: Speedup over Flash Attention 4 on each of the 24 FA4 configurations.

Figure 7: Speedup over the optimized production Pallas reference (horizontal line at 1.00

\times

) on every evaluated configuration of each TPU-Inference kernel, for DuoDiscover, Claude Code, PerfAgent, and OpenEvolve.

Panel of Figure 8 — Figure 8: Gain over the base code as the search runs. Thick solid lines are the geometric mean over the objectives at each true evaluation, DuoDiscover in blue and Claude Code in red. The main text shows the same curves without event markers (Figure 3 ). Thin dashed lines are the individual objectives behind that mean. Markers on the thick line are the search events in the legend. For Claude Code only confirmed gains are marked.

Measure	Tuned unmodified vLLM	DuoDiscover	Change
p99 decode latency (inter-token latency), real workload	141 to 337 ms	22 to 26 ms	6 to 13 $\times$ lower
GPU power at equal throughput	4594 to 4968 W	3038 to 3237 W	34% lower
Tokens per joule, fixed low clock	baseline	1.314 $\times$ baseline	31.4% higher
Light-client first-token latency under a heavy client (vs. first come first served)	9654 ms	1114 ms	8.7 $\times$ lower
p99 decode latency under 8 concurrent long prefills	221.6 to 507.7 ms (baseline cap)	24.3 ms	9 to 21 $\times$ lower
Peak throughput	baseline	unchanged	within 1%

Appendix

Table 12: Each mechanism of the vLLM solution measured after the run on 8

\times

B200 serving GLM-5.2-NVFP4, under the fixed-rate conditions given in the text. The first-token row is against first-come-first-served scheduling, the throughput row is the full solution.

Model	Batch	Unmodified	DuoDiscover	Speedup
GLM-5.2-NVFP4 (753B, 40B active)	1	6.75	7.9	1.17 $\times$
GLM-5.2-NVFP4 (753B, 40B active)	8	10.10	17.8	1.76 $\times$

Appendix

Table 13: FreeToken on one L40S, decode tokens per second against the unmodified build, measured by the true evaluation, three runs each.

Panel of Figure 9 — Figure 9: Gain over the base code as the search runs, for SQLite and zstd, each a single objective, the zstd axis an instruction-count ratio. Lines and markers as in Figure 8 .

Figure 11: SOL score against agent count

N

, 8-hour runs on 50 SOL-Exec kernels.

Figure 12: Yardstick quality by kind on 15 of the 50 SOL-Exec kernels. (a) Yardstick time against true-evaluation time, one point per target, dashed lines at 1, 10 and 100 $\times$ faster. (b) Rank correlation between yardstick and true evaluation, one point per truly evaluated candidate, its rank by the yardstick against its rank by the true evaluation, pooled over the candidates of each kind, with Kendall $\tau$ per kind.

		SOL-Exec kernels			SWE-fficiency tasks
Yardstick kind	Scores a candidate on	Share	Faster by	Validated	Share	Faster by	Validated
Reduced inputs	smaller or low-precision inputs	33%	63 $\times$	75%	42%	20 $\times$	67%
Profiling harness	an isolated hot loop	42%	28 $\times$	81%	29%	11 $\times$	58%
Filtered test subsuite	tests on the changed code	14%	78 $\times$	69%	24%	31 $\times$	51%
Static cost model	instruction or memory counts	11%	89 $\times$	32%	5%	52 $\times$	18%

Appendix

Table 15: The four kinds of yardstick agents build on the 50 SOL-Exec kernels and the 20 SWE-fficiency tasks. Weighted by share, the kernel columns give the 53

\times

and 11 of 15 of Table 3 , the repository columns the 22

\times

and 5.3 of 9.2.

	SWE-fficiency (20 tasks)			SOL-Exec (50 kernels)
	Claude Code	DuoDiscover	ratio	Claude Code	DuoDiscover	ratio
Targets pursued	2 to 4	9.2	–	3 to 7	15	–
Yardsticks validated	–	5.3	–	–	11	–
Time per candidate check	15 min	40 s	22 $\times\,\uparrow$	14 min	16 s	53 $\times\,\uparrow$
Candidates scored	72	358	5.0 $\times\,\uparrow$	76	285	3.8 $\times\,\uparrow$
True evaluations	72	59	0.8 $\times\,\downarrow$	76	54	0.7 $\times\,\downarrow$

Appendix

Table 16: Search per task, means over 20 SWE-fficiency tasks and 50 SOL-Exec kernels, times are medians. Ratio is DuoDiscover over Claude Code, arrow up for more or faster, down for fewer true evaluations. Confirmed gains include the combined edits. Gains per true evaluation counts only edits found on a yardstick.

		DuoDiscover		Claude Code
Token class	$ per M tokens	tokens	cost	tokens	cost
generated	25	4.4M	$110	3.6M	$90
cache writes	10	13.6M	$136	11.2M	$112
cache reads	0.50	258.7M	$129	213.2M	$107
total		276.7M	$375	228.0M	$309

Appendix

Table 17: Tokens and API cost of one 8-hour, four-agent run on one SOL-Exec kernel, mean over 50 kernels, at Opus 5 list prices with cache writes billed at the one-hour rate. GPU time for true evaluations is not included.

Condition	What happens	Confirmed score
No reliable yardstick	search runs on true evaluations alone until the orchestrator moves the agent on	equals Claude Code’s search, the method’s floor
Unreliable yardstick	rejected before validation, 5 to 9% of true evaluations spent	never below the baseline, search resumes on a rebuilt yardstick
Noisy true evaluation	the noise is measured and folded into the test	no false gain from timing jitter
$\alpha$ from 0.01 to 0.10	the fidelity test is stricter or looser	within a 1.4% band, lowest at 0.01
Three seeds, 50 kernels	each kernel rerun three times	within $\pm$ 1.12% of the mean, 78% of gains shared, Claude Code $\pm$ 3.27%
Three seeds, 20 repository tasks	each task rerun three times	within $\pm$ 1.8% of the mean, 81% of gains shared, Claude Code $\pm$ 5.21%

Appendix

Table 18: DuoDiscover under missing, unreliable or noisy yardsticks, and its score sensitivity to

\alpha

and seed, on the 50 SOL-Exec kernels and 20 SWE-fficiency tasks.

Autonomous Research Agents

Autonomous research agents use language models and tools to search literature, design and run experiments, analyze results, and support scientific discovery.

See all 318 papers

AgentDiscover: Autonomous Discovery with Minimal Search Scaffolding
Oct 4, 2026Mahdi Farahbakhsh, Ilan Sela, Fatemeh Doudi +4AI Agents for Scientific Discovery Automated Algorithm Discovery
Texas A&M University
SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery
Jul 2, 2026Yuvraj Virk, Zack Edds, Chunqiu Steven Xia +1Automated Algorithm Discovery Large Language Model-Guided Optimization
University of Illinois Urbana-Champaign
AlgoEvo: Self-Evolving Agentic Search for Automated Algorithm Discovery
Sep 14, 2026Junhao Qiu, Qinglong Hu, Ji Cheng +3Automated Algorithm Discovery Automated Heuristic Design
Department of Computer Science, City University of Hong Kong · Huawei Noah’s Ark Lab · Institute of Advanced Intelligence and Computing, A*STAR

Oct 4, 2026J/K move · Enter open · S save

AgentDiscover: Autonomous Discovery with Minimal Search Scaffolding

Mahdi Farahbakhsh, Ilan Sela, Fatemeh Doudi, Vishnu Teja Kunde+3

Texas A&M University

AI Agents for Scientific Discovery Automated Algorithm Discovery Automated Heuristic Design Autonomous Scientific Discovery LLM Agent Planning Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Frameworks that use large language models for scientific discovery typically rely on a fixed, human-designed algorithm that decides what the model sees at each step, leaving the model only the role of proposer. The model knows nothing of the search beyond what it is shown. As models grow more capable, a question arises: does a search strategy chosen by a human before the run scale better than promoting the model from proposer to planner and letting it own the search? The Bitter Lesson suggests that choosing the strategy in advance is the kind of hand-designed structure that general methods eventually outscale. We introduce AgentDiscover, in which a coding agent plans the search using its context as working memory, runs experiments, and records every attempt in a database of ideas, candidates, and their relations. This database serves as the agent's long-term memory and is structured so that the selection rules of classical algorithms such as MAP-Elites and Monte Carlo tree search each reduce to a single query, which the agent is free to use, combine, or replace. A server maintains the database and steers the agent after every submission, keeping it on course over long runs. In our experiments, AgentDiscover is more cost-efficient than existing frameworks, reaching better scores at lower cost. On tasks in kernel engineering, biology, algorithm design, and mathematics, AgentDiscover outperforms prior discovery frameworks. Its programs would have placed first among human competitors in seven past AtCoder heuristic contests, and on eleven mathematical and systems optimization tasks it matches or exceeds every baseline that uses the same model. Our code is available at https://github.com/mhdfb/AgentDiscover.

Abstract

Autonomous Research Agents

Explore similar work

AgentDiscover: Autonomous Discovery with Minimal Search Scaffolding

SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery

AlgoEvo: Self-Evolving Agentic Search for Automated Algorithm Discovery