Goal-Persistent Coding Agents as Scientific Performance Engineers: A Fixed-Radius Nearest-Neighbor Case Study

Xiangyang Ju

cs.AISep 25, 2026

Goal-Persistent Coding Agents as Scientific Performance Engineers: A Fixed-Radius Nearest-Neighbor Case Study

Authors: Xiangyang Ju

Organizations: Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Abstract

Coding agents can pursue persistent objectives across many tool-use turns, but evidence that general-purpose agents can conduct rigorous scientific performance engineering remains limited. We present a repository-scale case study in which off-the-shelf Codex and Claude Code agents optimize fixed-radius nearest-neighbor (FRNN) search for particle tracking. Starting from a PyTorch-dependent CUDA implementation, the agents follow an executable goal that specifies exact-correctness tests, profiling requirements, and acceptance criteria without prescribing code transformations. In the primary sequential trajectory, they autonomously remove the PyTorch dependency and conduct hypothesis-driven optimization experiments. The resulting standalone C++/CUDA library exactly reproduces the targeted reference result. Its synchronous NumPy interface achieved 1.6-fold speedup over the original GPU-resident PyTorch interface, despite including host transfers. Similar speedups were observed across different GPU architectures and software stacks. An independent optimization rerun followed a different sequence of hypotheses and reached even better performance on the target workload. These results show that goal-persistent coding agents can act as experimental performance engineers, and that executable scientific contracts are needed both to guide and to validate their optimization.

Figures & tables

Figure 1: Overview of the primary campaign. Five stages proceed from left to right, with the human, agent, and repository roles shown from top to bottom. Each execution stage expands into the hypothesis loop in Figure 2 ; goal construction does not.

Figure 2: The two-level autonomous optimization loop. The outer loop iterates over hypotheses. Within the dashed region, the inner loop adjusts implementation hyperparameters until a variant improves latency or the agent concludes that further tuning is unlikely to help.

Variant	Device $p_{50}$ (ms)	Outcome
Codex insertion top- $K$ , SM52	370.772	Pre-selection baseline
Codex always-heap top- $K$ , SM52	346.110	Rejected: sparse regression
Codex hybrid top- $K$ (32), SM52	358.924	Superseded
Codex hybrid top- $K$ (24), SM52	357.715	Retained design
Codex handoff, SM52	356.277	Exact checkpoint
Claude baseline, SM75	360.341	Session baseline

Table 1: Representative real-workload optimization experiments. Each row is a warm-device

p_{50}

. Codex top-

K

prototypes used 3 warm-up and 10 measured calls; all other rows used 10 warm-up and 30 measured calls.

GPU	Stack	frnn_cuda host	Original FRNN	Speedup
RTX 2070 SUPER	2.13.0 / 13.0	299.6 [299.1, 300.0]	473.0 [472.3, 474.3]	1.58 [1.58, 1.58]
A100-SXM4 (40 GB)	2.13.0 / 13.0	166.6 [163.1, 171.1]	244.7 [244.5, 244.9]	1.47 [1.43, 1.50]
A100-SXM4 (80 GB)	2.13.0 / 13.0	164.2 [162.8, 170.8]	246.7 [246.5, 246.9]	1.50 [1.45, 1.52]
A100-SXM4 (40 GB)	2.8.0 / 12.9	177.2 [171.2, 189.1]	244.7 [244.5, 244.9]	1.38 [1.30, 1.43]

Table 2: Real-workload comparison across GPU generations. Values are median latencies in milliseconds over 30 interleaved rounds; brackets give 95% bootstrap confidence intervals. Speedup is the paired ratio of the original FRNN latency to the frnn_cuda latency, with its 95% bootstrap interval. Stack lists the PyTorch and CUDA versions. Every row reproduces the complete 9,279,672-edge reference set and favors frnn_cuda in all 30 rounds (two-sided exact sign test,

p=1.86\times 10^{-9}

Variant	Device $p_{50}$ (ms)	Outcome
Native SM75 baseline	307.861	Reference
Warp-cooperative search	537.598	Rejected: slower
Append then post-sort	292.526	Core test passed
Grouped D12 distance loads	252.996	Core test passed
No distance early exit	268.761	Rejected: slower
Heap-free $K\geq 256$ dispatch	250.198	Unsafe for dense inputs

Table 3: Independent continuation-stage Claude rerun on the real workload. Values are warm-device medians. The final candidate is exact on this workload but its heap-free dispatch is not exact for arbitrary densities.

Coding Agents

Coding agents use language models, tools, and feedback to modify software and complete engineering tasks.

See all 141 papers

Evolving Agent Teams
Oct 27, 2026Shiyi Cao, Ziming Mao, Dacheng Li +2Multi-Agent Coding Self-Evolving Agents
University of California, Berkeley
VeRO: A Harness for Agents to Optimize Agents
Feb 25, 2026Varun Ursekar, Apaar Shanker, Veronica Chatrath +2Agent Harness Optimization LLM Agent Harnesses
Scale AI.
PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization
Jul 22, 2026Ryan Deng, Yuanzhe Liu, Bastian Lipka +4AI Coding Agents Code Optimization
Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · Rensselaer Polytechnic Institute, Troy, New York, USA · IBM, Ehningen, Germany +2

Oct 27, 2026J/K move · Enter open · S save

Evolving Agent Teams

Shiyi Cao, Ziming Mao, Dacheng Li, Joseph E. Gonzalez+1

University of California, Berkeley

Multi-Agent Coding Self-Evolving Agents LLM Agent Workflow Optimization GPU Kernel Optimization

Open paper AskPDF arXiv

The rapid progress of software engineering agents drives interest in applying them to open-ended problems such as GPU kernel optimization. However, we find that existing coding agents converge to local optima on real-world, expert-written kernels (e.g., FlashInfer). We introduce Evolving Agent Teams, a multi-agent framework that escapes such local optima. It features (1) a central supervisor that diagnoses each local optimum and restructures a team of agents; and (2) a persistent file system that comprises a learning document, an evolution history, and per-team artifacts, grounding restructuring with accumulated experience. Under a matched token budget, Evolving Agent Teams consistently outperforms strong coding-agent baselines. With an extended token budget, Evolving Agent Teams generates a CUDA kernel that reaches 1.2x mean speedup over the production FlashInfer MLA paged decode kernel from scratch and wins on 36 of 47 production workloads from FlashInfer-Bench (0.86-0.99x on the remaining 11). It also demonstrates superior performance on tasks such as KDA prefill on AMD MI350X (3.01x over vLLM) and NVIDIA B200 (3.44x over FlashKDA), MoE expert-offload serving on Apple silicon (30.3 tokens/s decode for Qwen3.8-Flash-Next in MLX 4-bit on a 48 GB M4 Pro), and RoboDojo robot manipulation (62.1 on held-out seeds vs. 55.3 for the best leaderboard entry). Code is available at https://github.com/caoshiyi/EAT.

Abstract

Coding Agents

Explore similar work

Evolving Agent Teams

VeRO: A Harness for Agents to Optimize Agents

PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization