NextEval: An Agent-Based Solver and Benchmark for Derivative-Free Optimization

Shuo Chen; Li Cui; Cunxin Huang; Ya-Feng Liu; Minghang Xie; Meng Xu; Zaikun Zhang

cs.LGOct 27, 2026

NextEval: An Agent-Based Solver and Benchmark for Derivative-Free Optimization

Authors: Shuo Chen, Li Cui, Cunxin Huang, Ya-Feng Liu, Minghang Xie, Meng Xu, Zaikun Zhang

Organizations: Independent Researcher · Beijing Normal University · The Hong Kong Polytechnic University · Beijing University of Posts and Telecommunications ·

Abstract

Derivative-free optimization (DFO) addresses problems where derivatives are unavailable or costly to obtain. This setting often arises in simulation-based and experimental applications, where evaluating the black-box objective can itself be costly. Thus, it remains a central challenge in DFO to effectively exploit accumulated observations to select informative evaluation points under a limited budget. In this light, we introduce the NextEval solver, an agent-based DFO solver, which integrates the capabilities of a large language model (LLM) to guide the sequential selection of evaluation points. Specifically, we establish a system that combines an LLM agent with a numerical environment. The environment provides tools for analyzing past evaluations and executing numerical methods, while maintaining a shared evaluation history and method-specific state to support continuation across method switches. Using observations and tool feedback, the LLM agent selects tools, configures their parameters, and allocates the evaluation budget, while numerical methods compute and evaluate candidate points. Building on this system, we also introduce NextEval Bench, a benchmark protocol for comparing agent configurations on shared DFO tasks using the same numerical tools under matched evaluation budgets and information constraints. Numerical results on low-dimensional DFO problems demonstrate that the NextEval solver outperforms the state-of-the-art DFO baselines in function-evaluation efficiency.

Figures & tables

Figure 1: Conceptual view of NextEval. The agent selects tools using past evaluations. The illustrated landscape is neither an agent input nor an experimental trajectory.

Figure 2: Overview of NextEval Solver: the agent selects tools and their parameters, while the numerical environment executes the requests, maintains the evaluation history and the numerical state of each method, and returns feedback.

Figure 3: NextEval Bench runs agent configurations on matched tasks through the shared numerical environment and verifies run records before comparing performance using OptiProfiler.

Study	Research question
Solver performance	Can the NextEval solver improve evaluation efficiency over classical DFO solvers?
Case study	How does the agent adapt its search in one audited trajectory?
Model choice	How does the choice of LLMs affect optimization performance?
Reasoning setting	Does additional reasoning improve optimization performance?
Execution harness	How sensitive is optimization performance to the execution harness?

Table 1: Research questions guiding the experiments.

Figure 4: DeepSeek V4 Flash feature-suite overview. History-performance (top) and history-data (bottom) profiles at

10^{-6}

; shading shows the repetition range. The first two columns use historical three-repeat cohorts; rotation uses the built-in runner with thinking off and ten repeats. These are not matched feature ablations.

Figure 5: Recorded SNAIL trajectories (repetition 9). Red: evaluations 81–98. Inset: shared initial region. Clean contours are evaluator-only.

Solver	Aggregate score	Perf. $10^{-4}$	Data $10^{-4}$	Perf. $10^{-6}$	Data $10^{-6}$
NextEval	1.000	1.000	1.000	1.000	1.000
COBYQA	0.805	0.847	0.899	0.796	0.856
NEWUOA (PDFO)	0.820	0.862	0.913	0.807	0.864
Powell (SciPy)	0.354	0.370	0.466	0.300	0.393
BFGS (finite differences)	0.810	0.843	0.793	0.818	0.756

Table 2: Normalized profile-area scores for perturbed starts with rotation. Perf. and Data report

S_{\mathrm{OP}}(\widetilde{\rho}_{s,k})

and

S_{\mathrm{OP}}(\widetilde{\delta}_{s,k})

; Aggregate score reports

S_{s}

. Higher is better; scores are not win rates.

Figure 6: Three agent-configuration comparisons. History-performance profiles at

10^{-6}

, with repetition ranges. (a) Codex with medium reasoning; (b) Qwen 3.8 Flash with Codex; (c) DeepSeek V4 Flash with thinking off. All three comparisons use the full 92-problem suite. Targets are computed separately within each panel. Data profiles and trial accounting are in Appendices D.1 – D.2 .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Field	DeepSeek Flash feature suite	Model configuration studies
Repetitions	Earlier cohorts: 3; rotated: 10	Ten in each study
Features	Perturbed starts alone, with noise, or with rotation; start/noise levels $10^{-3}$	Perturbed starts with noise; both levels $10^{-3}$
Harness	Earlier cohorts: historical task-mode; rotated: built-in task-mode runner	GPT and Qwen thinking: Codex; harness study: built-in vs. Claude Code
Models	DeepSeek V4 Flash (observed ID below)	GPT configurations, Qwen 3.8 Flash, and DeepSeek V4 Flash
Thinking	Earlier cohorts: historical setting; rotated: disabled	GPT: medium; Qwen thinking: off/medium; harness comparison: off
Session wall limit	Earlier cohorts: 1,800 seconds; rotated: 3,600 seconds	GPT: 1,800 seconds; Qwen thinking: no solver-side wall cap; harness: 3,600 seconds

Appendix

Table 3: Execution settings. Wall limits are operational caps; none of the paper’s scores use elapsed time.

Quantity	Perturbed start	Perturbed start + noise
Problems	92	92
All displayed solver trials	1,380	1,380
Agent trials	276	276
Runner interruptions	8	12
Session wall limits	17	47
Invalid initial values	2	2

Appendix

Table 4: Earlier perturbed-start study accounting after the classical-baseline supplement. Agent events refer to the unchanged historical runs.

Figure 7: NextEval–DFO histories for ROSENBR, CLIFF, and HIMMELBG under perturbed starts and, for the lower row, observation noise. All three repetitions are included in each archived plot. These histories describe outcomes rather than a tool-level causal ablation.

Figure 8: Two illustrative GPT histories from the earlier three-repeat campaign. They are not ten-repeat summaries; aggregate results are determined by the full comparison set rather than these selected examples.

Repetition	Qwen medium	Qwen off	Best classical	Best classical value	Medium strict improvements
rep02	$0.1843$	$13.88$	PDFO	$100.7$	35
rep08	$0.9525$	$100.6$	PDFO	$100.8$	13

Appendix

Table 5: Observed incumbent values for paired two-dimensional ROSENBRTU records under the listed Qwen settings. The classical entry is the best observed value among COBYQA, Powell, and PDFO in the same repetition.

Figure 9: Observed incumbent curves for paired two-dimensional ROSENBRTU records under Qwen 3.8 Flash with thinking set to medium or off and the three paired classical references. Each curve is the cumulative minimum of the method’s recorded noisy observations. The vertical axis reports the observed incumbent and the horizontal axis reports evaluator calls.

Case	Model and recorded evaluation sequence	Paired classical observations
ROSENBRTU rep04	GPT-5.5; evaluations 53–58; incumbent $100.8$ before three line-search calls without a strict decrease; scale_restart followed, with observed values $92.82$ (evaluation 58), $3.642$ (61), and $0.8773$ (89).	Powell: 96/100 calls after its last strict improvement, then evaluation-limit termination at $100.7$ ; COBYQA/PDFO: 43/45 and 46/48 plateau calls before a trust-region radius lower bound.
SNAIL rep09	GPT-5.6 Terra; evaluations 78–82; see the audited case above for the full recorded action sequence; observed value $1.157\times 10^{-3}$ at evaluation 98.	Powell: successful termination after 73 calls at $14.23$ , with 50 calls after its last strict improvement; COBYQA/PDFO: 11/52 and 7/59 plateau calls, with PDFO near $13.93$ .
LOGHAIRY rep08	GPT-5.6 Terra; evaluations 38–39; incumbent $0.4815$ and a Newton candidate of $0.8568$ ; subsequent trust-region calls reached $0.1805$ at evaluation 67.	Powell: 72/100 calls after its last strict improvement and evaluation-limit termination at $6.539$ ; COBYQA/PDFO: 32/49 and 10/48 plateau calls before a trust-region radius lower bound, with best observed value near $6.529$ .

Appendix

Table 6: Recorded observations for three action-level two-dimensional ten-repeat traces. All objective values are observed values; the final column reports paired classical observations and reported termination conditions.

Figure 10: Recorded decision trajectories for three two-dimensional ten-repeat traces. Contour colors show post-hoc clean S2MPJ objective evaluations on a logarithmic scale. Blue, orange, purple, and green paths and markers show NextEval+GPT, Powell, COBYQA, and PDFO, respectively. Each straight connector joins consecutive recorded evaluator calls and does not denote continuous optimizer motion. Red overlays highlight the segment from the evaluation immediately preceding the selected action change to the evaluation attaining the selected NextEval best. Stars mark each method’s best recorded observation, and insets enlarge the shared initial regions. The clean contours were not available to the agent.

Case	Dim.	Budget	Final observed	Recorded evaluation sequence	Paired classical observations
SNAIL rep08	2	100	$-2.191\times 10^{-3}$	Line search without a strict decrease near $13.93$ $\rightarrow$ broad trust region to $1.987\times 10^{-3}$ $\rightarrow$ Newton to $6.75\times 10^{-4}$ $\rightarrow$ trust-region continuation.	Powell/COBYQA/PDFO best values $14.24/\allowbreak 14.14/\allowbreak 13.98$ ; plateau tails 23/100, 3/51, and 17/55 calls.
ECKERLE4LS rep08	3	150	$2.06\times 10^{-4}$	Coordinate search without a strict decrease $\rightarrow$ trust-region restart to $0.4459$ $\rightarrow$ line search to $0.4198$ $\rightarrow$ trust-region refinement.	Powell/COBYQA/PDFO best values $0.6974/\allowbreak 0.6979/\allowbreak 0.6973$ ; plateau tails 61/141, 35/66, and 46/75 calls.
MGH09LS rep03	4	200	$-3.94\times 10^{-4}$	Line search and broad trust-region calls without a strict decrease $\rightarrow$ coordinate search.	Powell/COBYQA/PDFO best values $2.627\times 10^{-3}/\allowbreak 3.200\times 10^{-3}/\allowbreak 1.231\times 10^{-2}$ ; plateau tails 8/200, 29/98, and 27/145 calls.

Appendix

Table 7: Recorded observations for three supplementary ten-repeat traces. The dimension and budget columns identify the experimental setting; the final column reports paired classical observations.

Figure 11: GPT model comparison with Codex and medium reasoning fixed. History-performance and history-data profiles at

10^{-6}

use all 92 problems and ten repetitions. Targets are computed within the four models; shading shows the repetition range, not a confidence interval.

Figure 12: Ten-repeat Qwen reasoning-setting comparison. Both configurations use NextEval with Codex and Qwen 3.8 Flash; only the reasoning setting (off versus medium) changes. OptiProfiler history-performance and data profiles are shown at

10^{-6}

over all 92 problems. References are recomputed within these two agents, and shading shows the repetition range rather than a confidence interval.

Figure 13: DeepSeek V4 Flash with the built-in runner and Claude Code. OptiProfiler history-performance and data profiles at

10^{-6}

use all 92 problems with ten repeats. Targets are computed within these two configurations; shading shows the repetition range.

Harness	Performance	Data
Built-in	0.947	0.966
Claude Code	0.999	1.000

Appendix

Table 8: History-based profile scores, averaged over tolerances

10^{-1}

through

10^{-10}

. Higher is better.

Operation	Numerical work	Retained information
Trust-region campaign	Quadratic interpolation, step acceptance, geometry repair	Radius, resolution, sample set, Hessian
Line-search campaign	Bracketing and safeguarded parabolic search	Direction and successful scale
Poll / coordinate search	Directional tests and step-size adaptation	Poll length / one scale per axis
Newton jump	Safeguarded minimizer of a fitted quadratic	Model checks and recent outcome
Geometry rebuild	Construct a usable local sample set	Samples and geometry report
Model preview	Fit a surrogate and inspect its candidate	No new objective evaluations

Appendix

Table 9: The numerical action space. Method state is separate across families. The final two rows use existing observations only.

Derivative-Free Optimization

Derivative-free optimization searches for better solutions using objective evaluations instead of gradients, especially for black-box or non-differentiable problems.

See all 22 papers

DUET: Co-Evolving Solver and Grader Agents
Oct 2, 2026Fengyu Gao, Sourav Pal, Austin Z. Henley +2LLM Agent Self-Improvement LLM Agent Evaluation
University of Virginia, Charlottesville, United States · Microsoft, Redmond, United States
RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgets
Apr 6, 2026Andrew Borthwick, Stephen Ash, Anthony GalczakLLMs for Evolutionary Optimization Large Language Model-Guided Optimization
Independent Researchers
A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization
Oct 8, 2026Ming Chen, Rong-Xi Tan, Ke Xue +8Black-Box Optimization AI Agent Benchmarks
State Key Laboratory of Novel Software Technology, Nanjing University · School of Artificial Intelligence, Nanjing University

Oct 2, 2026J/K move · Enter open · S save

DUET: Co-Evolving Solver and Grader Agents

Fengyu Gao, Sourav Pal, Austin Z. Henley, Arjun Radhakrishna+1

University of Virginia, Charlottesville, United States · Microsoft, Redmond, United States

LLM Agent Self-Improvement LLM Agent Evaluation

Open paper AskPDF arXiv

Agentic workflows are increasingly used across domains such as technology, finance, and enterprise operations. As these agents become more widely deployed, continually improving them becomes increasingly important. This raises an immediate challenge: How should the agent evolve? This evolution requires effective evaluation that can assess outcomes and provide useful feedback for optimization. As the agent evolves, its behaviors and failure modes may also change, making a fixed evaluator increasingly inadequate. Another fundamental question: How should we evaluate an evolving agent? These two challenges are inherently coupled; changes in agent behavior can expose limitations of the current evaluator, while a stronger evaluator provides more informative feedback for improving the agent. Motivated by this interaction, we introduce DUET, a framework that jointly optimizes a solver agent and a grader agent to improve both. DUET iteratively selects training tasks, executes them with the solver, evaluates the resulting outcomes with the grader, and uses a tool-using update module to revise the solver and the grader, alternating between the two across rounds. By updating the grader within the optimization loop, DUET turns evaluation from a fixed source of feedback into a first-class optimization objective that adapts alongside the solver. Experiments across four agent benchmarks show that DUET improves both solver and grader performance and consistently outperforms baselines that optimize the solver with a fixed grader.

Abstract

Derivative-Free Optimization

Explore similar work

DUET: Co-Evolving Solver and Grader Agents

RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgets

A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization