Black-Box Optimization

Follow this topic Ask about this topic

Black-box optimization studies how to choose inputs that maximize or minimize an objective when its internal workings or gradients are unavailable, often because each evaluation is costly. Common approaches include Gaussian-process Bayesian optimization, evolutionary strategies such as CMA-ES and differential evolution, and bandit or partitioning methods that balance exploration with refinement. Surrogate models, multi-fidelity evaluations, and learned or generative proposals extend these methods to noisy, constrained, discrete, and high-dimensional tasks, including scientific design and system tuning. Benchmarks such as COCO/BBOB and CEC help compare methods, but performance often depends on the problem and evaluation budget, leaving robust transfer and fair comparisons across settings as central challenges.

Latest papers

177 papers25 papers in the last four weeks, up 25% on the four weeks before. 0.1% of all new papers.

Sep 24, 2026

MF-SCBO : Multi-fidelity Scalable Constrained Bayesian Optimization

Many real-world optimization problems rely on expensive simulations or experiments, making the efficient use of available data essential. Multi-fidelity optimization of high-dimensional black-box functions subject to black-box constraints is increasingly relevant as the cost of objective evaluations continues to rise in applications such as machine learning, engineering, and control. To our knowledge, no existing method simultaneously addresses high-dimensionality, black-box constraints, an arbitrary number of fidelity levels, and non-nested sampling. In this work, we extend the Scalable Constrained Bayesian Optimization method to the multi-fidelity setting, resulting in the MF-SCBO method. The proposed approach is evaluated on standard benchmark functions as well as challenging problems. The experimental results demonstrate that MF-SCBO generally achieves better convergence than both the single-fidelity SCBO and the other multi-fidelity method considered in this high-dimensional and constrained settings.

Lucas Palazzolo, Mickaël Binois, Laëtitia Giraldi

Université Côte d’Azur, Inria, Calisto team, Sophia Antipolis, France · Université Côte d’Azur, Inria, Acumes team, CNRS, LJAD, Sophia Antipolis, France

Multi-Fidelity Optimization Bayesian Optimization Black-Box Optimization

Sep 24, 2026

SPADE-DFL: Communication-Efficient Decentralized Federated Learning via Derivative-Free Linearized ADMM

Reducing communication in derivative-free decentralized learning requires controlling the disagreement accumulated over multiple local updates. This paper develops SPADE-DFL, a primal--dual method that allows the number of local function-value updates between neighbor exchanges to grow with the computation budget while preserving the nonprivate convergence order. For smooth nonconvex objectives under uniform query-moment bounds, the prescribed nonprivate schedule achieves a time-averaged stationarity and consensus bound of

\mathcal{O}(T^{-1/3})

using only

Θ(T^{2/3})

communication rounds, where

T

is the number of local updates per client. For private training, the accumulated data-dependent increment is isolated from the graph correction, allowing one protected state per client and round to generate all outgoing messages. We prove client-level differential privacy for the full interactive transcript and quantify the resulting optimization error over a finite horizon. Experiments on four classification tasks show that SPADE-DFL achieves higher mean test accuracy than existing decentralized learning methods.

Mengli Wei, Mengkai Zhu, Jiawen Chen +2

School of Mathematics, Southeast University, Nanjing 210096, China

Communication-Efficient Distributed Training Black-Box Optimization Alternating Direction Method of Multipliers Decentralized Learning Differentially Private Federated Learning Derivative-Free Optimization+1

Sep 22, 2026

BOBA: Dynamic Bayesian Optimization through Bayesian Active Inference

Dynamic black-box optimization presents significant challenges for Bayesian Optimization (BO), as the objective function evolves over time, causing optimal locations to shift continuously. Existing dynamic BO (DBO) methods using standard acquisition functions such as Upper Confidence Bound (UCB) fail to explicitly account for temporal variations, leading to suboptimal sample allocation and poor tracking of moving optima. Here, we propose BOBA (Bayesian Optimization through Bayesian Active Inference), a novel acquisition function inspired by free energy principles from active inference that explicitly minimizes predictive uncertainty about future states in dynamic environments. BOBA extends traditional acquisition functions by incorporating a forward-looking uncertainty quantification that estimates uncertainty in function changes, enabling more informed exploration-exploitation trade-offs in non-stationary settings. We evaluate BOBA on synthetic dynamic benchmarks, comparing against state-of-the-art DBO methods. Our experiments demonstrate that BOBA significantly improves regret in query-restricted settings, while remaining competitive in time-limited settings. We further analyze variants of BOBA with different exploration strategies, showing how the exploration-exploitation balance can be tuned for different types of dynamic functions. This work contributes both a free energy-based acquisition function for DBO and insights into how active inference principles can enhance optimization in non-stationary environments, with implications for real-time applications requiring continuous adaptation.

Merlin Angel Kelly, Rishan Patel, Alexander Thomas +4

Department of Computer Science UCL Aspire Create University College London · Department of Electronic and Electrical Engineering University College London · UCL Aspire Create University College London

Bayesian Optimization Black-Box Optimization

Sep 21, 2026

Online Automated Algorithm Design with Large Language Models

Large language models (LLMs) enable automated algorithm design (AAD) through reasoning and code synthesis. However, most existing LLM-based AAD methods separate algorithm design from target optimization, deploying a fixed design even as the optimization state evolves. Conventional adaptive optimizers can respond to such changes, but their adjustments remain confined to predefined parameters, operators, or strategies. To address these limitations, we introduce online LLM-based AAD, a novel optimization paradigm that treats the algorithm itself as a state-dependent decision variable. At each stage, LLM agents synthesize an algorithm with new behavior logic from the current optimization state. Executing the generated algorithm advances the search and provides feedback for subsequent designs, coupling algorithm design with target optimization without requiring a separate offline algorithm pretraining stage. To implement this paradigm, we propose OnDesign, a multi-agent framework that reconciles competing design perspectives to synthesize executable algorithms and uses execution feedback to refine how runtime evidence is interpreted for subsequent designs. We evaluate OnDesign across two mainstream black-box optimization paradigms on three scenarios: Bayesian optimization, evolutionary continuous optimization, and evolutionary mixed-variable optimization. Extensive experiments on six benchmark suites and one engineering problem across multiple problem dimensions demonstrate superior overall performance over conventional optimizers and offline LLM-based AAD methods.

Zhiyao Zhang, Yichen Li, Xingyu Wu +2

Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University, Hong Kong SAR 999077, China · College of Computer Science, Chongqing University, Chongqing 400044, China

Automated Algorithm Discovery Black-Box Optimization LLM-Based Program Synthesis

Sep 19, 2026

Adaptive Differential Evolution and Multistart Search for Noisy QAOA Optimization

We benchmark classical optimization of a fixed low-depth Quantum Approximate Optimization Algorithm (QAOA) ansatz across four cost-Hamiltonian families at

N=12

,

p=3

, and

D=6

. Ten optimizers are compared over 25 independent runs under common ceilings of 10,000 and 30,000 function evaluations (FEs), first with exact statevector objectives and then with two additive observation-noise levels. Exact objectives favor multistart BFGS and multistart CMA-ES. Under noisy feedback, adaptive population methods become more competitive, but the ranking depends on whether performance is measured by the best exact point visited or by the point selected from noisy observations. A targeted extension over all three pre-screened instances per family confirms this regime change while showing that named adaptive-DE winners are instance dependent: jSO-lite leads low-noise oracle search, iL-SHADE high-noise oracle search, and L-SRTDE high-noise selected solutions in the equal-instance summaries. Bootstrap analysis quantifies a non-negligible high-noise search--selection gap, and a retrospective fixed-budget verification proxy shows that reserving a small measurement budget for final re-evaluation improves selected quality across all ten methods at 30,000 FEs. A supplementary structure-aware study further shows that QAOA cross-depth restriction and continuous basin refinement are more useful in these conditions than standalone Monte Carlo tree-search selection. Overall, optimizer choice depends jointly on landscape structure, observation noise, and final-point identification.

Vojtěch Novák, Ivan Zelinka, Swagatam Das +1

Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VSB–Technical University of Ostrava, Ostrava, Czech Republic · IT4Innovations National Supercomputing Center, VSB–Technical University of Ostrava, 708 00 Ostrava, Czech Republic · Department of Informatics and Statistics, Marine Research Institute, Klaipeda University, Lithuania +2

Stochastic Optimization Black-Box Optimization Quantum Approximate Optimization Algorithm Differential Evolution

Sep 19, 2026

D-IMPL: A Diffusion-based Solver for Parameterized BBOs

Diffusion models have demonstrated strong power in generative modeling tasks across multiple domains, exhibiting a remarkable capability of learning complex distributions from samples. In this paper, we leverage such capability to design an efficient universal diffusion-based solver for parameterized black-box optimizations (BBO), where the optimizer has only black-box access to queries of the objective function at the learning stage, yet is able to reduce the additional computational cost at the inference stage for each BBO instance while also capturing the potential multi-modal landscape of non-convex objectives. To cast our formulation as a compatible generative modeling task, we introduce the notion of minimization policy as a new solution concept, which defines a sampling distribution over the solutions that should concentrate around the minimizer set for each BBO instance. We then propose Diffusion-based Iterative Minimization Policy Learning (D-IMPL), a practical generative-model-based solver for solving parameterized BBOs that employs diffusion models to learn a minimization policy, whose density is proportional to the exponential of the negated objective values, thereby amortizing the computational costs across different BBO instances. Furthermore, we demonstrate the performance of our D-IMPL algorithm by establishing a sample complexity guarantee showing that a

δ

-approximate minimization policy can be effectively learned within

O(\log(1/δ))

iterations, and by extensive empirical evaluations over a range of constrained and unconstrained BBO tasks.

Yang Hu, Na Li

School of Engineering and Applied Sciences, Harvard University, Boston, MA

Black-Box Optimization Learning to Optimize Diffusion Models

Sep 18, 2026

A Confidence-Driven Evolutionary Algorithm for Noisy Optimization with Joint Chance Constraints

Many real-world optimization problems involve noisy objective evaluations and probabilistic constraints, particularly in the form of joint chance constraints, which are computationally expensive to evaluate. In this work, we propose CR-EA-C, a confidence-driven evolutionary algorithm for solving noisy black-box optimization problems under joint chance constraints. CR-EA-C introduces three key components: (1) analytical feasibility estimation for joint chance constraints, (2) a pairwise statistical ranking mechanism for robust comparison under noise, and (3) a modified infeasibility-driven survival strategy to accelerate convergence. These components enable statistically reliable decision-making while improving the efficiency of function evaluations. The proposed method is evaluated against four recent metaheuristic algorithms under various uncertainty distributions. Furthermore, its practical effectiveness is also assessed on two additional real-world optimization problems and compared with conventional static sampling methods. Experimental results show that CR-EA-C consistently satisfies the prescribed joint chance constraints while achieving competitive objective values overall. This demonstrates that CR-EA-C is an effective general-purpose approach for noisy optimization.

Enrico Halim, Hemant Singh, Tapabrata Ray

School of Engineering and Technology, The University of New South Wales, Canberra, Australia

Chance-Constrained Optimization Stochastic Optimization Evolutionary Optimization Black-Box Optimization

Sep 16, 2026

Artificial Neural Networks as Surrogate Models in Black Box Optimization

Black-Box Optimization (BBO) is often applied in several engineering fields and can utilize an advancement of numerical measure- ments and simulation technologies. It deals with the optimization func- tions, where an analytical description is unavailable. It relies on meth- ods that require only an input point in the search space, paired with its corresponding objective function value, obtained through non-analytical means, e.g., sensors, experiments, or simulations. Common approaches include evolutionary optimization and other metaheuristics. Since BBO methods rely solely on objective function values, they typically require many evaluations, which becomes problematic when evaluating the ob- jective function is time-consuming or expensive. This leads to using surrogate-based optimization which evaluates selected true objective val- ues and trains a regression model to approximate the objective function across the search space. Surrogate-assisted black-box optimization is a small-data learning problem because the optimizer must approximate an expensive objective function from limited evaluations. Surrogate models act as data-efficient regressors, guiding the search toward promising or informative points under a restricted evaluation budget. In this paper, a new surrogate model using artificial neural networks, called Adaptive- Fidelity Nexus Covariance Matrix Adaptation Evolution Strategy (AFN- CMA-ES), is proposed for the selective evaluation of objective functions. The experimental results show its competitive performance compared to state-of-the-art surrogate-assisted BBO methods.

Md Khadimul Islam Zim, Martin Holeňa

Czech Academy of Sciences, Institute of Computer Science, Prague, Czech Republic · Czech Academy of Sciences, Institute of Computer Science, Pod Vodárenskou věží 2, Prague, Czech Republic

Surrogate-Assisted Optimization Black-Box Optimization Neural Surrogate Modeling Covariance Matrix Adaptation Evolution Strategy

Sep 16, 2026

Benchmarking Tabular Foundation Models as Surrogates in Expensive Evolutionary Optimization

Surrogate-assisted evolutionary algorithms (SAEAs) are effective methods for solving expensive optimization problems (EOPs), where surrogate models replace most expensive evaluations and critically influence the final optimization results. In recent years, tabular foundation models have advanced rapidly, and the Tabular Prior-data Fitted Network (TabPFN) has been adopted as a surrogate model for EOPs due to its strong predictive capability, demonstrating promising performance. Motivated by its potential as a surrogate model in SAEAs, this work conducts a comprehensive study that combines extensive experiments with in-depth theoretical analysis to investigate the effectiveness of TabPFN. Specifically, we perform experiments across both offline and online SAEA settings, covering diverse problem scenarios such as single-objective, multi-objective, constrained, combinatorial, mixed-variable, and engineering optimization problems. In addition, we further analyze the advantages and limitations of TabPFN within SAEAs and provide practical guidelines for its application in different optimization settings. Results show that the effectiveness of TabPFN is highly problem dependent, and it cannot replace conventional surrogates universally. Overall, TabPFN should be adopted selectively according to data availability, landscape complexity, search space characteristics, and its role within the algorithm. Customized model management strategies and role-specific algorithm design are necessary to fully exploit its advantages and avoid its pitfalls.

Lu Han, Jin Wang, Yuchen Li +5

School of Artificial Intelligence, Xidian University, Xi’an 710071, China

Tabular Foundation Models Surrogate-Assisted Optimization Black-Box Optimization

Sep 15, 2026

Derivative-Free Structured Updates for Muon

Muon updates matrix-valued neural-network parameters by orthogonalizing a gradient-based momentum matrix. Its reliance on derivatives limits its use when gradients are unavailable or unreliable. We develop a derivative-free framework that constructs Muon-style updates from structured finite differences. Four variants are considered: full entrywise recovery, random low-rank surrogates, basis-aligned rank-one probing, and direct structured search. Exhaustive basis-aligned probing is equivalent, up to positive scaling before ideal polar orthogonalization, to coordinate finite differences. Matrix-regression experiments show that random rank-one probing can reduce the number of function evaluations substantially, at the cost of less accurate updates. Controlled noisy-gradient experiments on regression and a neural network illustrate when accurate function values can compensate for an unreliable gradient oracle. A small CartPole study further examines orthogonal rank-one probes under a fixed episode budget. These results support structured probing as a practical option for selected black-box problems; they do not establish a general convergence guarantee or an advantage over accurate, inexpensive gradients.

Pengcheng Xie

Lawrence Berkeley National Laboratory & University of California Berkeley, CA 94720, USA

Black-Box Optimization Zeroth-Order Optimization Neural Network Optimization Muon Optimizer Derivative-Free Optimization

Sep 10, 2026

Trajectory Bundle Method in SE(3) for Black-Box Fixed-Wing Aircraft Trajectory Optimization

Dynamically feasible trajectory optimization for rigid-body systems is naturally formulated on the special Euclidean group SE(3) but is challenging when dynamics are available only as black-box computations without derivatives. This paper formulates the Trajectory Bundle Method (TBM) for motion planning implicitly on SE(3). Bundles are constructed in the Lie algebra and propagated through nonlinear rigid-body dynamics using exponential and logarithmic maps, enabling derivative-free planning of non-Euclidean trajectories. We show that Euclidean TBM interpolation error is bounded quadratically by bundle diameter and extend this result to SE(3), where the bound additionally depends on a local Lipschitz constant of the Log map. Numerical experiments corroborate these bounds. Finally, we demonstrate SE(3) TBM by optimizing an acrobatic, collision-free fixed-wing maneuver through a rotated aperture without explicit models or derivatives of the vehicle dynamics, aerodynamics, or collision model.

Matthew D. Osburn, Cameron K. Peterson, John L. Salmon

Department of Electrical and Computer Engineering, Brigham Young University, Provo, UT 84602, USA · Department of Mechanical Engineering, Brigham Young University, Provo, UT 84602, USA

Trajectory Optimization Trajectory Generation Black-Box Optimization Lie Group Methods Motion Planning Derivative-Free Optimization

Sep 9, 2026

A practical DIRECT-type algorithm for medium-scale black-box global optimization

The DIRECT algorithm is a deterministic global optimization method known for its versatility and balanced exploration-exploitation strategy. However, DIRECT-type algorithms are primarily effective for low-dimensional problems and often exhibit slow convergence as dimensionality increases, limiting their applicability to more complex optimization tasks. To address this limitation, this paper introduces X-DTC-GL, a novel DIRECT-type algorithm that incorporates dynamic partitioning and hybridization techniques. The dynamic partitioning approach adaptively refines the search space based on local one-dimensional surrogate models, enabling rapid subdivision of promising hyper-rectangles. The hybridization strategy selectively employs a hill-climbing method to exploit promising regions identified by the surrogate models. Extensive experiments on four diverse benchmark suites demonstrate that X-DTC-GL significantly outperforms existing DIRECT-type baselines, achieving improvements of ~12% in solvability and ~27% in solution quality. Performance-profile analyses indicate the fastest convergence on up to ~40% of instances, the best runtime performance on ~17% of problems, and competitive overall execution times. By improving performance within the partition-based framework, these advances strengthen the algorithm's competitiveness in state-of-the-art black-box optimization.

Linas Stripinis, Remigijus Paulavičius

Vilnius University Institute of Data Science and Digital Technologies, Akademijos 4, Vilnius, Lithuania, LT-08663

Black-Box Optimization Global Optimization

Sep 8, 2026

Code-to-Harness: Distilling Black-Box Optimizers from Self-Play

Can an agent learn a numerical search strategy through executable practice and then transfer that strategy as text? We study low-budget black-box optimization, where unaided language models remain well below strong classical optimizers. During development, an agent repeatedly writes and evaluates optimizer programs. It then distills the resulting program and practice record once into a 197-word primary Harness A, which is frozen before evaluation. Harness A reduces Gemini Flash regret by 48% in an independent

N=30

study (

p<.001

), enters the GP-BO performance range on the practice family, and lowers mean regret on all three held-out BBOB landscapes. The same text improves every tested Gemini executor and transfers to Claude Sonnet, reducing regret by 43% and 49% (

p\leq.005

). An independent end-to-end replication produces Harness B, a different program and text at the same performance tier. The same framework also attains the lowest regret on a sealed YouTube reward-tuning production benchmark. Executable practice is thus a viable way to discover a search policy, and language a portable medium for deploying it.

Yi Wu, Zheng Ren, Zhiyu Hu +8

Google Mountain View, USA

Black-Box Optimization Large Language Model-Guided Optimization Learning to Optimize LLM-Based Program Synthesis

Sep 8, 2026

Constraint-Aware Discrete Black-Box Optimization Using Tensor Decomposition

Discrete black-box optimization is often addressed using approaches such as Sequential Model-Based Optimization (SMBO), which aims to improve sample efficiency by fitting surrogate models that approximate a costly objective function over a discrete search space. In many real-world problems, the set of feasible inputs is often given by logical constraints known in advance. However, existing surrogate modeling techniques generally fail to capture the symbolic rules governing feasibility in discrete input spaces. In this paper, we propose a surrogate modeling approach based on tensor decomposition that captures the structure of discrete search spaces while directly integrating feasibility information. To implement this approach, we formulate surrogate model training as a constrained polynomial optimization problem and solve a relaxed formulation using a differentiable penalty term derived from T-norms. Our experiments on both synthetic and real-world benchmarks, including a pressure vessel design task, demonstrate that the proposed method improves sample efficiency by effectively guiding the search away from infeasible regions.

Keisuke Onoue, Ryosuke Kojima

Nara Institute of Science and Technology, Nara, Japan · Graduate School of Medicine, Kyoto University, Kyoto, Japan · RIKEN Center for Biosystems Dynamics Research, Japan

Surrogate-Assisted Optimization Black-Box Optimization Constrained Optimization

Sep 6, 2026

Threshold-Based Selection for Continuous Optimization: A Leaf-Abscission Instantiation

This paper formalizes threshold-based selection as an evaluation-gating architecture in which each incumbent is tested before variation and a replacement is generated and evaluated only when contextual pressure exceeds intrinsic strength. The mechanism is instantiated as Leaf Abscission Optimization (LAO), using rank-based strength, a phenological seasonal signal, diversity modulation, environmental pressure, and a base regrowth kernel. A blocked two-to-the-fourth-power factorial analysis at dimension 10 on the CEC 2017 suite reduces the original multi-layer design to a parsimonious core: drift is harmful, while the other three auxiliary layers show no robust independent evidence of benefit. The resulting LAO-Core attains the third-best mean Friedman rank among nine optimizers at dimensions 10, 30, and 50 under the equal evaluation budget. A four-budget sweep shows budget-dependent relative performance, with adaptive differential-evolution baselines gaining relative advantage at larger budgets; the nine-cell dimension-budget analysis establishes neither an evaluation-budget-per-dimension-only law nor a statistically significant dimension-budget interaction. A paired intervention shows that diversity modulation changes late-run replacement behaviour without a detectable effect on final error at the tested budget. The evidence supports LAO as a parsimonious evaluation-gating mechanism with regime-qualified competitiveness, rather than as a generally superior optimizer.

Nasser Khalili

Graduate School of Management and Economics, Sharif University of Technology, Tehran, Iran.

Evolutionary Optimization Black-Box Optimization

Sep 2, 2026

Frontier LLMs are effective batch optimizers: Assessing reasoning models in continuous and discrete settings

Frontier large language models (LLMs) have become attractive priors for optimization due to their large-scale pretraining that enables them to navigate a variety of optimization settings. However, the effectiveness of modern reasoning LLMs in batch optimization settings remains underexplored. Here we investigate the performance of the current generation of frontier LLMs as batch optimizers in both continuous and discrete settings. We find that while LLMs are competitive zero-shot batch optimizers for numerical test functions, their performance is brittle compared to classical non-LLM optimization approaches. However, LLM priors are significantly better in semantically rich settings, indicating that their batch optimization behavior is highly effective when navigating and reasoning over the discrete spaces most similar in structure to their pretraining data.

Frank Hu, Shriram Chennakesavalu, David Graff

Prescient Design, Genentech South San Francisco, CA, USA

Black-Box Optimization Large Language Model-Guided Optimization Combinatorial Optimization Ising Optimization

Sep 1, 2026

Rethinking Learnability in Offline Data-driven Optimization

Black-Box Optimization (BBO) has broad applications, while traditional algorithms such as evolutionary algorithms and Bayesian optimization face efficiency challenges as real-world BBO problems grow increasingly complex. Data-driven optimization has been the most popular paradigm to improve the efficiency of BBO, by learning from data. Offline data-driven optimization seeks high-quality solutions using only a fixed set of previous evaluations, attracting substantial attention because it requires no additional online evaluations. Many offline optimization methods have been proposed, but a fundamental question remains unanswered: what learnability is sufficient for offline optimization? Prior theoretical studies show that Probably Approximately Correct (PAC) learnability is insufficient, as the optimal region may remain poorly learned even when most regions are well learned. In this paper, we propose algorithm-dependent learnability, which requires accuracy only on the optimizer's trajectory. We prove that its value-query form is sufficient for representative discrete settings, including greedy and local search for submodular maximization, while its first-order analogue is sufficient for projected gradient descent on convex minimization. Motivated by this notion, we formalize a trajectory-learning framework comprising trajectory construction, trajectory modeling, and candidate generation, and analyze existing trajectory-based methods under it. We further propose Uncertainty-aware Gradient-guided Trajectory Learning (UGTL), which constructs locally coherent improvement trajectories reflecting plausible search paths, models them with conditional diffusion, and selects a diverse candidate set. Our experiments show that UGTL achieves the best average rank, 3.1/25, among 25 methods on Design-Bench tasks, and confirm that our trajectory construction plays a significant role in the improvement.

Chao Qian, Chen-Guang Wang, Rong-Xi Tan +1

State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China

Black-Box Optimization Statistical Learning Theory

Aug 28, 2026

Accurate Plate Reverb Parameter Estimation Using Two-Stage Evolutionary Search

We describe our submission to Task A of the 1st DAFx parameter estimation challenge. The task is to recover the six physical parameters of a simulated metal-plate reverberator -- its dimensions and material properties -- from a single impulse response (IR). We treat this as a black-box optimization: candidate parameter sets are fed to the simulator and scored by a loss against the target IR. The method has two stages. The first uses CMA-ES, an evolutionary optimizer, to recover five of the six parameters, comparing IRs under an amplitude-normalized loss. Amplitude normalization makes the search robust but discards the cue to the sixth parameter, the plate's surface density; a second stage therefore estimates it alone, with a ternary search on the un-normalized loss. As the choice of loss strongly affects the search, we select it beforehand, and analyze why compression in the common multi-scale spectral loss degrades recovery. Finally, we test our method on a validation set of 50 IRs, discuss a pathological failure mode, and ablate to justify having two different stages instead of a unified CMA-ES search.

Byunghoo Park, Jayeon Yi, Takyoung Kim +1

Siebel School of Computing and Data Science University of Illinois Urbana-Champaign Urbana, USA

Parameter Estimation Black-Box Optimization

Aug 24, 2026

JANUS: Online Jacobian-Aligned Infill for Black-Box Optimization

Population optimizers such as CMA-ES, DE, and multi-objective evolutionary algorithms drive search mainly through selection signals that are scalar or rank based: such a signal indicates that one candidate outperforms another, but not the local direction responsible for the improvement. JANUS (Jacobian-Aligned Newton-Unified Search) is a plug-and-play infill module that extracts this missing local geometric signal without replacing the host optimizer. It estimates a local Jacobian from the recent evaluation trace; the same Jacobian yields both a damped Gauss--Newton exploitation candidate and a trace-preserving exploration metric, reserving a fraction of the host's per-generation candidate slots for geometry-guided infill rather than spending evaluations on top of the host's budget. Unlike MetaBBO methods, JANUS needs no offline training or task distribution, estimating this geometry on the fly from the current run alone, while the host keeps full control of selection, survival, covariance adaptation, and step-size control. Under same-protocol comparisons, JANUS improves the CMA-ES host on 11--15/16 BBOB functions across

d\in\{30,100,500\}

. It also attains the best mean error on 13 of the 16 functions at

d{=}500

in the complete NN-BBO/MetaBBO baseline comparison, with no training cost, and yields a

936\times

geometric-mean improvement over the host on a

d{=}1000

BBOB subset. On structured and multi-objective tasks, JANUS gives the best mean cost on 1135-dimensional UAV path planning (

-12.8\%

vs.\ the strongest baseline), and it improves SMS-EMOA/AGE-MOEA2 hosts on 12/38 multi-objective tasks with zero significant regressions. Code is available at https://github.com/hongyuanyu/JANUS.

Hongyuan Yu, Pufan Xu, Jiaojiao Yi +4

Multimedia Department, Xiaomi Inc. · School of Integrated Circuits, Tsinghua University

Black-Box Optimization Evolutionary Optimization Multi-Objective Evolutionary Optimization

Aug 19, 2026

ZoAQ: Adaptive Zeroth-Order Querying via Query-Reuse Coupling

Zeroth-order optimization (ZOO) estimates updates from function evaluations, making perturbation queries a primary cost. Fixed budgets spend the same number of queries at every step, while adaptive controllers may offset their savings by using additional oracle calls to test estimator reliability. We introduce ZoAQ, an adaptive ZOO method built around query reuse. Rather than discarding past evaluations after each step, ZoAQ makes them useful for both the next update and the decision to query further. This enables adaptive query allocation without extra validation queries. Our analysis characterizes when this agreement identifies an update that supports descent and guides the controller to a sufficient query budget. On synthetic objectives, ZoAQ reduces queries by 43-48% relative to fixed baselines using 1.2M queries. In black-box attacks, it reaches 100% success with 320 and 625 average queries on MNIST and CIFAR-10, respectively. Across four OPT fine-tuning settings, ZoAQ saves 43-46% forward evaluations relative to fixed K=4, with accuracy changes within tasks ranging from -0.018 to +0.010.

Yangyang Feng, Yao Shu

The Hong Kong University of Science and Technology (Guangzhou)

Zeroth-Order Optimization Black-Box Optimization

Aug 12, 2026

High-dimensional Multi-objective Bayesian Optimization with Learned Variable Interactions

Multi-objective Bayesian optimization (MOBO) is effective in identifying the Pareto fronts for expensive black-box problems. However, most current MOBO approaches are limited to low-dimensional decision space due to its exponential sampling complexity. This paper presents decision variable interaction analysis-based MOBO, ViaMOBO, a generic framework for expensive multi-objective problems with high-dimensional decision space. The key idea of ViaMOBO is that it utilizes a variable interaction analysis model to determine whether the decision space can be completely or partially divided, and then performs local Bayesian optimization in the divided decision subspaces. Through the variable analysis model, it can be derived whether the objectives in black-box problems are separable, partially separable, or non-separable based on the potential independent or interdependent relationships among decision variables without any strong assumptions. We compare ViaMOBO with the state-of-the-art MOBO methods on both synthetic and real-world benchmarks. The experimental results demonstrate that ViaMOBO outperforms other related MOBO baselines in approximating the Pareto front of high-dimensional expensive multi-objective problems.

Hongyan Wang, Jiayu Huang, Haotian Zheng +6

School of Vehicle and Mobility Tsinghua University Beijing, China · Faculty of Engineering The Hong Kong Polytechnic University Hong Kong, China

Bayesian Optimization Black-Box Optimization Multi-Objective Optimization

Aug 9, 2026

A New Approach to Characterising Optimisation Problems Using Programmatic Representation and Complexity Measures

Characterising optimisation problem instances is a fundamental part of understanding the behaviour and performance of different algorithms as well as providing information for algorithm selection and configuration. In this paper we propose a novel approach to problem characterisation based on the representation of instances when implemented as a program. The intuition is that the complexity of the code required to express an objective function should relate to the complexity of the search landscape. We identify the Halstead volume as a measure of code complexity, which can be seen as a simplified version of the entropy of the program. Given a code implementation of the objective function, the Halstead volume and entropy can be quickly calculated using existing libraries. We apply the proposed complexity measures to the well-known BBOB optimisation problem suite and the simple feed-forward neural network training task. We also show that the measures are negatively correlated with algorithm performance and therefore show potential as predictive meta-features for algorithm selection and other problem analysis. We envisage the proposed measures as complementary to other problem characterisation approaches, but with the advantages of not requiring any sampling of the search space, being invariant to transformations, and being very quick to calculate automatically.

Marcus Gallagher, Katherine M. Malan

School of Electrical Engineering and Computer Science, University of Queensland, Australia. · Department of Decision Sciences, University of South Africa, Pretoria, South Africa

Black-Box Optimization

Aug 8, 2026

A Hybrid Nested Harness for Decoupling Structure and Parameters in LLM-Driven Optimization

In evolutionary algorithms powered by language models, the LLM acts as a single operator that simultaneously updates structural components (like control flow) and continuous parameters. While LLMs can be good at the first, they are not efficient at the second, wasting tokens taking discrete jumps inside a trial and error loop. We resolve this by formalizing a hybrid nested search, in which an outer loop has the LLM propose a structural sketch, with numeric gaps, and an inner numerical optimizer tunes the sketch. Both the outer and inner solvers are pluggable: any text-based optimizer can be combined with a zero-order optimizer (CMA-ES), gradient-based routines, or MCMC samplers. We validate our framework across three scientific domains: (i) meta-optimizers on closed-form test functions, (ii) code-based policies for systems research and social dilemmas; and (iii) approximate Bayesian inference tasks. Across all three, the hybrid optimizer is superior to both vanilla LLM-driven search and pure numerical optimization baselines. Code at: https://github.com/vicgalle/hybrid-nested-search

Víctor Gallego

Komorebi AI Technologies

Black-Box Optimization Large Language Model-Guided Optimization LLMs for Evolutionary Optimization

Aug 7, 2026

Adaptive Hybrid Particle Swarm Optimization with Gradient Descent

Gradient injection helps Particle Swarm Optimization (PSO) only when the swarm has identified a basin with smooth local structure, not universally. We propose Adaptive Hybrid PSO (AHPSO), which uses a sigmoid function on swarm diversity to automatically modulate gradient influence: near-zero during exploration, near-maximum during exploitation, with no manual phase-switching. Under budget-normalized comparison (PSO given equivalent total function evaluations), PSO wins 52.5% of 40 configurations versus AHPSO's 20% (p = 7.0e-5, Friedman). AHPSO retains advantage specifically on problems with smooth local basins (F8, F24-F27) where directed descent outperforms undirected sampling even at equal cost. Under iteration-matched comparison across 29 functions (42 configurations, 14,700 runs), AHPSO-Adadelta ranks first of 9 methods including CMA-ES (p = 9.75e-4). The contribution is a principled characterization of when gradient injection provides value in swarm-based search, not a claim of universal superiority.

Aryan Gurudeo

Black-Box Optimization Gradient Descent Particle Swarm Optimization

Jul 31, 2026

Linear Proposal Operators and Stochastic Search Geometry in SOMA and Differential Evolution

Swarm and evolutionary algorithms are usually analyzed as complete procedural systems in which nonlinear selection, replacement, and adaptation obscure simpler structure within candidate generation. This paper introduces an operator--selection factorization that separates objective-independent variation from boundary repair and fitness-dependent selection, and uses it to study the proposal geometry of the Self-Organizing Migrating Algorithm (SOMA) and Differential Evolution (DE). The canonical SOMA proposal is shown to be affine in the search space and exactly linear in an augmented migrant--leader state. In leader-relative coordinates, the resulting operator provides a direct interpretation of interpolation, projection, overshooting, and coordinate masking. Under Bernoulli perturbation masks, we derive closed-form expressions for the proposal mean, covariance, expected squared step length, expected squared distance from the leader, active dimensionality, and coordinate coverage. For canonical DE/rand/1/bin, we derive the finite-population moments of differential mutation and characterize the additional covariance and coordinate dependence induced by forced-coordinate binomial crossover. Exact enumeration and Monte Carlo experiments verify the analytical identities and quantify the effects of mask conditioning, boundary repair, and fitness-based selection. The analysis further motivates geometry-controlled and rotation-aware SOMA variants, together with an adaptive population-reducing extension of iSOMA. Experiments on the complete noiseless BBOB benchmark show that these operator-guided variants substantially improve upon canonical SOMA and are competitive with established DE methods in several dimension--budget regimes. The results demonstrate how proposal-level operator analysis can support both the interpretation and design of population-based optimizers.

Vojtěch Novák, Ivan Zelinka

Department of Computer Science, Faculty of Electrical Engineering and Computer Science, VSB - Technical University of Ostrava, Ostrava, Czech Republic · IT4Innovations National Supercomputing Center, VSB - Technical University of Ostrava, 708 00 Ostrava, Czech Republic · Department of Informatics and Statistics, Marine Research Institute, Klaipeda University, Lithuania

Differential Evolution Evolutionary Optimization Black-Box Optimization Metaheuristic Optimization

Jul 30, 2026

Oracle-Budgeted Molecular Optimization with Short-Term Graph Memory

Molecular optimization is commonly performed under a limited oracle budget, which makes deciding what to evaluate as important as deciding what to generate. We introduce short-term graph memory, a plug-in module that preserves the generator architecture and native update rule while learning from previously evaluated molecules to prioritize subsequent oracle queries. The module maintains an online graph neural surrogate that pre-screens each round's candidate pool, so the fixed oracle budget is spent on molecules with higher predicted utility. Applied to a fragment-based generator on a standard molecular optimization benchmark, it improves the mean top-10 score at no extra oracle cost and never falls behind the base on any oracle; the gain extends to all four generators we tested at a tight budget of one thousand calls. We then analyze how surrogate-guided selection interacts with the exploration and exploitation behavior of different generators. Its benefit at larger budgets is consistent with two properties of the backbone: how broadly it searches, and how effectively its native search already exploits oracle feedback. We provide a simple way to spend a fixed oracle budget more selectively, and evidence on which generators benefit from it.

Jiannan Yang, Veronika Thost, Xiang Ling +1

Computer Science Stony Brook University Stony Brook, New York, USA · Molecular AI Novo Nordisk Lexington, Massachusetts, USA · Biomedical Engineering Stony Brook University Stony Brook, New York, USA +1

Molecular Optimization Surrogate-Assisted Optimization Black-Box Optimization Molecular Generation

Jul 28, 2026

Optimization with Dynamic Constraint Learning (DCL)

We propose Dynamic Constraint Learning (DCL), a data-driven framework for constrained optimization when constraint functions are unknown and cannot be queried during optimization. At each iteration, the method learns a local surrogate from nearby data and solves a subproblem within a data-supported trust region. Compared with offline global constraint learning, the approach uses local surrogates that adapt to the data distribution during optimization and can achieve solution quality comparable to that of global models while using simpler local models and smaller optimization subproblems. We demonstrate the performance of DCL on a synthetic test problem and two case studies from the literature.

Ezgi Oztekin, Figen Oztoprak, S. Ilker Birbil

Department of Mathematics, Gebze Technical University · Department of Industrial Engineering, Gebze Technical University · Amsterdam Business School, University of Amsterdam

Surrogate-Assisted Optimization Black-Box Optimization Constrained Optimization

Jul 26, 2026

NeurGO: Learning to Generate Elite Candidates for Meta-Black-Box Expensive Optimization

Expensive black-box optimization is ubiquitous in science and engineering, where function evaluations are costly and the evaluation budget is limited. Traditional evolutionary algorithms and Meta-BlackBox Optimization (MetaBBO) approaches typically consume most evaluations on candidate selection, often wasting precious budget on inferior solutions. Although surrogate-assisted evolution and Bayesian optimization aim to reduce evaluations through surrogate models, constructing an accurate global model from limited data remains challenging, and model bias can easily trap the search in local optima. To overcome these limitations, we propose NeurGO, a generative MetaBBO framework that directly synthesizes elite candidates from historical population states. Specifically, we employ an attention-based encoder to capture the population-level search trend and condition a decoder on this representation to generate high-quality candidates, avoiding the expensive evaluation of large offspring pools. We then design a quality-diversity loss to maintain solution quality and population diversity throughout the search. Through extensive benchmarking on CEC 2008 and the COCO BBOB test suites, our method achieves better optimization performance under the same evaluation budget and exhibits faster convergence.

Jintao He, Huixiang Zhen, Wenyin Gong

China University of Geosciences, Wuhan, China

Black-Box Optimization Learning to Optimize

Jul 24, 2026

Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias

Many organizations aim to adapt language models for internal use, both to improve performance on domain-specific tasks and to address privacy concerns around sensitive data. However, such adaptation remains non-trivial: it often requires operationally challenging fine-tuning of open-source models or ad hoc prompt optimization. We study a minimal alternative based on a simple API-level control: allowing users to bias the model's logits with a user-defined vector. We develop a black-box method for learning a single context-independent logit-bias vector, added at every decoding step, without modifying model weights or requiring gradients. Starting from a KL-regularized reinforcement learning (RL) objective, we characterize when such a fixed logit-bias vector can approximate the optimal prefix-dependent correction and derive a closed-form inverse-propensity estimator from rollouts, rewards, and token probabilities. Empirically, this simple decoding-time intervention improves over base models on mathematical and reasoning benchmarks while using far fewer trainable parameters than conventional fine-tuning. Our results suggest that learned logit bias is a lightweight mechanism for adapting language models under minimal access requirements.

Ofek I. Cohen, Lior Shani, Aviv Rosenberg +3

Tel Aviv University · Google Research · Columbia University

Black-Box Optimization Domain Adaptation for LLMs Online Language Model Adaptation Inference-Time Adaptation of Language Models

Jul 24, 2026

Optimization of time-consuming experimental conditions using pseudo-experimental data guided by adaptive polynomial regression

Bayesian optimization (BO) is an optimization method that sequentially proposes the next candidate explainable variables for optimizing target variables by balancing exploration and exploitation. BO is often used under a limited evaluation budget, such as hyperparameter tuning of deep learning. Despite its effectiveness, conventional BO may have poor convergence in practical experimental science where each evaluation is often costly and time-consuming. Recently, BO methods have been proposed that accelerate optimization by using pseudo-experimental data that simulate experimental data. However, when only a limited number of experimental data are available, the generated pseudo-experimental data may be of insufficient quality. In this study, we developed PolyBO to improve optimization time by generating high-quality pseudo-experimental data even when the number of trials is limited. PolyBO performs BO efficiently by generating pseudo-experimental data with an adaptively updated versatile parametric model. This low-capacity polynomial regression model is intended to enable efficient BO even with limited experimental data. PolyBO updates the BO surrogate model with a combined dataset consisting of experimental data and pseudo-experimental data and then performs optimization. Using synthetic benchmark functions with diverse landscapes, we found that PolyBO reduced the optimization time by a median of 42%. For a real-world material composition optimization problem, PolyBO reduced the optimization time by a median of 96% compared with conventional methods. Overall, PolyBO achieves efficient optimization in settings where each experiment requires a long time.

Hirotaka Sugawara, Yujin Taguchi, Kei Minagawa +3

1Graduate School of Science and Technology, Keio University, Kanagawa, Japan · Department of Biosciences and Informatics, Keio University, Kanagawa, Japan

Surrogate-Assisted Optimization Bayesian Optimization Black-Box Optimization