NeuroPareto: Calibrated Acquisition for Costly Many-Goal Search in Vast Parameter Spaces

The problem of multiobjective optimization under uncertainties is often approached by taking the expectation of each objective. In this work, we propose instead to formulate this as a Bayesian decision problem and to rely on the expected value of the hypervolume, which is to be maximized with respect to a finite set of input points. We show that this can be performed using methods based on gradients in a stochastic optimization framework, provided that care is taken with respect to dominated points. Moreover, in the absence of readily available differentiable code, we propose to use Gaussian Processes as differentiable surrogate models, in order to perform the optimization. An additional contribution in this work are some active learning strategies, through acquisition functions which helps construct a surrogate model well-designed for the multiobjective optimization problem at stake. These strategies are compared on simple analytical problems to assess their performances.

Expensive black-box optimization is ubiquitous in science and engineering, where function evaluations are costly and the evaluation budget is limited. Traditional evolutionary algorithms and Meta-BlackBox Optimization (MetaBBO) approaches typically consume most evaluations on candidate selection, often wasting precious budget on inferior solutions. Although surrogate-assisted evolution and Bayesian optimization aim to reduce evaluations through surrogate models, constructing an accurate global model from limited data remains challenging, and model bias can easily trap the search in local optima. To overcome these limitations, we propose NeurGO, a generative MetaBBO framework that directly synthesizes elite candidates from historical population states. Specifically, we employ an attention-based encoder to capture the population-level search trend and condition a decoder on this representation to generate high-quality candidates, avoiding the expensive evaluation of large offspring pools. We then design a quality-diversity loss to maintain solution quality and population diversity throughout the search. Through extensive benchmarking on CEC 2008 and the COCO BBOB test suites, our method achieves better optimization performance under the same evaluation budget and exhibits faster convergence.

Offline multi-objective optimization (Offline MOO) seeks Pareto-optimal designs from static datasets without additional environment interactions. Existing generative methods typically guide sampling with external surrogate or preference models, which adds training complexity and may provide unreliable guidance. We propose ParetoPilot, a plug-and-play method that guides designs to Pareto front at inference time using a pre-trained conditional diffusion model without any surrogate. ParetoPilot introduces an Infer-Perturb-Guide (IPG) engine within the reverse diffusion process. IPG first infers the individual conditional target for each sample in the batch by aligning its conditional and unconditional predictions. It then perturbs these targets collectively across the batch, balancing convergence toward the Pareto front and diversity among samples. Finally, the engine guides the generative trajectory toward the Pareto front by injecting these perturbed targets via standard Classifier-Free Guidance (CFG). Experiments on 51 tasks demonstrate that ParetoPilot achieves the best overall ranking among 16 methods and competitive hypervolume improvement.