One-Shot Localisation of the Global Minimum of a Noisy One-Dimensional Function: An Iterative Neural Minimizer Compared with Set Transformers and Classical Estimators

Qusay Muzaffar; David Levin; Michael Werman

cs.LGApr 4, 2026

One-Shot Localisation of the Global Minimum of a Noisy One-Dimensional Function: An Iterative Neural Minimizer Compared with Set Transformers and Classical Estimators

Authors: Qusay Muzaffar, David Levin, Michael Werman

Organizations: Department of Computer Science, The Hebrew University of Jerusalem, Israel · Department of Applied Mathematics, Tel Aviv University, Israel

Abstract

We study a passive form of global optimisation: from twenty noisy samples of an unknown one-dimensional function, predict where its global minimum lies, with no further queries. We introduce the Neural Function Minimizer (NFM), an iterative model that walks a position across the domain and, at every step, reads the samples near that position and attends to all twenty of them, and we compare it with two Set Transformers of the same size trained on the same data, one that answers with a single point and one that answers with a mixture of candidate locations, and with classical zero-query estimators, over three training seeds per learned model. On held-out cases from the training families the three learned models are tied on location error, and each is more accurate on average than every classical estimator; against the Gaussian-process posterior argmin the NFM's error is lower by 1.3 points of the domain. The NFM has lower regret than the single-point Set Transformer, and its per-case uncertainty has a better likelihood than that model's and orders the cases by error better than the mixture model's. The Gaussian process and the mixture model have lower regret, and on functions outside the training families the Gaussian process is the most accurate estimator. Where two valleys are equally deep, the form of an estimator's answer, not its architecture, decides whether it commits to a valley or answers between them: every estimator that returns one point fitted to distance, learned or spline-based, hedges in a fifth to nearly a third of exact ties, while estimators that name a mode commit, and one Gaussian-process posterior does both, depending on whether it is summarised by its median or its mode. We will release the benchmark and a self-checking harness; its cases are identical on different machines and its results agree to rounding.

Figures & tables

Figure 1: The NFM. Top: the four per-sample channels (with

y

normalised to

[0,1]

) are lifted, fused position-wise by a U-Net, pooled into the state

\mathbf{e}

, normalised, and conditioned once on the five global descriptors

g

by FiLM. Bottom, repeated

T=15

times: the local read returns a Cauchy-windowed summary of the samples around

x_{t}

(Figure 4 ); the iterator maps state, position, previous step,

g

and the local read to a bounded direction and a bounded step; the updater attends to the twenty samples from the new position and adds a correction to the state through a zero-initialised gate (Figure 3 ). The answer is

x_{T}

; the spread head reads the final state. Blue dots mark the normalisation and residual gate of Section 4.3 .

Method	Mean	Median	Right basin	Regret	Method $-$ NFM, error	Method $-$ NFM, regret
NFM (3 seeds)	6.72 $\pm$ 0.07	2.31	82.4%	12.80	—	—
NFM, per-case updater (ablation)	6.79 $\pm$ 0.32	2.59	82.3%	13.89	$+0.07$ [ $-0.22$ , $+0.43$ ]	$+1.09$ [ $+0.36$ , $+1.91$ ] ∗
ST (3 seeds)	6.89 $\pm$ 0.22	2.81	82.7%	14.90	$+0.18$ [ $-0.10$ , $+0.44$ ]	$+2.10$ [ $+1.18$ , $+2.92$ ] ∗
MH-ST (3 seeds)	6.99 $\pm$ 0.41	1.64	81.6%	8.46	$+0.27$ [ $-0.18$ , $+0.69$ ]	$-4.33$ [ $-5.08$ , $-3.56$ ] ∗
GP posterior argmin	8.02	1.40	78.8%	5.53	$+1.30$ [ $+0.99$ , $+1.62$ ] ∗	$-7.27$ [ $-7.79$ , $-6.75$ ] ∗
GP argmin posterior, median	7.79	1.66	78.6%	7.54	$+1.08$ [ $+0.78$ , $+1.37$ ] ∗	$-5.26$ [ $-5.81$ , $-4.70$ ] ∗

Table 1: Location error (percent of the domain) and regret (percent of the function’s range) on the 5,200 in-family cases (13 families

\times

\lambda\in\{0,0.25,0.5,1\}

\times

100). Every method answers from the same 20 samples of the same case. Learned models are means over three training seeds (

\pm

the standard deviation of the seed means). The last two columns are paired differences, the row’s method minus the NFM, with 95% intervals from a bootstrap that resamples seeds within each model and cases within each (family, level) cell; positive values favour the NFM, and ∗ marks an interval that excludes zero. Bold marks the best value in each column. Every surrogate argmin is taken between the first and last sample. The median of the NFM, ST and GP answers (not shown) reaches 6.18 (Section 6.8 ).

		Exact tie (gap 0%)			Hedging, gap
Form of the answer	Estimator	Hedging	In a floor	Capped regret	2%	10%
One point, fitted to distance	NFM (3 seeds)	29.2%	36.5%	18.55	31.3%	25.2%
	NFM, per-case updater (3 seeds)	24.2%	33.9%	14.72	26.2%	21.8%
	ST (3 seeds)	29.2%	38.2%	19.32	30.1%	20.5%
Minimum of a smoothed curve	Spline argmin	24.0%	40.8%	14.58	25.2%	24.0%
	Differential evolution on spline	20.0%	45.5%	11.63	21.5%	20.0%

Table 2: Behaviour at tied minima, grouped by the form of the answer: the harness’s tie set, double wells whose two wells differ by 0, 2 or 10% of the barrier, at four noise levels (400 cases per gap). Hedging : the answer is more than 5% of the domain from every valley floor. In a floor : within 2% of one. Capped regret :

f(\hat{x})-f(x^{*})

in percent of the barrier, with any value above one barrier counted as one barrier. Paired differences over all tie cases, with tie shapes resampled: ST

-

NFM hedging

-2.0

[

-14.6

+8.9

] points, capped regret

-0.90

[

-8.61

+5.98

]; MH-ST

-

NFM

-27.8

[

-40.0

-18.0

] ∗ and

-17.27

[

-24.84

-11.28

] ∗ . Bold marks the best value in each column.

Figure 2: Where the answers land on the tied minima of Griewank: all 600 cases (five windows

\times

six noise levels

\times

20 repeats) and all three seeds of each model, mapped back to the native coordinate, against the function (grey). Dotted lines mark valley floors (the global one at 0 and the two nearly equal ones at

\pm 2\pi

), dashed lines the ridges between them. The ST’s answers (top) spread across the slopes between the valleys; the NFM’s (middle) concentrate more on the floors; the MH-ST’s (bottom) almost never lie on a slope, but in the outer valleys often lie beyond the floor.

Model / reading	$\bar{s}/\overline{\|e\|}$	dev	PIT	ENCE	NLL	CRPS	$\rho(s,\|e\|)$	E-AURC
NFM	1.13	0.045	0.044	0.115	$-1.36$	5.25	0.60	1.43
NFM, per-case updater (ablation)	1.03	0.027	0.020	0.073	$-1.30$	5.22	0.58	1.53
ST, Huber read-back	1.03	0.033	0.028	0.093	$-1.26$	5.27	0.53	1.59
ST, as produced	0.77	0.104	0.051	0.587	$-1.11$	5.31	0.53	1.59
MH-ST, derived spread	1.05	0.037	0.055	0.083	$-1.33$	5.71	0.46	1.74
MH-ST, full mixture	—	—	0.044	—	$-$ 1.91	4.87	—	—

Table 3: Per-case uncertainty on the 5,200 in-family cases, every spread read as the scale

s

of a Laplace distribution centred on the answer. Targets:

\bar{s}/\overline{|e|}=1

; coverage of

|e|\leq s,2s,3s

63.2

86.5

and

95.0\%

, whose mean absolute deviation is dev ; PIT and ENCE 0. NLL is the Laplace negative log-likelihood; CRPS is in percent of the domain; E-AURC is the excess area under the risk–coverage curve, in percent of the domain. The ST’s head was trained on the Huber value of its error; Huber read-back converts it exactly to the Laplace scale. The MH-ST’s derived spread is

\mathbb{E}|X-\hat{x}|

under its mixture. The GP’s spread is the mean distance of its 256 argmin draws from its answer, floored at 0.01 like the learned models’ heads; the full posterior is sample-based, so it has CRPS and PIT but no NLL. Intervals as in Table 1 , 2,000 draws. After one cross-fitted scale factor (

\times 0.90

for the NFM), the NFM’s dev is level with every learned model’s and its NLL remains better than the ST’s (

+0.10

∗ ).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Weight	Basins	Multi-basin	Gap	$d/R$ , median [10%, 90%]	Range $R$	$w_{b}$	Resolvable	Tie cases
Bowl	0.09	1 (1)	0.0%	—	1.34 [0.00, 42.9]	0.98	0.706	100.0%	0
Polynomial	0.08	2 (4)	97.9%	16.5	15.6 [0.27, 93.4]	0.94	0.254	100.0%	28
Periodic	0.09	5 (17)	100.0%	7.2	7.24 [1.23, 24.9]	3.65	0.097	95.1%	8
Regular periodic	0.09	6 (9)	100.0%	4.3	4.25 [1.92, 10.1]	5.00	0.127	99.3%	0
Double well	0.10	2 (2)	100.0%	0.3	0.32 [0.03, 3.5]	46.02	0.693	100.0%	99
Multi-well	0.08	2 (4)	86.8%	32.3	32.3 [2.85, 99.8]	2.39	0.071	95.8%	5

Appendix

Table 4: Properties of the thirteen families, measured on 1,872 draws from the training generator (144 per family, 24 at each noise level). Basins : median (maximum) number of watershed basins of

f

G_{3000}

. Multi-basin : share of cases with more than one basin. Gap : median height of the runner-up basin’s floor above the global minimum, in percent of

R

(undefined for a single basin).

d/R

: well depth in percent of the range, median and 10th and 90th percentiles; the median differs by a factor of about 176 between double wells and deceptive functions, which is why the noise is scaled to

d

and not to

R

. Range

R

: median, in the function’s own units.

w_{b}

: median width of the global basin at 10% of the range. Resolvable : a sample lies within

\max(w_{b},0.06)

x^{*}

. Tie cases : gap below 1%.

Unseen (never used in development)
Function	$f(x)$	Domain
Forrester	$(6x-2)^{2}\sin(12x-4)$	$[0,1]$
Gramacy–Lee	$\sin(10\pi x)/(2x)+(x-1)^{4}$	$[0.5,2.5]$
Ackley	$-20e^{-0.2\|x\|}-e^{\cos 2\pi x}+20+e$	$[-4,4]$
Michalewicz	$-\sin x\,\sin^{20}(x^{2}/\pi)$	$[0,\pi]$
Shubert	$\sum_{j=1}^{5}j\cos\big((j+1)x+j\big)$	$[-2,4]$

Appendix

Table 5: The out-of-family functions, in their native form and domain.

Measure	Other model	Other $-$ NFM [95% CI]	$p$	Holm $p$
In-family error	ST	$+0.18$ [ $-0.10$ , $+0.44$ ]	0.21	0.61
In-family error	MH-ST	$+0.27$ [ $-0.19$ , $+0.70$ ]	0.22	0.61
In-family error	GP posterior argmin	$+1.30$ [ $+1.00$ , $+1.60$ ]	$<$ 0.0002	0.0016
In-family regret	ST	$+2.10$ [ $+1.21$ , $+2.93$ ]	$<$ 0.0002	0.0016
Tie hedging (points)	ST	$-2.0$ [ $-14.4$ , $+8.8$ ]	0.76	0.76
NLL	ST, read-back	$+0.10$ [ $+0.04$ , $+0.16$ ]	0.0005	0.003

Appendix

Table 6: The declared primary comparisons, against the NFM. Positive values favour the NFM except for tie hedging, where a negative value means the other model hedges less. Bold: significant after Holm’s correction over the eight comparisons (Holm’s adjusted values are monotone in the ordered

p

, so the two error rows share one value). The intervals come from the bootstrap draws of this family and can differ from those of the same comparison elsewhere in the paper in the second decimal.

Samples per case $n$	20
Iterations $T$	15
Model width $d_{\text{model}}$ / summary width $d_{\mathbf{e}}$ / iterator hidden	128 / 64 / 256
Inputs	$y$ normalised by its range $D_{y}$ ; five descriptors, the fifth $\mathrm{clip}_{[-8,8]}(\ln D_{y})$
Local read	Cauchy window at five dyadic widths $2^{s}h$ , gated to $5\cdot 2^{s}$ nearest samples; bandwidth multiplier in $[0.5,4]$ , floor $1/20$ ; mixture exponent $\beta\in[0.25,4]$
Updater	query $98\to 256$ , tokens $131\to 256$ , 4 heads, output projection and layer norm, two-layer network of width 496, read-out to 64; residual $\mathbf{e}+\tanh(\gamma)\,\Delta\mathbf{e}\,\sqrt{64}/\overline{\lVert\Delta\mathbf{e}\rVert}$ , $\gamma$ initialised at 0, running norm with momentum 0.999, gate gradient divided by 64
Activation	learned clamped cubic, leak 0.02 below zero

Appendix

Table 7: NFM training and architecture settings, read from the released training source ( train_source.py in each run folder).

Figure 3: The updater. The query is built from the normalised state, the new position, the step and the local vector; each of the twenty samples becomes a token carrying its fused encoder features, its normalised value, its slope and its offset from the new position. Four attention heads read the tokens; a two-layer network combines the result with the query and emits a 64-dimensional correction, which is added to the state through the gate after normalisation by its running norm. A detached auxiliary head estimates the noise during training only.

Figure 4: The local read. A Cauchy window of width

h=\delta\,\mathrm{bw}

(floor

1/20

) is evaluated at five dyadic widths, each gated to its

k=5\cdot 2^{s}

nearest samples until the gate covers every sample, normalised, and mixed under a power-law envelope with a single learned exponent

\beta

. The mixed weights read the fused per-sample features into a 32-dimensional local vector and three statistics.

Explore similar work

May 29, 2026cs.LG

GLENS: Global Search via Learning from Solver Iterates with Diffusion Models

We consider the problem of generating a large collection of initial guesses for local minima of multimodal non-convex continuous optimization problems. The goal is for these initial guesses to be high-quality (i.e., a numerical solver converges quickly) and diverse (i.e., represent many different local minima). Identifying multiple locally optimal solutions enables flexible downstream decision-making, but typically requires expensive global search. Existing data-driven methods predict initial guesses using only the final converged optima from offline solver runs, which discards information about the local neighborhoods of solutions and limits the available training data. We propose GLENS (Global Search via Learning from Solver Iterates), a data-efficient global search method that leverages intermediate solver iterates as free data augmentation. GLENS consists of two components: a neighborhood structure model that uses diffusion models to learn the local geometry around optima conditioned on problem parameters, and a solver behavior model that learns refinement directions to further guide samples towards nearby optima during diffusion sampling. Experiments on modified non-convex benchmark problems and a two-robot obstacle-avoidance navigation problem show that GLENS generates high-quality initial guesses while preserving the multimodal distribution of diverse local optima. The resulting initial guesses lead to faster solver convergence across different problem settings and solvers. We also analyze how key hyperparameter choices affect the performance.

Anjian Li, Bartolomeo Stellato, Ryne Beeson

Department of Electrical and Computer Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Operations Research and Financial Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Mechanical and Aerospace Engineering, Princeton University, Princeton, 08544, NJ, USA.

Large Neighborhood Search Convex Optimization Solver Iterations Diffusion Models Obstacle Avoidance

Ask PDF

May 4, 2026stat.ML

Black-box optimization of noisy functions with unknown smoothness

We study the problem of black-box optimization of a function f of any dimension, given function evaluations perturbed by noise. The function is assumed to be locally smooth around one of its global optima, but this smoothness is unknown. Our contribution is an adaptive optimization algorithm, POO or parallel optimistic optimization, that is able to deal with this setting. POO performs almost as well as the best known algorithms requiring the knowledge of the smoothness. Furthermore, POO works for a larger class of functions than what was previously considered, especially for functions that are difficult to optimize, in a very precise sense. We provide a finite-time analysis of POO's performance, which shows that its error after n evaluations is at most a factor of sqrt(ln n) away from the error of the best known optimization algorithms using the knowledge of the smoothness.

Jean-Bastien Grill, Michal Valko, Rémi Munos

SequeL team, INRIA Lille - Nord Europe, France · Google DeepMind, UK∗

Black-Box Optimization Smoothness Noisy Data

Ask PDF

May 13, 2026math.OC

Stochastic global optimization of continuous functions via random walks on Grassmannians

We introduce a stochastic global optimization method based on random walks on Grassmannian manifolds. To minimize a continuous objective

\ell:\mathbb{R}^d\rightarrow\mathbb{R}

, the method repeatedly samples random

k

-dimensional linear subspaces (with

k\ll d

), solves the resulting low-dimensional restrictions of these problems to these subspaces using an arbitrary black-box optimizer, and updates the iterate (which monotonically improves upon the previous iterate). Unlike classical optimization analyses that rely on convexity, smoothness, Lipschitz bounds, or Polyak-Lojasiewicz-type conditions, our convergence guarantees depend only on the geometric distribution of restricted minima across the

k

-dimensional subspaces passing through a given point in

\mathbb{R}^d

. We identify a gap parameter -- an analogue of a spectral gap for random walks -- that controls the rate at which the iterates approach the global minimum value. Finally, we argue that the same analysis yields a blind-spot robustness property: sufficiently narrow, deep dips of the loss function (small-measure regions where

\ell

spikes downward) have limited influence on the algorithm's trajectory, since they are unlikely to be encountered by random subspace sampling.

Kartik Gupta, Stephen D. Miller, Pradeep Ravikumar +1

Stochastic Optimization Flat Minima Low-Dimensional Structure Subsampling Continuum

Ask PDF