One-Shot Localisation of the Global Minimum of a Noisy One-Dimensional Function: An Iterative Neural Minimizer Compared with Set Transformers and Classical Estimators

Qusay Muzaffar; David Levin; Michael Werman

cs.LGApr 4, 2026

One-Shot Localisation of the Global Minimum of a Noisy One-Dimensional Function: An Iterative Neural Minimizer Compared with Set Transformers and Classical Estimators

Authors: Qusay Muzaffar, David Levin, Michael Werman

Organizations: Department of Computer Science, The Hebrew University of Jerusalem, Israel · Department of Applied Mathematics, Tel Aviv University, Israel

Abstract

We study a passive form of global optimisation: from twenty noisy samples of an unknown one-dimensional function, predict where its global minimum lies, with no further queries. We introduce the Neural Function Minimizer (NFM), an iterative model that walks a position across the domain and, at every step, reads the samples near that position and attends to all twenty of them, and we compare it with two Set Transformers of the same size trained on the same data, one that answers with a single point and one that answers with a mixture of candidate locations, and with classical zero-query estimators, over three training seeds per learned model. On held-out cases from the training families the three learned models are tied on location error, and each is more accurate on average than every classical estimator; against the Gaussian-process posterior argmin the NFM's error is lower by 1.3 points of the domain. The NFM has lower regret than the single-point Set Transformer, and its per-case uncertainty has a better likelihood than that model's and orders the cases by error better than the mixture model's. The Gaussian process and the mixture model have lower regret, and on functions outside the training families the Gaussian process is the most accurate estimator. Where two valleys are equally deep, the form of an estimator's answer, not its architecture, decides whether it commits to a valley or answers between them: every estimator that returns one point fitted to distance, learned or spline-based, hedges in a fifth to nearly a third of exact ties, while estimators that name a mode commit, and one Gaussian-process posterior does both, depending on whether it is summarised by its median or its mode. We will release the benchmark and a self-checking harness; its cases are identical on different machines and its results agree to rounding.

Figures & tables

Figure 1: The NFM. Top: the four per-sample channels (with

y

normalised to

[0,1]

) are lifted, fused position-wise by a U-Net, pooled into the state

\mathbf{e}

, normalised, and conditioned once on the five global descriptors

g

by FiLM. Bottom, repeated

T=15

times: the local read returns a Cauchy-windowed summary of the samples around

x_{t}

(Figure 4 ); the iterator maps state, position, previous step,

g

and the local read to a bounded direction and a bounded step; the updater attends to the twenty samples from the new position and adds a correction to the state through a zero-initialised gate (Figure 3 ). The answer is

x_{T}

; the spread head reads the final state. Blue dots mark the normalisation and residual gate of Section 4.3 .

Method	Mean	Median	Right basin	Regret	Method $-$ NFM, error	Method $-$ NFM, regret
NFM (3 seeds)	6.72 $\pm$ 0.07	2.31	82.4%	12.80	—	—
NFM, per-case updater (ablation)	6.79 $\pm$ 0.32	2.59	82.3%	13.89	$+0.07$ [ $-0.22$ , $+0.43$ ]	$+1.09$ [ $+0.36$ , $+1.91$ ] ∗
ST (3 seeds)	6.89 $\pm$ 0.22	2.81	82.7%	14.90	$+0.18$ [ $-0.10$ , $+0.44$ ]	$+2.10$ [ $+1.18$ , $+2.92$ ] ∗
MH-ST (3 seeds)	6.99 $\pm$ 0.41	1.64	81.6%	8.46	$+0.27$ [ $-0.18$ , $+0.69$ ]	$-4.33$ [ $-5.08$ , $-3.56$ ] ∗
GP posterior argmin	8.02	1.40	78.8%	5.53	$+1.30$ [ $+0.99$ , $+1.62$ ] ∗	$-7.27$ [ $-7.79$ , $-6.75$ ] ∗
GP argmin posterior, median	7.79	1.66	78.6%	7.54	$+1.08$ [ $+0.78$ , $+1.37$ ] ∗	$-5.26$ [ $-5.81$ , $-4.70$ ] ∗

Table 1: Location error (percent of the domain) and regret (percent of the function’s range) on the 5,200 in-family cases (13 families

\times

\lambda\in\{0,0.25,0.5,1\}

\times

100). Every method answers from the same 20 samples of the same case. Learned models are means over three training seeds (

\pm

the standard deviation of the seed means). The last two columns are paired differences, the row’s method minus the NFM, with 95% intervals from a bootstrap that resamples seeds within each model and cases within each (family, level) cell; positive values favour the NFM, and ∗ marks an interval that excludes zero. Bold marks the best value in each column. Every surrogate argmin is taken between the first and last sample. The median of the NFM, ST and GP answers (not shown) reaches 6.18 (Section 6.8 ).

		Exact tie (gap 0%)			Hedging, gap
Form of the answer	Estimator	Hedging	In a floor	Capped regret	2%	10%
One point, fitted to distance	NFM (3 seeds)	29.2%	36.5%	18.55	31.3%	25.2%
	NFM, per-case updater (3 seeds)	24.2%	33.9%	14.72	26.2%	21.8%
	ST (3 seeds)	29.2%	38.2%	19.32	30.1%	20.5%
Minimum of a smoothed curve	Spline argmin	24.0%	40.8%	14.58	25.2%	24.0%
	Differential evolution on spline	20.0%	45.5%	11.63	21.5%	20.0%

Table 2: Behaviour at tied minima, grouped by the form of the answer: the harness’s tie set, double wells whose two wells differ by 0, 2 or 10% of the barrier, at four noise levels (400 cases per gap). Hedging : the answer is more than 5% of the domain from every valley floor. In a floor : within 2% of one. Capped regret :

f(\hat{x})-f(x^{*})

in percent of the barrier, with any value above one barrier counted as one barrier. Paired differences over all tie cases, with tie shapes resampled: ST

-

NFM hedging

-2.0

[

-14.6

+8.9

] points, capped regret

-0.90

[

-8.61

+5.98

]; MH-ST

-

NFM

-27.8

[

-40.0

-18.0

] ∗ and

-17.27

[

-24.84

-11.28

] ∗ . Bold marks the best value in each column.

Figure 2: Where the answers land on the tied minima of Griewank: all 600 cases (five windows

\times

six noise levels

\times

20 repeats) and all three seeds of each model, mapped back to the native coordinate, against the function (grey). Dotted lines mark valley floors (the global one at 0 and the two nearly equal ones at

\pm 2\pi

), dashed lines the ridges between them. The ST’s answers (top) spread across the slopes between the valleys; the NFM’s (middle) concentrate more on the floors; the MH-ST’s (bottom) almost never lie on a slope, but in the outer valleys often lie beyond the floor.

Model / reading	$\bar{s}/\overline{\|e\|}$	dev	PIT	ENCE	NLL	CRPS	$\rho(s,\|e\|)$	E-AURC
NFM	1.13	0.045	0.044	0.115	$-1.36$	5.25	0.60	1.43
NFM, per-case updater (ablation)	1.03	0.027	0.020	0.073	$-1.30$	5.22	0.58	1.53
ST, Huber read-back	1.03	0.033	0.028	0.093	$-1.26$	5.27	0.53	1.59
ST, as produced	0.77	0.104	0.051	0.587	$-1.11$	5.31	0.53	1.59
MH-ST, derived spread	1.05	0.037	0.055	0.083	$-1.33$	5.71	0.46	1.74
MH-ST, full mixture	—	—	0.044	—	$-$ 1.91	4.87	—	—

Table 3: Per-case uncertainty on the 5,200 in-family cases, every spread read as the scale

s

of a Laplace distribution centred on the answer. Targets:

\bar{s}/\overline{|e|}=1

; coverage of

|e|\leq s,2s,3s

63.2

86.5

and

95.0\%

, whose mean absolute deviation is dev ; PIT and ENCE 0. NLL is the Laplace negative log-likelihood; CRPS is in percent of the domain; E-AURC is the excess area under the risk–coverage curve, in percent of the domain. The ST’s head was trained on the Huber value of its error; Huber read-back converts it exactly to the Laplace scale. The MH-ST’s derived spread is

\mathbb{E}|X-\hat{x}|

under its mixture. The GP’s spread is the mean distance of its 256 argmin draws from its answer, floored at 0.01 like the learned models’ heads; the full posterior is sample-based, so it has CRPS and PIT but no NLL. Intervals as in Table 1 , 2,000 draws. After one cross-fitted scale factor (

\times 0.90

for the NFM), the NFM’s dev is level with every learned model’s and its NLL remains better than the ST’s (

+0.10

∗ ).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Weight	Basins	Multi-basin	Gap	$d/R$ , median [10%, 90%]	Range $R$	$w_{b}$	Resolvable	Tie cases
Bowl	0.09	1 (1)	0.0%	—	1.34 [0.00, 42.9]	0.98	0.706	100.0%	0
Polynomial	0.08	2 (4)	97.9%	16.5	15.6 [0.27, 93.4]	0.94	0.254	100.0%	28
Periodic	0.09	5 (17)	100.0%	7.2	7.24 [1.23, 24.9]	3.65	0.097	95.1%	8
Regular periodic	0.09	6 (9)	100.0%	4.3	4.25 [1.92, 10.1]	5.00	0.127	99.3%	0
Double well	0.10	2 (2)	100.0%	0.3	0.32 [0.03, 3.5]	46.02	0.693	100.0%	99
Multi-well	0.08	2 (4)	86.8%	32.3	32.3 [2.85, 99.8]	2.39	0.071	95.8%	5

Appendix

Table 4: Properties of the thirteen families, measured on 1,872 draws from the training generator (144 per family, 24 at each noise level). Basins : median (maximum) number of watershed basins of

f

G_{3000}

. Multi-basin : share of cases with more than one basin. Gap : median height of the runner-up basin’s floor above the global minimum, in percent of

R

(undefined for a single basin).

d/R

: well depth in percent of the range, median and 10th and 90th percentiles; the median differs by a factor of about 176 between double wells and deceptive functions, which is why the noise is scaled to

d

and not to

R

. Range

R

: median, in the function’s own units.

w_{b}

: median width of the global basin at 10% of the range. Resolvable : a sample lies within

\max(w_{b},0.06)

x^{*}

. Tie cases : gap below 1%.

Unseen (never used in development)
Function	$f(x)$	Domain
Forrester	$(6x-2)^{2}\sin(12x-4)$	$[0,1]$
Gramacy–Lee	$\sin(10\pi x)/(2x)+(x-1)^{4}$	$[0.5,2.5]$
Ackley	$-20e^{-0.2\|x\|}-e^{\cos 2\pi x}+20+e$	$[-4,4]$
Michalewicz	$-\sin x\,\sin^{20}(x^{2}/\pi)$	$[0,\pi]$
Shubert	$\sum_{j=1}^{5}j\cos\big((j+1)x+j\big)$	$[-2,4]$

Appendix

Table 5: The out-of-family functions, in their native form and domain.

Measure	Other model	Other $-$ NFM [95% CI]	$p$	Holm $p$
In-family error	ST	$+0.18$ [ $-0.10$ , $+0.44$ ]	0.21	0.61
In-family error	MH-ST	$+0.27$ [ $-0.19$ , $+0.70$ ]	0.22	0.61
In-family error	GP posterior argmin	$+1.30$ [ $+1.00$ , $+1.60$ ]	$<$ 0.0002	0.0016
In-family regret	ST	$+2.10$ [ $+1.21$ , $+2.93$ ]	$<$ 0.0002	0.0016
Tie hedging (points)	ST	$-2.0$ [ $-14.4$ , $+8.8$ ]	0.76	0.76
NLL	ST, read-back	$+0.10$ [ $+0.04$ , $+0.16$ ]	0.0005	0.003

Appendix

Table 6: The declared primary comparisons, against the NFM. Positive values favour the NFM except for tie hedging, where a negative value means the other model hedges less. Bold: significant after Holm’s correction over the eight comparisons (Holm’s adjusted values are monotone in the ordered

p

, so the two error rows share one value). The intervals come from the bootstrap draws of this family and can differ from those of the same comparison elsewhere in the paper in the second decimal.

Samples per case $n$	20
Iterations $T$	15
Model width $d_{\text{model}}$ / summary width $d_{\mathbf{e}}$ / iterator hidden	128 / 64 / 256
Inputs	$y$ normalised by its range $D_{y}$ ; five descriptors, the fifth $\mathrm{clip}_{[-8,8]}(\ln D_{y})$
Local read	Cauchy window at five dyadic widths $2^{s}h$ , gated to $5\cdot 2^{s}$ nearest samples; bandwidth multiplier in $[0.5,4]$ , floor $1/20$ ; mixture exponent $\beta\in[0.25,4]$
Updater	query $98\to 256$ , tokens $131\to 256$ , 4 heads, output projection and layer norm, two-layer network of width 496, read-out to 64; residual $\mathbf{e}+\tanh(\gamma)\,\Delta\mathbf{e}\,\sqrt{64}/\overline{\lVert\Delta\mathbf{e}\rVert}$ , $\gamma$ initialised at 0, running norm with momentum 0.999, gate gradient divided by 64
Activation	learned clamped cubic, leak 0.02 below zero

Appendix

Table 7: NFM training and architecture settings, read from the released training source ( train_source.py in each run folder).

Figure 3: The updater. The query is built from the normalised state, the new position, the step and the local vector; each of the twenty samples becomes a token carrying its fused encoder features, its normalised value, its slope and its offset from the new position. Four attention heads read the tokens; a two-layer network combines the result with the query and emits a 64-dimensional correction, which is added to the state through the gate after normalisation by its running norm. A detached auxiliary head estimates the noise during training only.

Figure 4: The local read. A Cauchy window of width

h=\delta\,\mathrm{bw}

(floor

1/20

) is evaluated at five dyadic widths, each gated to its

k=5\cdot 2^{s}

nearest samples until the gate covers every sample, normalised, and mixed under a power-law envelope with a single learned exponent

\beta

. The mixed weights read the fused per-sample features into a 32-dimensional local vector and three statistics.

Explore similar work

GLENS: Global Search via Learning from Solver Iterates with Diffusion Models
May 29, 2026Anjian Li, Bartolomeo Stellato, Ryne BeesonLarge Neighborhood Search Convex Optimization
Department of Electrical and Computer Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Operations Research and Financial Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Mechanical and Aerospace Engineering, Princeton University, Princeton, 08544, NJ, USA.
Black-box optimization of noisy functions with unknown smoothness
May 4, 2026Jean-Bastien Grill, Michal Valko, Rémi MunosBlack-Box Optimization Smoothness
SequeL team, INRIA Lille - Nord Europe, France · Google DeepMind, UK∗
Stochastic global optimization of continuous functions via random walks on Grassmannians
May 13, 2026Kartik Gupta, Stephen D. Miller, Pradeep Ravikumar +1Stochastic Optimization Flat Minima

May 29, 2026, cs.LGJ/K move · Enter open · S save

GLENS: Global Search via Learning from Solver Iterates with Diffusion Models

Anjian Li, Bartolomeo Stellato, Ryne Beeson

Department of Electrical and Computer Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Operations Research and Financial Engineering, Princeton University, Princeton, 08544, NJ, USA. · Department of Mechanical and Aerospace Engineering, Princeton University, Princeton, 08544, NJ, USA.

Large Neighborhood Search Convex Optimization Solver Iterations Diffusion Models Obstacle Avoidance

Open paper AskPDF arXiv

We consider the problem of generating a large collection of initial guesses for local minima of multimodal non-convex continuous optimization problems. The goal is for these initial guesses to be high-quality (i.e., a numerical solver converges quickly) and diverse (i.e., represent many different local minima). Identifying multiple locally optimal solutions enables flexible downstream decision-making, but typically requires expensive global search. Existing data-driven methods predict initial guesses using only the final converged optima from offline solver runs, which discards information about the local neighborhoods of solutions and limits the available training data. We propose GLENS (Global Search via Learning from Solver Iterates), a data-efficient global search method that leverages intermediate solver iterates as free data augmentation. GLENS consists of two components: a neighborhood structure model that uses diffusion models to learn the local geometry around optima conditioned on problem parameters, and a solver behavior model that learns refinement directions to further guide samples towards nearby optima during diffusion sampling. Experiments on modified non-convex benchmark problems and a two-robot obstacle-avoidance navigation problem show that GLENS generates high-quality initial guesses while preserving the multimodal distribution of diverse local optima. The resulting initial guesses lead to faster solver convergence across different problem settings and solvers. We also analyze how key hyperparameter choices affect the performance.

Abstract

Explore similar work

GLENS: Global Search via Learning from Solver Iterates with Diffusion Models

Black-box optimization of noisy functions with unknown smoothness

Stochastic global optimization of continuous functions via random walks on Grassmannians