LAGO: A Local-Global Optimization Framework Combining Trust Region Methods and Bayesian Optimization

Eliott Van Dieren; Tommaso Vanzan; Fabio Nobile

cs.LGMar 3, 2026

LAGO: A Local-Global Optimization Framework Combining Trust Region Methods and Bayesian Optimization

Authors: Eliott Van Dieren, Tommaso Vanzan, Fabio Nobile

Organizations: Institute of Mathematics, EPFL, Lausanne, Switzerland · Dipartimento di Scienze Matematiche, Politecnico di Torino, Turin, Italy

Abstract

We introduce LAGO, a LocAl-Global Optimization framework coupling Bayesian Optimization (BO) and gradient-based trust region local refinement through an adaptive competition mechanism for smooth expensive-to-evaluate objective functions with available gradients. At each iteration, global and local optimization strategies independently propose candidate points, and the next evaluation is selected based on predicted improvement. LAGO separates global exploration from local refinement at the proposal level: the BO acquisition function is optimized outside the active trust region, while local candidates are proposed within the trust region. Points in the vicinity of the accepted local step are incorporated in the global GP dataset only when satisfying a lengthscale-based minimum-distance criterion, hence reducing the risk of numerical instability during local exploitation. LAGO enhances BO with efficient local refinement when reaching promising regions, and reverts to exploratory behavior when local steps are not competitive.

Figures & tables

Figure 1: One LAGO iteration. The TR is centered at the current best minimizer. A local candidate is proposed inside the region, while a global candidate is proposed outside. The shaded contours are the GP posterior mean outside of the TR, and the quadratic surrogate inside it.

\mathbf{x}_{G}\leftarrow\argmax_{\mathbf{x}\in\mathcal{X}\setminus B(\mathbf{x}_{c},\Delta)}\operatorname{EI}(\mathbf{x}).

Algorithm 1 LAGO: Local-Global Framework Combining BO and SR1-TR.

Panel of Figure 2 — Figure 2: Convergence comparison. Convergence of optimization routines on a PDE-constrained optimization problem and synthetic benchmarks (Median $\pm$ IQR). LAGO performs consistently well across benchmarks, both convex and multimodal ones. It also performs strongly in higher dimensional settings, where other hybrid methods start struggling. Gradient evaluations cost $d$ function evaluations, except for the PDE example where gradients have unit cost (leveraging adjoints).

Figure 3: RQ1. Performance comparison of LAGO and phase-switching methods on synthetic benchmarks (Median

\pm

IQR error). LAGO exhibits more robust performance, particularly on benchmarks where phase-based switching fails to identify the correct basin, leading to high variance.

Figure 4: RQ2. Convergence of LAGO (Median

\pm

IQR) for different values of

\nu

on Lévy (left), and increase in the kernel matrix condition number ratio (Median

\pm

IQR) when

\nu=0

, shown over a window of

\pm 5

iterations around the iteration where the first trigger of ( 7 ) happens (right). Setting

\nu

too high delays convergence, while setting it to

0

increases the ill-conditioning of the GP.

Figure 5: RQ3. Comparison of LAGO and LAGO-grad, together with their BO and gradBO counterparts (median

\pm

IQR, shaded). LAGO-grad improves over LAGO primarily on ill-conditioned problems such as Rosenbrock but can underperform in multimodal settings such as Lévy.

Figure 6: RQ4. Median optimization error (

\pm

IQR) on Rastrigin and Griewank (2D), and Styblinski-Tang (10D). LAGO-BO closely matches BO, indicating negligible overhead in highly multimodal regimes, and outperforms BO in 10D on this example. This confirms that LAGO adapts to the problem structure and reverts to BO-like behavior when exploration dominates.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Objective	Definition $f:\mathbb{R}^{d}\to\mathbb{R}$	Domain	Global minimum
Branin	$\left(\mathbf{x}_{2}-\frac{5.1}{4\pi^{2}}\mathbf{x}_{1}^{2}+\frac{5}{\pi}\mathbf{x}_{1}-6\right)^{2}+10\left(1-\frac{1}{8\pi}\right)\cos(\mathbf{x}_{1})+10$	$[-5,10]\times[0,15]$	$0.39789$
Styblinski-Tang	$\frac{1}{2}\sum_{i=1}^{d}\left(\mathbf{x}_{i}^{4}-16\mathbf{x}_{i}^{2}+5\mathbf{x}_{i}\right)$	$[-5,5]^{d}$	$-39.16599d$
Rosenbrock	$(1-\mathbf{x}_{1})^{2}+100(\mathbf{x}_{2}-\mathbf{x}_{1}^{2})^{2}$	$[-5,10]^{2}$	$0.0$
Lévy	$\sin^{2}(\pi w_{1})+(w_{1}-1)^{2}\left[1+10\sin^{2}(\pi w_{1}+1)\right]$	$[-10,10]^{2}$	$0.0$
	$+(w_{2}-1)^{2}\left[1+\sin^{2}(2\pi w_{2})\right],\;w_{i}=1+\frac{\mathbf{x}_{i}-1}{4}$
Perturbed Branin	$\operatorname{Branin}(\mathbf{x})+10^{-6}\\|\mathbf{x}-[-\pi,12.275]\\|^{2}$	$[-5,10]\times[0,15]$	$0.39789$

Appendix

Table 1: Information about the synthetic test functions used in Section 4 .

Panel of Figure 7 — Figure 7: Contour and surface plots of the 2D synthetic test functions. The red stars show the global minima. Surface plots are used for highly multimodal functions for clarity purposes, and the Rastrigin one is zoomed in a smaller domain $[-5,5]^{2}$ .

Figure 8: Landscape of the reduced cost function

J

for the PDE-constrained optimization problem. The objective exhibits multiple local minima and a distinct global minimizer, highlighting the need for global exploration combined with local refinement.

Panel of Figure 9 — Figure 9: Diffusion coefficient $\kappa$ , desired state $y_{d}$ , optimal forcing term $u$ , and obtained state variable $y$ . The forcing term is approximately placed at the highest point of the desired state $y_{d}$ , as the resulting state $y$ will minimize the $L_{2}$ distance with $y_{d}$ .

Figure 10:

\boldsymbol{\gamma}

-ablation. Performance of LAGO (Median

\pm

IQR) on the Lévy function for different values of

\gamma

in ( 6 ) (default

\gamma=1

). Larger values of

\gamma

, which place greater emphasis on local refinement, lead to lower error on this synthetic benchmark.

Panel of Figure 12 — Figure 12: Local-global interleaving. LAGO trajectories across benchmarks with local (green) and global (blue) steps. LAGO predominantly selects global steps in moderate and highly multimodal settings, while increasing the use of local refinement in convex or weakly multimodal regimes.

Mostly global behavior: local steps $<5\%$
Objective	Local steps (%)
Styblinski-Tang (10D)	1.0 $\pm$ 0.2
Styblinski-Tang (5D)	1.5 $\pm$ 0.3
Griewank (2D)	1.9 $\pm$ 0.9
Lévy (2D)	2.1 $\pm$ 0.5
Rastrigin (2D)	3.9 $\pm$ 2.0

Appendix

Table 3: Percentage of local steps versus global steps across benchmark problems (reported as mean

\pm

standard deviation).

Explore similar work

Local Preferential Bayesian Optimization
Jun 1, 2026Johanna Menn, Miriam Kober, Paul Brunzema +2Pairwise Preference Learning Pairwise Preference Evaluation
Institute for Data Science in Mechanical Engineering, RWTH Aachen University, Aachen, Germany · Department of Clinical Research, University of Bern, Bern, Switzerland · Center for Reproducible Science and Research Synthesis, University of Zurich, Zurich, Switzerland +1
STREGO: A Scalable Implementation of a Trust-Region Framework for Noisy Bayesian Optimization
Oct 9, 2026Youssef Diouane, Issmail El Hallaoui, Meisseme Kadri +1Bayesian Optimization Trust-Region Optimization
GERAD & Department of Mathematics and Industrial Engineering, Polytechnique Montr´eal. · GERAD & Department of Management, Universit´e du Qu´ebec `a Trois-Rivi`eres.
How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO)
Oct 13, 2026Aditya Kulkarni, Davide Sartor, Carlos Soto +1LLM Fine-Tuning Parameter-Efficient Fine-Tuning
Department of Physics, University of Massachusetts Amherst · Department of Mathematics and Statistics, University of Massachusetts Amherst

Jun 1, 2026J/K move · Enter open · S save

Local Preferential Bayesian Optimization

Johanna Menn, Miriam Kober, Paul Brunzema, David Stenger+1

Institute for Data Science in Mechanical Engineering, RWTH Aachen University, Aachen, Germany · Department of Clinical Research, University of Bern, Bern, Switzerland · Center for Reproducible Science and Research Synthesis, University of Zurich, Zurich, Switzerland+1

Pairwise Preference Learning Pairwise Preference Evaluation Bayesian Optimization Local Search Safe Bayesian Optimization Trust-Region Optimization

Open paper AskPDF arXiv

Bayesian optimization (BO) is a popular and effective approach for tuning expensive, noisy experiments, but requires the formulation of an explicit objective function. Preferential BO (PBO) removes this requirement by learning from pairwise human feedback, yet existing methods struggle to efficiently optimize beyond low- and medium-dimensional problems due to their global search approaches. We address this limitation by developing a family of local PBO methods that transfer key ideas from high-dimensional BO to the preferential setting. In particular, we introduce local PBO methods which adapt trust-region and derivative-informed local search to pairwise preference feedback, where the latter exploits first- and second-order derivatives of the Laplace-approximated GP posterior. Our benchmark on GP sample paths, standard optimization benchmark functions, and policy-search tasks shows that local PBO methods are especially effective in high-dimensional and complex landscapes with steep optima. Compared with global preference-based baselines, they can substantially reduce cumulative regret, making them particularly useful for real-world preference-based optimization tasks such as policy search.

Abstract

Explore similar work

Local Preferential Bayesian Optimization

STREGO: A Scalable Implementation of a Trust-Region Framework for Noisy Bayesian Optimization

How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO)