SABLE: Scalable Affine Bayesian Latent-space Exploration

Erik Bodin; Lisa Maria Kreusser; Donney Fan; Henry Moss; Geoff Pleiss; Nicolas Durrande

cs.LGOct 22, 2026

SABLE: Scalable Affine Bayesian Latent-space Exploration

Authors: Erik Bodin, Lisa Maria Kreusser, Donney Fan, Henry Moss, Geoff Pleiss, Nicolas Durrande

Organizations: Monumo · University of British Columbia · Lancaster University · Vector Institute

Abstract

Complex objects such as engineering designs and proteins are often difficult to optimise because their design spaces exhibit high-dimensional structure and objective evaluations are expensive. Generative models provide continuous latent representations, but their latent spaces can have thousands to tens of thousands of dimensions, creating a regime where the latent dimension $D$ greatly exceeds the evaluation budget. We introduce SABLE, a Bayesian optimisation method for high-dimensional latent spaces with an isotropic distribution and limited budget. SABLE uses a Bayesian affine surrogate for prediction and acquisition, treating higher-order structure as surrogate discrepancy. After $N$ observations, the slope update has dimension at most $N-1$ , yielding a decomposition of each query into an in-span component optimised by expected improvement (EI) in at most $N$ dimensions and an off-span component governed by a scalar norm. We show that the off-span norm implicitly selected by affine EI can lead to underexploration, and SABLE instead prescribes it using a finite-horizon schedule, without restricting search to a learned subspace. A recompression scheme reduces information to $\mathcal O(M)$ pseudo-observations, bounding refitting cost by $\mathcal O(M^2D+M^3)$ while keeping parameter inference, prediction, and acquisition decisions to tens of milliseconds. Across synthetic benchmarks and latent-space optimisation tasks, SABLE achieves high sample efficiency and reduces decision time by several orders of magnitude at large budgets.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of SABLE. (a) Each query is decomposed into an in-span component and an orthogonal off-span component. EI chooses the in-span direction, while the exploration schedule prescribes the off-span norm. Because the design span $\mathcal{V}_{N}$ has dimension at most $N$ , EI optimisation requires at most $N$ coordinates, independently of $D$ . (b) As observations accumulate, their representation is recompressed to $\mathcal{O}(M)$ pseudo-observations. Strong information is retained and weaker information is summarised, making model-refitting cost independent of $N$ for fixed $M$ .

Panel of Figure 2 — Figure 2: Effect of state compression. Left: best observed value for compression budgets $M\in\{10,100,300,500\}$ at $D=10{,}000$ , compared with the uncapped representation (dashed) and the mean global maximum of the ten objective draws. Lines show means over ten runs (and objective draws) and bands show $5$ th– $95$ th percentiles. Right: measured wall-clock time per decision.

		$\cos(\hat{\bm{s}},\bm{v}_{\star})$				attained value
		affine	mild	mod.	severe	affine	mild	mod.	severe
Standard EI (no schedule)	MLE	1.00	-0.00	-0.01	-0.01	1.00	0.48	0.32	0.16
	MAP	0.78	0.44	0.30	0.16	0.78	0.60	0.54	0.50
	Marg.	0.77	0.44	0.29	0.19	0.77	0.59	0.54	0.49
naive model-derived schedule: $\lambda_{k}=\nicefrac{{\sigma_{1,k}^{2}}}{{\sigma_{\varepsilon,k}^{2}}}$	MLE	0.13	0.00	0.00	0.01	0.49	0.49	0.36	0.23
	MAP	0.98	0.59	0.41	0.23	0.98	0.67	0.55	0.45

Table 1: Model fitting and off-span exploration at

N=D=1000

with the uncapped representation. Left: recovery of the known degree-one component,

\cos(\hat{\bm{s}},\bm{v}_{\star})

. Right: attained value: maximum of the best observed point and the posterior-mean recommendation. Entries are means over ten seeds. Bold: the column best and every entry within two standard deviations.

Figure 3: Controlled long-horizon evaluation, $D=N=10{,}000$ . Lines show mean best observed value over ten runs, with $5$ th– $95$ th percentile bands. Markers at $N=10{,}000$ show attained value, the better of the best observed point and the posterior-mean recommendation.

Figure 4: Scientific latent-space optimisation. Table: best observed value, reported as mean and $5$ th– $95$ th percentile range over ten seeds; bold: within two standard errors of the column best. † : truncated at 48 h, reaching $N=303$ – $1{,}132$ . Bottom: best observed value against wall-clock time.

Figure 5: High-budget DS3 optimisation . Best observed value against evaluations; thick lines are medians and thin lines the individual runs.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Problem	$D$	Generative model	Generated object	Objective	Prior median
Turbulence	$6{,}000$	diffusion-lagr	velocity trajectory	DNS distribution match	$0.57$
Peptides	$13{,}440$	AMP-Diffusion	peptide sequence	hydrophobic moment	$0.34$
Calorimeter DS2	$6{,}480$	CaloDiffusion	shower voxel grid	energy-deposition match	$0.07$
Calorimeter DS3	$40{,}500$	CaloDiffusion	shower voxel grid	energy-deposition match	$0.05$

Appendix

Table 2: Generative-model optimisation problems. “Prior median” is the median objective value obtained from independent draws of the generative model’s latent prior

\mathcal{N}(\bm{0},I_{D})

. Datasets 2 and 3 form a matched calorimeter pair at different detector discretisations.

		$\cos(\hat{\bm{s}},\bm{v}_{\star})$				attained value
		affine	mild	mod.	severe	affine	mild	mod.	severe
no schedule	MLE	1.00	-0.00	-0.01	-0.01	1.00	0.48	0.32	0.16
	MAP	0.78	0.44	0.30	0.16	0.78	0.60	0.54	0.50
	Marg.	0.77	0.44	0.29	0.19	0.77	0.59	0.54	0.49
$\lambda_{\mathrm{ref}}$ from model	MLE	0.13	0.00	0.00	0.01	0.49	0.49	0.36	0.23
	MAP	0.98	0.59	0.41	0.23	0.98	0.67	0.55	0.45

Appendix

Table 3: Complete schedule-by-treatment grid, noise-free , at

N=D=1000

with the uncapped posterior. Left: recovery of the known degree-one component,

\cos(\hat{\bm{s}},\bm{v}_{\star})

. Right: attained value, the better of the best observed point and the posterior-mean recommendation. Means over ten paired seeds. Bold: the column best and every entry within two standard deviations of it. Table 1 shows the no-schedule, model-derived, and

\lambda_{\mathrm{ref}}=10^{5}

rows.

		$\cos(\hat{\bm{s}},\bm{v}_{\star})$				attained value
		affine	mild	mod.	severe	affine	mild	mod.	severe
no schedule	MLE	0.01	0.02	0.01	0.01	0.37	0.26	0.19	0.13
	MAP	0.14	0.10	0.07	0.06	0.15	0.15	0.15	0.16
	Marg.	0.16	0.11	0.08	0.05	0.16	0.16	0.16	0.15
$\lambda_{\mathrm{ref}}$ from model	MLE	-0.00	-0.00	-0.00	-0.01	0.48	0.33	0.24	0.17
	MAP	0.36	0.27	0.19	0.11	0.36	0.30	0.25	0.21

Appendix

Table 4: Complete schedule-by-treatment grid, noisy observations (

\sigma_{\varepsilon}=0.05

), at

N=D=1000

with the uncapped posterior; layout and bold rule as in Table 3 . Best-so-far and recommendations are scored on the noiseless objective.

Figure 6: Per-step diagnostics, marginalised variance scales, noise-free. $N=D=1000$ , uncapped posterior, ten paired seeds (mean and $5$ th– $95$ th percentiles). Rows: best-so-far, value of the posterior-mean recommendation $F(\hat{\bm{s}})$ , degree-one recovery $\cos(\hat{\bm{s}},\bm{v}_{\star})$ , off-span norm $t$ of the selected query, fitted amplitude $\hat{a}=\sqrt{D}\,\hat{\sigma}_{1}$ , and fitted discrepancy scale $\hat{\sigma}_{\varepsilon}$ . Columns: the affine control and the mild, moderate, and severe non-affine objectives. Curves: no schedule (dashed), the model-derived reference (dash-dotted), and the fixed ladder $\lambda_{\mathrm{ref}}\in\{10^{0},10^{1},10^{3},10^{5}\}$ .

Figure 7: Per-step diagnostics, MAP variance scales, noise-free. Layout as in Figure 6 .

Figure 8: Per-step diagnostics, maximum-likelihood variance scales, noise-free. Layout as in Figure 6 . Without a schedule the fitted discrepancy scale collapses and the off-span norm stays near one for the whole run.

Figure 9: Per-step diagnostics, marginalised variance scales, noisy observations ( $\sigma_{\varepsilon}=0.05$ ). Layout as in Figure 6 ; best-so-far and the recommendation are scored on the noiseless objective.

Figure 10: Per-step diagnostics, MAP variance scales, noisy observations ( $\sigma_{\varepsilon}=0.05$ ). Layout as in Figure 6 .

Figure 11: Per-step diagnostics, maximum-likelihood variance scales, noisy observations ( $\sigma_{\varepsilon}=0.05$ ). Layout as in Figure 6 .

	affine	mild	mod.	severe
Comparisons
Random design + posterior argmax	0.93 [0.92, 0.93]	0.46 [0.45, 0.47]	0.31 [0.29, 0.33]	0.17 [0.16, 0.19]
Orthogonal design + posterior argmax	1.00 [1.00, 1.00]	0.45 [0.42, 0.47]	0.28 [0.25, 0.31]	0.13 [0.12, 0.14]
sep-CMA-ES	0.60 [0.59, 0.61]	0.50 [0.49, 0.51]	0.48 [0.46, 0.49]	0.45 [0.44, 0.46]
Thompson sampling	0.79 [0.78, 0.79]	0.49 [0.48, 0.51]	0.35 [0.30, 0.38]	0.20 [0.11, 0.27]
Ours (marginalised, $M=300$ )

Appendix

Table 5: Attained value at

D=N=10{,}000

M=300

: the better of the best observed point and the posterior-mean recommendation, mean and

5

–

95\%

over ten paired seeds. Bold: the column best and all within two standard deviations. The model-free baselines expose no recommendation and report their best observed point. Prescribing the off-span schedule beats every comparison on every non-affine objective.

Figure 12: Scientific latent-space optimisation against evaluation count. Best observed value against evaluation count for the runs of Figure 4 ; lines show means and bands $5$ th– $95$ th percentiles over ten seeds. Standard BO and Linear BO timed out at $48$ h of wall clock.

Bayesian Optimization

Bayesian optimization uses probabilistic models and sequential experiments to find strong solutions when evaluating a black-box objective is costly.

See all 240 papers

Search at the Cost of Sampling: Nearly-Instant Latent Space Bayesian Optimization
Sep 16, 2026Donney Fan, Colin Doumont, Aleksandra Kalisz +4Surrogate Modeling Bayesian Optimization
University of British Columbia · Vector Institute · Tübingen AI Center +3
Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization
Oct 17, 2025Luo Long, Coralia Cartis, Paz Fink ShustinVariational Autoencoders Bayesian Optimization
Mathematical Institute, University of Oxford, Radcliffe Observatory, Andrew Wiles Building, Woodstock Rd, Oxford, OX2 6GG, United Kingdom.
Work While They Sleep: Exploiting Evaluation Latency for Fully Bayesian Optimization
Oct 6, 2026Gustavo Sutter, Alejandro Comas-Leon, David Holzmüller +4Bayesian Optimization Surrogate-Assisted Optimization
Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, Canada · Vector Institute, Toronto, ON, Canada · SODA Team, INRIA Saclay, Palaiseau, France +3

Sep 16, 2026J/K move · Enter open · S save

Search at the Cost of Sampling: Nearly-Instant Latent Space Bayesian Optimization

Donney Fan, Colin Doumont, Aleksandra Kalisz, Paul Duckworth+3

University of British Columbia · Vector Institute · Tübingen AI Center+3

Surrogate Modeling Bayesian Optimization Latent Space Optimization

Open paper AskPDF arXiv

Generative models are increasingly central to many de novo discovery pipelines, in which designs are generated at scale and filtered through virtual screens to determine a set of candidates to experimentally validate. While Bayesian optimization (BO) is a natural fit for this setting, as it uses past evaluations to guide future proposals, the computational overhead required for its sequential decision-making becomes a bottleneck when virtual screens are relatively cheap. We make BO practical in this regime by exploiting the unique combination of a linear model constrained to a spherical domain where high-dimensional latents concentrate. We build off recent work justifying the use of linear surrogates, while deriving nearly closed-form solutions to the surrogate modelling and acquisition problems that exploit spherical symmetry. The result is at least a 100x speedup over state-of-the art baselines, with matching or improved performance across molecular and image generation benchmarks. Altogether, our method makes BO a practical drop-in for de novo pipelines where it was previously too slow to consider.

Abstract

Bayesian Optimization

Explore similar work

Search at the Cost of Sampling: Nearly-Instant Latent Space Bayesian Optimization

Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization

Work While They Sleep: Exploiting Evaluation Latency for Fully Bayesian Optimization