Match the Geometry Before the Surrogate: Extreme Low-Budget Optimization in High Dimensions
Organizations: Faculty of Nuclear Sciences and Physical Engineering, Czech Technical University in Prague · Institute of Information Theory and Automation, Czech Academy of Sciences, Prague
Abstract
Low-budget molecular optimization is often framed as Bayesian optimization in a learned latent space, but standard benchmarks typically allow hundreds or thousands of evaluations, whereas a realistic wet-lab round may allow only a few dozen. We revisit this regime using a SELFIES molecular variational autoencoder (VAE) with a Gaussian prior and GuacaMol objectives at N = 30 evaluations. The main finding is not a new surrogate model, but an interaction in candidate generation. The encoder means of a reference set of molecules concentrate near a narrow spherical shell, while many high-dimensional optimizers propose candidates from boxes or box-shaped trust regions. We study a simple constraint, the on-sphere scaffold, that restricts candidate generation to this shell. We then introduce two minimal optimizers to probe what remains useful once the support is fixed. GeoWalk performs geodesic local search on the shell without a surrogate. After 30 evaluations, it shows no detected difference from a Gaussian-process (GP) surrogate on the shell, which is ahead earlier in the budget. GeoCMA adds covariance learning in a random subspace. It does not improve on GeoWalk in the molecular benchmark. On LassoBench it has a lower mean loss than GeoWalk, but full-dimensional CMA-ES remains stronger on most tasks. A factorial experiment with a single GP crosses the candidate support (shell or box) with the acquisition maximizer (a finite candidate set or a continuous optimizer). In this benchmark, shell candidates outperformed box candidates when acquisition was maximized over a candidate set, but not with continuous maximization by L-BFGS-B. These results come from a controlled study of one Gaussian-prior VAE benchmark and do not recommend VAEs for molecular optimization. The practical lesson is to control the candidate support and the acquisition maximizer before attributing a gain to the surrogate.
Figures & tables
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Implementation | Surrogate | Acquisition, maximizer | Support, trust region |
|---|---|---|---|---|
| Methods of this paper (GuacaMol, LassoBench and NAS-Bench-101) | ||||
| GeoWalk and its variants | this paper | none | 16 uniform draws, then geodesic steps, angle 0.60 to 0.15 rad | shell of radius (NAS: mean embedding norm; LassoBench: clipped) |
| GeoCMA and its variants | this paper; CMA-ES from pycma | none (CMA-ES) | 16-dim. subspace, population 5, step 0.5 (NAS: sd of cold starts) | shell of radius (NAS: whole space, mean norm; LassoBench: box) |
| Random search, on the shell and on other supports | this paper | none | uniform draws; Gaussian fit: draws; NAS: 30 table entries | shell, PCA sphere, convex combinations of latents, Gaussian; LassoBench: |
| GP recipes adapted in this repository | ||||
| Matérn-5/2 GP (DSP prior) | own BoTorch GP; adapts Hvarfner et al. 2024 | Matérn-5/2, ARD, DSP lengthscale prior, MAP fit | qLogEI; argmax of 2,048 draws or L-BFGS-B | shell, box , or cube directions at |
Molecular Optimization
Molecular optimization computationally designs or edits chemical structures to improve desired properties while respecting chemical and practical constraints.
See all 115 papers