Bayesian Optimization

Bayesian optimization studies how to find good inputs to an expensive or unknown process—such as a simulation, experiment, or model configuration—using as few evaluations as possible. It typically fits a probabilistic surrogate, often a Gaussian process, then uses an acquisition function to balance trying promising inputs with exploring uncertain ones. Current approaches extend this loop with trust regions, multi-fidelity data, constraints, multiple objectives, changing conditions, and human or language-model guidance. A central challenge is choosing models and search strategies that remain reliable in high-dimensional or structured spaces without making inference and candidate selection too costly; theoretical guarantees do not always predict practical performance.