Gaussian Process Regression

Also known as GP

Ask about this topic

Overview

Gaussian process regression (GPR) places a probability distribution over functions, with kernels encoding assumptions such as smoothness, periodicity, or shared structure; conditioning on observations yields predictions and uncertainty, especially useful when data are scarce or decisions depend on confidence. Research develops richer models—including nonstationary, additive, multi-output, deep, and physics-informed GPs—and applies them to time series, spatial fields, dynamical systems, scientific surrogates, and Bayesian optimization. Because exact inference scales cubically with sample count, scalable approaches use inducing points and variational inference, nearest-neighbor or Vecchia approximations, sparse and low-rank kernels, and streaming or distributed updates. A central tension is balancing expressive models and calibrated uncertainty against computational cost and sensitivity to kernel and hyperparameter choices; reliable uncertainty under misspecification and scalable high-dimensional inference remain open challenges.

Latest papers

CardsList

111 papers14 papers in the last four weeks, down 7% on the four weeks before. 0.1% of all new papers.

Mar 23, 2026

Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Data

Supervised machine learning describes the practice of fitting a parameterized model to labeled input-output data. Supervised machine learning methods have demonstrated promise in learning efficient surrogate models that can (partially) replace expensive high-fidelity models, making many-query analyses, such as optimization, uncertainty quantification, and inference, tractable. However, when training data must be obtained through the evaluation of an expensive model or experiment, the amount of training data that can be obtained is often limited, which can make learned surrogate models unreliable. In many engineering and scientific settings, cheaper low-fidelity models may be available, for example arising from simplified physics modeling or coarse grids. These models may be used to generate additional low-fidelity training data. The goal of multifidelity machine learning is to use both high- and low-fidelity training data to learn a surrogate model which is cheaper to evaluate than the high-fidelity model, but more accurate than any available low-fidelity model. This work proposes a new multifidelity training approach for Gaussian process regression which uses low-fidelity data to define additional features that augment the input space of the learned model. Similarly to cokriging estimators, the proposed approach conditions the high-fidelity surrogate model on the predictions of all available low-fidelity surrogate models, while benefiting from the computational efficiency of autoregressive estimators. Numerical experiments on several test problems demonstrate both increased predictive accuracy and reduced computational cost relative to the state of the art.

Atticus Rex, Elizabeth Qian, David Peterson

School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA · School of Computational Science & Engineering, Georgia Institute of Technology, Atlanta, GA · Air Force Research Lab, Wright-Patterson AFB, Dayton, Ohio

Surrogate ModelingMultifidelity ModelingGaussian Process Regression
AskPDF
Mar 22, 2026

Time-adaptive infinite-dimensional Gaussian process regression on manifolds

This paper proposes a new formulation of functional Gaussian Process regression on manifolds, based on an Empirical Bayes approach, in the spatiotemporal random field context. We apply the machinery of tight Gaussian measures in separable Hilbert spaces, exploiting the invariance property of covariance kernels under the group of isometries of the manifold. The identification via characteristic function of these measures with the infinite product of one-dimensional Gaussian measures is then obtained, in terms of the eigenfunctions of the Laplace-Beltrami operator on the manifold. The involved time-varying angular spectrum constitutes the key tool for dimension reduction in the implementation of this regression approach, adopting a suitable truncation scheme depending on the functional sample size. The simulation study and synthetic data application illustrate the performance of the proposed functional regression predictor.

MD Ruiz-Medina, AE Madrid, A Torres-Signes +1

Empirical BayesBayesian InferenceGaussian ProcessesGaussian Process Regression
AskPDF
Mar 20, 2026

Infinite-dimensional spherical-radial decomposition for probabilistic functions, with application to constrained optimal control and Gaussian process regression

The spherical-radial decomposition (SRD) is an efficient method for estimating probabilistic functions and their gradients defined over finite-dimensional elliptical distributions. In this work, we generalize the SRD to infinite stochastic dimensions by combining subspace SRD with standard Monte Carlo methods. The resulting method, which we call hybrid infinite-dimensional SRD (hiSRD) provides an unbiased, low-variance estimator for convex sets arising, for instance, in chance-constrained optimization. We provide a theoretical analysis of the variance of finite-dimensional SRD as the dimension increases, and show that the proposed hybrid method eliminates truncation-induced bias, reduces variance, and allows the computation of derivatives of probabilistic functions. We present comprehensive numerical studies for a risk-neutral stochastic PDE optimal control problem with joint chance state constraints, and for optimizing kernel parameters in Gaussian process regression under the constraint that the posterior process satisfies joint chance constraints.

Kewei Wang, Georg Stadler

Courant Institute School of Mathematics, Computing and Data Science, New York University, New York, USA

Chance-Constrained OptimizationStochastic Optimal ControlGaussian Process Regression
AskPDF
Mar 18, 2026

On Additive Gaussian Processes for Wind Farm Power Prediction

Population-based Structural Health Monitoring (PBSHM) aims to share information between similar machines or structures. This paper takes a population-level perspective, exploring the use of additive Gaussian processes to reveal variations in turbine-specific and farm-level power models over a collected wind farm dataset. The predictions illustrate patterns in wind farm power generation, which follow intuition and should enable more informed control and decision-making.

Simon M. Brealy, Lawrence A. Bull, Daniel S. Brennan +4

Dynamics Research Group, Department of Mechanical Engineering, The University of Sheffield, Mappin Street, Sheffield, S1 3JD, UK · Computational Statistics and Machine Learning Group, Department of Engineering, University of Cambridge, Cambridge, CB3 0FA, UK · Vattenfall R&D, Vattenfall AB, Älvkarleby, Sweden

Wind Power ForecastingGaussian ProcessesGaussian Process Regression
AskPDF
Mar 18, 2026

Real-Time Online Learning for Model Predictive Control using a Spatio-Temporal Gaussian Process Approximation

Learning-based model predictive control (MPC) can enhance control performance by correcting for model inaccuracies, enabling more precise state trajectory predictions than traditional MPC. A common approach is to model unknown residual dynamics as a Gaussian process (GP), which leverages data and also provides an estimate of the associated uncertainty. However, the high computational cost of online learning poses a major challenge for real-time GP-MPC applications. This work presents an efficient implementation of an approximate spatio-temporal GP model, offering online learning at constant computational complexity. It is optimized for GP-MPC, where it enables improved control performance by learning more accurate system dynamics online in real-time, even for time-varying systems. The performance of the proposed method is demonstrated by simulations and hardware experiments in the exemplary application of autonomous miniature racing.

Lars Bartels, Amon Lahr, Andrea Carron +1

Institute for Dynamic Systems and Control, ETH Zürich, CH-8092 Zürich, Switzerland.

Model Predictive ControlGaussian ProcessesGaussian Process RegressionResidual Dynamics LearningRobot Dynamics Learning
AskPDF
Mar 18, 2026

Wasserstein-type Gaussian Process Regressions for Input Measurement Uncertainty

Gaussian process (GP) regression is widely used for uncertainty quantification, yet the standard formulation assumes noise-free covariates. When inputs are measured with error, this errors-in-variables (EIV) setting can lead to optimistically narrow posterior intervals and biased decisions. We study GP regression under input measurement uncertainty by representing each noisy input as a probability measure and defining covariance through Wasserstein distances between these measures. Building on this perspective, we instantiate a deterministic projected Wasserstein ARD (PWA) kernel whose one-dimensional components admit closed-form expressions and whose product structure yields a scalable, positive-definite kernel on distributions. Unlike latent-input GP models, PWA-based GPs (\PWAGPs) handle input noise without introducing unobserved covariates or Monte Carlo projections, making uncertainty quantification more transparent and robust.

Hengrui Luo, Xiaoye S. Li, Yang Liu +3

Department of Statistics Rice University, Houston, TX 77005 · Lawrence Berkeley National Laboratory, Berkeley, CA 94709

Gaussian Process RegressionUncertainty QuantificationKernel Methods
AskPDF
Mar 17, 2026

Optimal uncertainty bounds for multivariate kernel regression under bounded noise: A Gaussian process-based dual function

Non-conservative uncertainty bounds are essential for making reliable predictions about latent functions from noisy data, and thus, a key enabler for safe learning-based control. In this domain, kernel methods such as Gaussian process regression are established techniques, thanks to their inherent uncertainty quantification mechanism. Still, existing bounds either pose strong assumptions on the underlying noise distribution, are conservative, do not directly apply in the multi-output case, or are difficult to integrate into downstream tasks. This paper addresses these limitations by presenting a tight, deterministic bound for multi-output functions in Reproducing Kernel Hilbert Spaces (RKHSs) subject to bounded noise. It is obtained through an unconstrained, duality-based formulation, which shares the same structure as classic Gaussian process confidence bounds, and can thus be straightforwardly integrated into downstream optimization pipelines. We show that the proposed bound generalizes existing results and illustrate its application using an example inspired by quadrotor dynamics learning.

Amon Lahr, Anna Scampicchio, Johannes Köhler +1

Institute for Dynamical Systems and Control, ETH Zurich, Zurich, Switzerland

Kernel MethodsGaussian Process Regression
AskPDF
Mar 16, 2026

Kriging via variably scaled kernels

Classical Gaussian processes and Kriging models are commonly based on stationary kernels, whereby correlations between observations depend exclusively on the relative distance between scattered data. While this assumption ensures analytical tractability, it limits the ability of Gaussian processes to represent heterogeneous correlation structures. In this work, we investigate variably scaled kernels as an effective tool for constructing non-stationary Gaussian processes by explicitly modifying the correlation structure of the data. Through a scaling function, variably scaled kernels alter the correlations between data and enable the modeling of targets exhibiting abrupt changes or discontinuities. We analyse the resulting predictive uncertainty via the variably scaled kernel power function and clarify the relationship between variably scaled kernels-based constructions and classical non-stationary kernels. Numerical experiments demonstrate that variably scaled kernels-based Gaussian processes yield improved reconstruction accuracy and provide uncertainty estimates that reflect the underlying structure of the data

Gianluca Audone, Francesco Marchetti, Emma Perracchione +1

Faculty of Engineering and Physical Sciences, University of Southampton, United Kindom · Dipartimento di Matematica “Tullio Levi-Civita”, Universit`a di Padova, Italy · Dipartimento di Scienze Matematiche Giuseppe Luigi Lagrange, Politecnico di Torino, Italy +1

Gaussian ProcessesGaussian Process RegressionKernel MethodsUncertainty Quantification
AskPDF
Mar 12, 2026

Gaussian Process Regression-based Knowledge Distillation Framework for Simultaneous Prediction of Physical and Mechanical Properties of Epoxy Polymers

Epoxy polymers are widely used due to their multifunctional properties, but machine learning (ML) applications remain limited owing to their complex 3D molecular structure, multi-component nature, and lack of curated datasets. Existing ML studies are largely restricted to simulation data, specific properties, or narrow constituent ranges. To address these limitations, we developed an informed Gaussian Process Regression-based Knowledge Distillation (GPR-KD) framework for predicting multiple physical (glass transition temperature, density) and mechanical properties (elastic modulus, tensile strength, compressive strength, flexural strength, fracture energy, adhesive strength) of thermoset epoxy polymers. The model was trained on experimental literature data covering diverse monomer classes (9 resins, 40 hardeners). Individual GPR models serve as teacher models capturing nonlinear feature-property relationships, while a unified neural network student model learns distilled knowledge across all properties simultaneously. By encoding the target property as an input feature, the student model leverages cross-property correlations. Molecular-level descriptors extracted from SMILES representations using RDKit create a physics-informed model. The framework combines GPR interpretability and robustness with deep learning scalability and generalization. Comparative analysis demonstrates superior prediction accuracy over conventional ML models. Simultaneous multi-property prediction further improves accuracy through information sharing across correlated properties. The proposed framework enables accelerated design of novel epoxy polymers with tailored properties.

Sindu B. S., Jan Hamaekers

Special and Multifunctional Structures Laboratory, CSIR-Structural Engineering Research Centre, Taramani, Chennai, Tamil Nadu, India - 600113. · Fraunhofer Institute for Algorithms and Scientific Computing SCAI, Schloss Birlinghoven, 53757 Sankt Augustin, Germany

Materials Property PredictionPolymer Property PredictionGaussian Process RegressionKnowledge DistillationMulti-Task Learning
AskPDF
Mar 10, 2026

A Unified Hierarchical Multi-Task Multi-Fidelity Framework for Data-Efficient Surrogate Modeling in Manufacturing

Surrogate modeling is an essential data-driven technique for quantifying relationships between input variables and system responses in manufacturing and engineering systems. Two major challenges limit its effectiveness: (1) large data requirements for learning complex nonlinear relationships, and (2) heterogeneous data collected from sources with varying fidelity levels. Multi-task learning (MTL) addresses the first challenge by enabling information sharing across related processes, while multi-fidelity modeling addresses the second by accounting for fidelity-dependent uncertainty. However, existing approaches typically address these challenges separately, and no unified framework simultaneously leverages inter-task similarity and fidelity-dependent data characteristics. This paper develops a novel hierarchical multi-task multi-fidelity (H-MT-MF) framework for Gaussian process-based surrogate modeling. The proposed framework decomposes each task's response into a task-specific global trend and a residual local variability component that is jointly learned across tasks using a hierarchical Bayesian formulation. The framework accommodates an arbitrary number of tasks, design points, and fidelity levels while providing predictive uncertainty quantification. We demonstrate the effectiveness of the proposed method using a 1D synthetic example and a real-world engine surface shape prediction case study. Compared to (1) a state-of-the-art MTL model that does not account for fidelity information and (2) a stochastic kriging model that learns tasks independently, the proposed approach improves prediction accuracy by up to 19% and 23%, respectively. The H-MT-MF framework provides a general and extensible solution for surrogate modeling in manufacturing systems characterized by heterogeneous data sources.

Manan Mehta, Zhiqiao Dong, Yuhang Yang +1

Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, United States · Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI 41809, United States

Surrogate ModelingMultifidelity ModelingMulti-Task LearningGaussian Process Regression
AskPDF
Mar 2, 2026

Instrumental and Proximal Causal Inference with Gaussian Processes

Instrumental variable (IV) and proximal causal learning (Proxy) methods are central frameworks for causal inference in the presence of unobserved confounding. Despite substantial methodological advances, existing approaches rarely provide reliable epistemic uncertainty (EU) quantification. We address this gap through a Deconditional Gaussian Process (DGP) framework for uncertainty-aware causal learning. Our formulation recovers popular kernel estimators as the posterior mean, ensuring predictive precision, while the posterior variance yields principled and well-calibrated EU. Moreover, the probabilistic structure enables systematic model selection via marginal log-likelihood optimization. Empirical results demonstrate strong predictive performance alongside informative EU quantification, evaluated via empirical coverage frequencies and decision-aware accuracy rejection curves. Together, our approach provides a unified, practical solution for causal inference under unobserved confounding with reliable uncertainty.

Yuqi Zhang, Krikamol Muandet, Dino Sejdinovic +2

Department of Statistics and Actuarial Science, University of Hong Kong, Hong Kong · Rational Intelligence Lab, CISPA Helmholtz Center for Information Security, Germany · Australian Institute for Machine Learning, Adelaide University, Australia +1

Proximal Causal InferenceCausal InferenceCausal Inference with Unobserved ConfoundingInstrumental Variable EstimationEpistemic UncertaintyGaussian Process Regression
AskPDF
Feb 27, 2026

Scalable Gaussian process modeling of parametrized spatio-temporal fields

We introduce a scalable Gaussian process (GP) framework with deep product kernels for data-driven learning of parametrized spatio-temporal fields over fixed or parameter-dependent domains. The proposed framework learns a continuous representation, enabling predictions at arbitrary spatio-temporal coordinates, independent of the training data resolution. We leverage Kronecker matrix algebra to formulate a computationally efficient training procedure with complexity that scales nearly linearly with the total number of spatio-temporal grid points. A key feature of our approach is the efficient computation of the posterior variance at essentially the same computational cost as the posterior mean (exactly for Cartesian grids and via rigorous bounds for unstructured grids), thereby enabling scalable uncertainty quantification. Numerical studies on a range of benchmark problems demonstrate that the proposed method achieves accuracy competitive with operator learning methods such as Fourier neural operators and deep operator networks. On the one-dimensional unsteady Burgers' equation, our method surpasses the accuracy of projection-based reduced-order models. These results establish the proposed framework as an effective tool for data-driven surrogate modeling, particularly when uncertainty estimates are required for downstream tasks.

Srinath Dama, Prasanth B. Nair

University of Toronto Institute for Aerospace Studies, 4925 Dufferin Street, Toronto, M3H 5T6, Ontario, Canada

Gaussian ProcessesGaussian Process RegressionPDE Surrogate ModelingUncertainty Quantification
AskPDF
Feb 26, 2026

From Shallow Bayesian Neural Networks to Gaussian Processes: General Convergence, Identifiability and Scalable Inference

In this work, we study scaling limits of shallow Bayesian neural networks (BNNs) via their connection to Gaussian processes (GPs), with an emphasis on statistical modeling, identifiability, and scalable inference. We first establish a general convergence result from BNNs to GPs by relaxing assumptions used in prior formulations, and we compare alternative parameterizations of the limiting GP model. Building on this theory, we propose a new covariance function defined as a convex mixture of components induced by four widely used activation functions, and we characterize key properties including positive definiteness and both strict and practical identifiability under different input designs. For computation, we develop a scalable maximum a posterior (MAP) training and prediction procedure using a Nyström approximation, and we show how the Nyström rank and anchor selection control the cost-accuracy trade-off. Experiments on controlled simulations and real-world tabular datasets demonstrate stable hyperparameter estimates and competitive predictive performance at realistic computational cost.

Gracielle Antunes de Araújo, Flávio B. Gonçalves

Universidade do Estado de Minas Gerais (UEMG), FaEnge – Faculdade de Engenharia. · Universidade Federal de Minas Gerais (UFMG), ICEx – Instituto de Ciências Exatas.

Nyström MethodBayesian Neural NetworksShallow Neural NetworksGaussian ProcessesGaussian Process Regression
AskPDF
Feb 24, 2026

Generative Bayesian Computation as a Scalable Alternative to Gaussian Process Surrogates

Gaussian process (GP) surrogates are the default tool for emulating expensive computer experiments, but cubic cost, stationarity assumptions, and Gaussian predictive distributions limit their reach. We propose Generative Bayesian Computation (GBC) via Implicit Quantile Networks (IQNs) as a surrogate framework that targets all three limitations. GBC learns the full conditional quantile function from input--output pairs; at test time, a single forward pass per quantile level produces draws from the predictive distribution. Across fourteen benchmarks we compare GBC to four GP-based methods. GBC improves CRPS by 11--26% on piecewise jump-process benchmarks, by 14% on a ten-dimensional Friedman function, and scales linearly to 90,000 training points where dense-covariance GPs are infeasible. A boundary-augmented variant matches or outperforms Modular Jump GPs on two-dimensional jump datasets (up to 46% CRPS improvement). In active learning, a randomized-prior IQN ensemble achieves nearly three times lower RMSE than deep GP active learning on Rocket LGBB. Overall, GBC records a favorable point estimate in 12 of 14 comparisons. GPs retain an edge on smooth surfaces where their smoothness prior provides effective regularization.

Nick Polson, Vadim Sokolov

Booth School of Business, University of Chicago. · Department of Systems Engineering and Operations Research, George Mason University.

Surrogate ModelingGaussian Process Regression
AskPDF
Feb 24, 2026

MAST: A Multi-fidelity Augmented Surrogate model via Spatial Trust-weighting

In engineering design and scientific computing, computational cost and predictive accuracy are intrinsically coupled. High-fidelity simulations provide accurate predictions but at substantial computational costs, while lower-fidelity approximations offer efficiency at the expense of accuracy. Multi-fidelity surrogate modelling addresses this trade-off by combining abundant low-fidelity data with sparse high-fidelity observations. However, existing methods rely on global correlation assumptions that can often fail in practice to capture how fidelity relationships vary across the input space, leading to poor performance, particularly under tight budget constraints. We introduce MAST, a method that blends corrected low-fidelity observations with high-fidelity predictions, trusting high-fidelity near observed samples and relying on corrected low-fidelity elsewhere. MAST achieves this through explicit discrepancy modelling and distance-based weighting with closed-form variance propagation, producing a single heteroscedastic Gaussian process. Across multi-fidelity synthetic benchmarks, MAST shows a marked improvement over the current state-of-the-art techniques. Crucially, MAST maintains robust performance across varying total budget and fidelity gaps, conditions under which competing methods exhibit significant degradation or unstable behaviour. More broadly, MAST provides a spatially adaptive framework for multi-fidelity Gaussian-process modelling, in which the contribution of low-fidelity information is governed by its proximity to high-fidelity calibration data, opening a new direction for more reliable surrogate construction under sparse and budget-constrained settings.

Ahmed Mohamed Eisa Nasr, Ali Elham, Haris Moazam Sheikh

Department of Aeronautics and Astronautics, University of Southampton, SO17 1BJ, Southampton, United Kingdom

Surrogate ModelingMultifidelity ModelingGaussian Process Regression
AskPDF
Feb 20, 2026

Quasi-Periodic Gaussian Process Predictive Iterative Learning Control

Repetitive motion tasks are common in robotics, but performance can degrade over time due to environmental changes and robot wear and tear. Iterative learning control (ILC) improves performance by using information from previous iterations to compensate for expected errors in future iterations. This work incorporates the use of Quasi-Periodic Gaussian Processes (QPGPs) into a predictive ILC framework to model and forecast disturbances and drift across iterations. Using a recent structural equation formulation of QPGPs, the proposed approach enables efficient inference with complexity O(p3)\mathcal{O}(p^3) instead of O(i2p3)\mathcal{O}(i^2p^3), where pp denotes the number of points within an iteration and ii represents the total number of iterations, specially for larger ii. This formulation also enables parameter estimation without loss of information, making continual GP learning computationally feasible within the control loop. By predicting next-iteration error profiles rather than relying only on past errors, the controller achieves faster convergence and maintains this under time-varying disturbances. We benchmark the method against both standard ILC and conventional Gaussian Process (GP)-based predictive ILC on three tasks, autonomous vehicle trajectory tracking, a three-link robotic manipulator, and a real-world Stretch robot experiment. Across all cases, the proposed approach converges faster and remains robust under injected and natural disturbances while reducing computational cost. This highlights its practicality across a range of repetitive dynamical systems.

Unnati Nigam, Radhendushka Srivastava, Faezeh Marzbanrad +1

IITB-Monash Research Academy, IIT Bombay, Mumbai, India · Department of Mathematics, IIT Bombay, Mumbai, India · Department of Electrical and Computer Systems Engineering, Monash University, Clayton, Melbourne, Australia

Iterative Learning ControlRobotic ControlGaussian ProcessesAdaptive ControlGaussian Process RegressionRobot Trajectory Tracking
AskPDF
Feb 18, 2026

Nonplanar Model Predictive Control for Autonomous Vehicles with Recursive Sparse Gaussian Process Dynamics

This paper proposes a nonplanar model predictive control (MPC) framework for autonomous vehicles operating on nonplanar terrain. To approximate complex vehicle dynamics in such environments, we develop a geometry-aware modeling approach that learns a residual Gaussian Process (GP). By utilizing a recursive sparse GP, the framework enables real-time adaptation to varying terrain geometry. The effectiveness of the learned model is demonstrated in a reference-tracking task using a Model Predictive Path Integral (MPPI) controller. Validation within a custom Isaac Sim environment confirms the framework's capability to maintain high tracking accuracy on challenging 3D surfaces.

Ahmad Amine, Kabir Puri, Viet-Anh Le +1

Department of Electrical & Systems Engineering, University of Pennsylvania, Philadelphia, PA 19104, USA

Model Predictive ControlResidual Dynamics LearningGaussian Process Regression
AskPDF
Feb 2, 2026

Correcting Boundary Bias and Observation Independence in Bayesian Experimental Design

In many experimental settings, active learning can improve sample efficiency by sequentially selecting where to measure, which is particularly valuable when experiments are expensive. Gaussian processes with variance-based acquisition criteria are widely used for this purpose, but have two limitations. First, they are observation-independent: their posterior variance depends only on where samples are acquired, not on what is measured, impairing their sensitivity to the structure of the acquired data. Second, they inflate the variance near boundaries, leading to excessive sampling at the edges of the space compared to the interior. These limitations undermine the gains in sampling efficiency expected from sequential acquisition. We address both limitations. We derive a reconstruction-driven design density and use the posterior mean to build a training-free warp that places more measurements where the target function varies rapidly. A geometric equalizer separately corrects boundary bias. Across sixteen synthetic and two real-data benchmarks, the geometric equalizer consistently improves function reconstruction by correcting boundary bias, while the reconstruction warp provides further gains by concentrating measurements where the posterior mean varies rapidly.

Sanna Jarl, Jens Sjölund, Jonathan J. S. Scragg +1

Department of Material Science & Engineering Uppsala University RISE · Department of Information Technology Uppsala University · RISE

Active LearningAdaptive SamplingBayesian Experimental DesignGaussian Process RegressionActive Experimental Design
AskPDF
Nov 29, 2025

No-Regret Gaussian Process Optimization of Time-Varying Functions

Sequential optimization of black-box functions from noisy evaluations has been widely studied, with Gaussian Process bandit algorithms such as GP-UCB guaranteeing no-regret in stationary settings. However, for time-varying objectives, no-regret is unattainable under pure bandit feedback unless strong and often unrealistic assumptions are imposed. We propose a novel method for optimizing time-varying rewards in the frequentist setting, where the objective has bounded RKHS norm almost surely. Time variations are captured through uncertainty injection, enabling heteroscedastic Gaussian process regression that adapts past observations to the current time step. As no-regret is unattainable in general in the strict bandit setting, we relax the latter allowing additional queries on previously observed points. Building on sparse inference and the effect of uncertainty injection on regret, we propose W-SparQ-GP-UCB, an online algorithm that achieves no-regret with a vanishing number of additional queries per iteration. To assess the theoretical limits of this approach, we establish a lower bound on the number of additional queries required for no-regret, proving the efficiency of our method. Finally, we provide a comprehensive analysis linking the temporal regime of the function to achievable regret rates, together with upper and lower bounds on the number of additional queries needed in each regime.

Eliabelle Mauduit, Eloïse Berthier, Andrea Simonetto

Unit´e de Math´ematiques Appliqu´ees ENSTA, Institut Polytechnique de Paris Palaiseau, France · U2IS ENSTA, Institut Polytechnique de Paris Palaiseau, France

Multi-Armed BanditsBayesian OptimizationGaussian Process BanditsRegret MinimizationGaussian Process Regression
AskPDF
Oct 24, 2025

Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functions

We introduce Deep Jump Gaussian Processes (DJGP), a novel method for surrogate modeling of a piecewise continuous function on a high-dimensional domain. DJGP addresses the limitations of conventional Jump Gaussian Processes (JGP) in high-dimensional input spaces by integrating region-specific, locally linear projections with JGP modeling. These projections employ region-dependent matrices to capture local low-dimensional subspace structures, making them well suited to the inherently localized modeling behavior of JGPs, a variant of local Gaussian processes. To control model complexity, we place a Gaussian Process prior on the projection matrices, allowing them to evolve smoothly across the input space. The projected inputs are then modeled with a JGP to capture piecewise continuous relationships with the response. This yields a distinctive two-layer deep learning of GP/JGP. We further develop a scalable variational inference algorithm to jointly learn the projection matrices and JGP hyperparameters. Rigorous theoretical analysis and extensive empirical studies are provided to justify the proposed approach. In particular, we derive an oracle error bound for DJGP and decompose it into four distinct sources of error, which are then linked to practical implications. Experiments on synthetic and benchmark datasets demonstrate that DJGP achieves superior predictive accuracy and more reliable uncertainty quantification compared with existing methods.

Yang Xu, Chiwoo Park

Industrial & Systems Engineering, University of Washington, Seattle, WA, USA

Deep Gaussian ProcessesSurrogate ModelingGaussian ProcessesGaussian Process RegressionUncertainty Quantification
AskPDF
Jun 20, 2025

Gaussian Processes and Reproducing Kernel Hilbert Spaces: Connections and Equivalences

This monograph studies the relations between two approaches using positive definite kernels: probabilistic methods using Gaussian processes, and non-probabilistic methods using reproducing kernel Hilbert spaces (RKHS). They are widely studied and used in machine learning, statistics, and numerical analysis. We study connections and equivalences for fundamental topics such as regression, interpolation, numerical integration, distributional discrepancies, and statistical dependence, as well as sample path properties of Gaussian processes. A unifying perspective for these equivalences is established, based on the equivalence between the Gaussian Hilbert space and the RKHS. The monograph serves as a basis to bridge many other methods based on Gaussian processes and reproducing kernels, which are developed in parallel by the two research communities.

Motonobu Kanagawa, Philipp Hennig, Dino Sejdinovic +1

EURECOM · University of Tübingen, Tübingen AI Center Max Planck Institute for Intelligent Systems · Adelaide University Australian Institute for Machine Learning +1

Gaussian ProcessesReproducing Kernel Hilbert SpacesKernel MethodsGaussian Process Regression
AskPDF