Online Gradient Computation for Warping Gaussian Process Transformations

Emilio Ruiz-Moreno; Konstantinos Slavakis; Baltasar Beferull-Lozano

cs.LGSep 15, 2026

Online Gradient Computation for Warping Gaussian Process Transformations

Authors: Emilio Ruiz-Moreno, Konstantinos Slavakis, Baltasar Beferull-Lozano

Abstract

Warped Gaussian processes (GPs) handle non-Gaussian observations by mapping them into a latent standard GP via a parametric transformation called warping. Existing streaming variants, however, either optimize the warping parameters periodically or sacrifice analytical tractability for a higher model capacity. To bridge this gap, we show that the gradient of the instantaneous negative log-likelihood of a warped GP admits an exact recursive computation. Based on this result, we propose a novel online method for warped GPs that jointly updates the latent GP moments and optimizes the warping parameters.

Explore similar work

Warm-Starting Iterative Gaussian Processes for Faster Sequential Inference
Nov 20, 2025Alan Yufei Dong, Jihao Andreas Lin, José Miguel Hernández-LobatoGaussian Process Cold-Start
Scalable Derivative Gaussian Processes via Exact Gradient Reduction
Jun 1, 2026Hyunseok Seung, Matthias KatzfussGaussian Process Gradient Descent
Conditioning Gaussian Processes on Almost Anything
May 20, 2026Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite +5Gaussian Process 3D Gaussian Splatting

Nov 20, 2025 · cs.LGJ/K move · Enter open · S save

Warm-Starting Iterative Gaussian Processes for Faster Sequential Inference

Alan Yufei Dong, Jihao Andreas Lin, José Miguel Hernández-Lobato

Department of Engineering, University of Cambridge

Efficient Gaussian process (GP) inference is critical for sequential decision-making tasks such as active learning, online prediction, and Bayesian optimization. Iterative approaches of approximating the GP posterior using solvers like conjugate gradients, stochastic gradient descent, or alternating projections avoid cubic costs, but often require many iterations to converge, limiting their efficacy when the posterior is updated frequently with new data. To address this, we introduce three warm-start strategies that exploit solutions of smaller linear systems to substantially speed-up convergence when updating the posterior with new data. Our methods are supported by theoretical analysis showing reduced initialization error in reproducing kernel Hilbert space (RKHS) distance, and by empirical results on regression benchmarks and Bayesian optimization tasks. Across solvers, warm-starting achieves speed-ups of up to 19x when solving to tolerance, and produces more accurate posterior estimates under fixed compute budgets, directly improving optimization performance. These results establish warm-starting as a simple, effective, and broadly applicable tool for scaling Gaussian processes in sequential settings.

Gaussian Process Cold-Start Bayesian Optimization Posterior Spectral Preconditioning

Open paper Ask about it PDF

Abstract

Explore similar work

Warm-Starting Iterative Gaussian Processes for Faster Sequential Inference

Scalable Derivative Gaussian Processes via Exact Gradient Reduction

Conditioning Gaussian Processes on Almost Anything