Gaussian Process Regression

Also known as GP

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 67

Jun 19, 2026stat.ML

Orthogonal Discrepancy Kernels for Learning with Partial Physics

We introduce a semi-parametric framework for nonlinear system identification, which decouples discrepancy functions from physics-based components. Orthogonal Gaussian process regression balances sparse parameter selection (the white box) with discrepancy learning (the black box) to produce interpretable models from incomplete physics.

Swapnil Manna, Timothy J. Rogers, Lawrence Bull

IISER Pune, India · University of Sheffield, UK · University of Glasgow, UK

Semiparametric Inference Nonlinear System Identification Model Calibration Physics-Informed ML Gaussian Process Regression

Jun 18, 2026cs.RO

Learning-Based Modeling of Soft Robots via Cosserat Rod Theory

Modeling soft robot dynamics is challenging due to their continuum structure and typically nonlinear dynamics. Creating models based on first-order principles is typically time-demanding, and their expressiveness is limited, whereas data-driven models lack interpretability and physical consistency. This work aims to overcome these challenges by introducing a port-Hamiltonian Gaussian Process Regression framework for learning and simulating the dynamics of planar, rod-like soft robots. In detail, the proposed model integrates Cosserat rod theory and Hamiltonian physics with data-driven inference to preserve the system's energy structure while accurately learning the rod dynamics. Numerical simulations show that we can achieve accurate and energy-consistent representations of a rod-like soft robot, showing the potential for a robust and interpretable pathway for modeling complex continuum mechanics.

Mohammad Ali, Nithin Senthur Kumar, Eric J. Barth +1

Department of Computer Science, Vanderbilt University, Nashville, TN 37235, USA · Department of Mechanical Engineering, Vanderbilt University, Nashville, TN 37235, USA

Dynamical Systems Port-Hamiltonian Systems Soft Robotics Gaussian Process Regression

Jun 16, 2026cs.LG

Geometry-Aware Post-Hoc Uncertainty Quantification in Operator Learning

Neural operators provide fast surrogates for PDEs but their deterministic predictions limit their use in tasks requiring uncertainty quantification (UQ), especially under geometric variability. Existing approaches primarily model uncertainty in network parameters, largely overlooking the geometry-aware representations learned by the operator itself. We propose REEF-GP (Residual on Embedded Features Gaussian Process), a post-hoc UQ framework that fits a GP to the residuals of a frozen neural operator whose internal embeddings define the kernel feature space. Rather than learning a separate feature map, REEF-GP adapts the operator's intrinsic coordinate-feature representations to construct geometry-aware uncertainties. To ensure stability and scalability on unstructured domains, REEF-GP incorporates spectral-normalized projections, heteroscedastic geometry-aware noise, and efficient subset-based training that avoids restrictive low-rank approximations. Across five PDE benchmarks with varying geometries, REEF-GP preserves predictive accuracy while achieving calibrated uncertainty estimates competitive with deep ensembles but at a fraction of their cost. Our approach remains robust under geometric distribution shift, with uncertainty concentrating in physically meaningful regions (e.g., shock fronts). Our results demonstrate that accurate and scalable post-hoc UQ for neural operators can be achieved directly in their learned feature space, offering a practical alternative to parameter-centric approaches.

Oriol Vendrell-Gallart, Nima Negarandeh, Ramin Bostanabad

Department of Mechanical and Aerospace Engineering University of California, Irvine Irvine, CA 92617

Uncertainty Quantification PDE Surrogate Modeling Neural Operators Gaussian Process Regression

Jun 10, 2026cs.LG

Structure-Preserving Neural Surrogates with Tractable Uncertainty Quantification

Recent advances in scientific machine learning provide a means of near-real-time solution to partial differential equations (PDEs), but lack the theoretical underpinnings of conventional simulators that support contemporary verification and validation. In this work, we construct data-driven reduced-order models that serve as structure-preserving, real-time surrogates. Remarkably, the exterior calculus that imposes physical conservation structure also exposes topological structure that we use to build a Gaussian process (GP) representation of uncertainty in state-flux relationships, ultimately yielding a Dirichlet-to-Neumann map for quantities of interest with closed-form expressions for posterior uncertainty. We specifically propose structure-preserving

H(\mathrm{div})

--

L^2

subspaces of conventional Raviart--Thomas and

dgP_0

elements prescribed by a lightweight transformer. Reduced-order dynamics consistent with this subspace are learned by posing a conservation law in which a GP describes the fluxes between volumes. This work hinges on a novel interface between mixed FEM spaces and GP regression; when training is posed as the optimal recovery problem (ORP), the resulting GP regression can be written as an optimization problem with equality constraints that impose a conservation structure, amenable to a fast Schur-complement training strategy. The trained model can then be solved in real time with closed-form estimators for boundary fluxes driven by prescribed Dirichlet data. The paper includes RKHS posterior error bounds for linear functionals to support uncertainty quantification, as well as numerical experiments demonstrating the accuracy of the posterior distribution as a surrogate for error estimation.

Handi Zhang, Adrienne M. Propp, Brooks Kinch +2

Applied Mathematics and Computational Science, University of Pennsylvania, Philadelphia, PA, USA · Institute for Computational and Mathematical Engineering, Stanford University, Stanford, CA, USA · Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, USA +1

Uncertainty Quantification PDE Surrogate Modeling Reduced-Order Modeling Finite Element Method Gaussian Process Regression

Jun 4, 2026cs.LG

Gaussian Process Latent Factor Regression for Low-Data, High-Dimensional Output Problems

In the sciences, regression tasks often require predicting high-dimensional outputs from few training examples. Multi-output Gaussian processes excel in low-data regimes but typically struggle with high-dimensional outputs. Compress-then-predict pipelines such as PCA-GP (principal component analysis plus Gaussian process regression) handle high dimensionality, but rely on bases optimized for reconstruction rather than prediction. To address this gap, we propose a model that represents each output as a linear-Gaussian decoding of a low-dimensional latent state drawn from a Gaussian process prior. By analytically marginalizing the decoder weights, we couple compression and prediction in a single objective that scales to high-dimensional outputs. We refer to this model as Gaussian process latent factor regression (GPLFR). We demonstrate GPLFR by building the first spatially resolved emulator of global climate models for rocky exoplanets.

Edward T. Stevenson, Eric T. Wolf, Mei Ting Mak +2

University of Cambridge · University of Colorado Boulder · University of Oxford +1

Latent Variable Models Dimensionality Reduction Gaussian Process Regression

Jun 1, 2026stat.ML

Scalable Derivative Gaussian Processes via Exact Gradient Reduction

Gradient observations can substantially improve Gaussian process (GP) surrogates, particularly in high-dimensional settings where function evaluations are expensive. However, exact inference with

n

function values and

n

full gradients in

d

dimensions scales cubically in the joint state size, imposing an intractable

\mathcal{O}(n^3 d^3)

computational bottleneck. We introduce TERA, a highly scalable derivative GP method based on target-specific exact gradient reduction. We prove that for stationary kernels, the gradient components orthogonal to the directions connecting the target and conditioning points are conditionally independent of the target function value; consequently, the exact conditional density is fully characterized by at most

m^2

directional derivatives once a conditioning set of size

m

is specified. By using these reduced, dimension-free conditionals as local factors in a Vecchia approximation, TERA effectively decouples

n

and

d

from the dense matrix inversion. This reduces the per-target evaluation cost to

\mathcal{O}(dm^2 + m^6)

time and

\mathcal{O}(dm^2 + m^4)

memory, leaving the underlying derivative GP model mathematically unchanged. Empirical evaluations demonstrate that TERA achieves state-of-the-art predictive accuracy while operating orders of magnitude faster than standard derivative GPs. Crucially, both computation time and peak GPU memory remain essentially flat with respect to

d

, enabling highly scalable inference in high-dimensional spaces.

Hyunseok Seung, Matthias Katzfuss

Department of Statistics · University of Wisconsin–Madison · Madison, WI, 53706

Surrogate Modeling Gaussian Processes Gaussian Process Regression

May 31, 2026stat.ML

On the Uncertainty Quantification Ability of Tabular Foundation Models

Foundation models (FMs) have achieved substantial success in generalizing across tasks without problemspecific training or fine-tuning. However, many critical applications in mechanics and computational science require not only accurate predictions but also reliable uncertainty quantification (UQ). Herein we investigate the UQ capabilities of tabular FMs in regression tasks through a comprehensive empirical study comparing Tabular Prior-Data Fitted Networks (TabPFN) against Gaussian processes (GPs). We systematically evaluate these two methods across a host of regression problems with varying complexity, dataset sizes, and input dimensionalities. We use a default setting to build all the GPs and for a fair comparison against TabPFN v2.5. Our findings highlight an important trade-off between explicit and learned priors: while TabPFN achieves highly competitive performance for complex, high-dimensional problems with sufficient data, GPs often provide superior predictive accuracy and UQ in data-scarce settings. Moreover, when the chosen kernel constitutes a good prior for the underlying function, GP performance can substantially exceed that of TabPFN. Our results can be reproduced from https://github.com/kianswarehouse/GPvsPFN.

Tyler R. Johnson, Kian Ben-Jacob, Nima Negarandeh +2

Department of Mechanical and Aerospace Engineering, University of California, Irvine · Department of Civil and Environmental Engineering, University of California, Irvine

Tabular Foundation Models Uncertainty Quantification Prior-Data Fitted Networks Tabular Regression Gaussian Process Regression

May 30, 2026stat.AP

Hybrid Probabilistic Forecasting of Under-Five Malaria Admissions in Ghana: A Gaussian Process Regression with Holt-Winters Smoothing

Accurate malaria forecasting remains a major challenge in sub-Saharan Africa, where strong seasonality, reporting uncertainty, and non-stationary transmission dynamics reduce the reliability of conventional models. In Ghana, district-level malaria surveillance requires forecasting frameworks that are probabilistically rigorous and robust under limited data. This study proposes a hybrid framework integrating Gaussian Process Regression (GPR) with Holt-Winters exponential smoothing for modelling monthly under-five malaria admissions. GPR captures non-linear behaviour and predictive uncertainty, while Holt-Winters stabilises long-horizon forecasts and preserves seasonal structure. Using ten years of district-level data (2014-2023), performance was evaluated via rolling-origin expanding-window validation. The hybrid model achieved

R^2 = 0.9906

versus

0.8213

for Holt-Winters alone, with

94.2\%

of residuals within

\pm 2σ

bounds. Forecasts for 2024-2028 project average monthly admissions from approximately 8{,}000 to 12{,}200 cases. Spatio-temporal analysis revealed pronounced ecological heterogeneity: northern high-burden districts exhibited stable relative patterns despite large absolute fluctuations. The framework provides a scalable probabilistic approach for malaria early warning and operational planning in endemic settings, supporting Ghana's national malaria control strategy.

T. Ansah-Narh, Y. Asare Afrane, J. Bremang Tandoh

Epidemic Forecasting Time Series Forecasting Spatiotemporal Forecasting Probabilistic Time Series Forecasting Gaussian Process Regression

May 29, 2026physics.comp-ph

Physically Constrained Ensemble Gaussian Process Modelling for Expensive Quantum Systems with Heteroskedastic Noise

Accurate modeling of quantum many-body systems often requires computationally expensive simulations such as Density Matrix Renormalization Group (DMRG) or Quantum Monte Carlo (QMC) calculations. These methods, while precise, impose significant time and resource constraints, limiting their use in exhaustive parameter exploration. Moreover, these expensive simulations can contain variable errors over the large unknown parameter space, which needs to be quantified and propagated. Thus, predictive modelling is required to estimate the functional space accurately over scarcely sampled data with heteroskedastic noise, while preserving the physical relevance of the estimation. Therefore, we present a Physically Constrained Ensemble Gaussian Process (pc-EGP) framework designed to efficiently model complex and noisy quantum systems under physical consistency constraints. The proposed method first enforces physical constraints as a user controlled weighted penalty to the data-driven loss function of the Gaussian Process (GP) surrogates. Then an ensemble of such GP models is trained with variable noisy simulations via numerical quadrature method where these multiple GP(s) at different nodes is integrated as a quadrature weighted average. We first demonstrate the framework on synthetically generated data before applying to quantum systems. In the first case study, we leverage DMRG simulations of the Bose-Hubbard Model to predict the critical interaction parameter Uc governing the superfluid-to-Mott-insulator transition. In the second case study, we demonstrate our method on QMC simulations, of a quantum liquid confined inside a nanoporous silicate with the goal of optimizing a chemical environment to realize a one-dimensional superfluid. Compared to conventional GP, pc-EGP achieves a better balance of accuracy and physically meaningful predictions.

Arpan Biswas, Sutirtha Paul, Joseph Agada +2

University of Tennessee-Oak Ridge Innovation Institute, University of Tennessee, Knoxville, TN 37996, USA · Institute for Advanced Materials and Manufacturing, University of Tennessee, Knoxville, Tennessee 37996, USA · Department of Physics and Astronomy, University of Tennessee, Knoxville, TN 37996, USA. +2

Surrogate Modeling Uncertainty Quantification Heteroscedastic Regression Physics-Informed ML Quantum Many-Body Physics Gaussian Process Regression

May 26, 2026q-fin.RM

Forward-Looking Stress Testing Under Macro Scenarios: Stable SVaR Estimation Using a Hybrid GPR-HS Framework with SACS

Regulatory stress testing frameworks, including the Comprehensive Capital Analysis and Review (CCAR) and the Internal Capital Adequacy Assessment Process (ICAAP), require robust Stressed Value-at-Risk (SVaR) estimation under forward-looking macroeconomic scenarios. Traditional parametric approaches often exhibit numerical instability under extreme shocks, reducing the reliability of capital projections. This paper extends the Hybrid Gaussian Process Regression Historical Simulation (GPR-HS) framework of Vadrevu (2026) to forward-looking stress scenarios, demonstrating stability across three regimes: West Asia War, Climate Risk, and AI Bubble/Regulation. A key contribution is the Scenario-Averaged Covariance Stabilization (SACS) framework, which constructs stress covariance as a weighted aggregation of historical crisis regimes, providing stable and interpretable dependence structures. Stressed return paths are generated over a 252-day horizon using deterministic drift and stochastic residuals, while volatility is modeled via Gaussian Process Regression with Aggressive Noise Initialization (ANI). The framework exhibits consistent convergence across all assets and scenarios. SVaR ranges from -2.1020% to -2.2231%, with the coherence property |SES| > |SVaR| preserved. The results support GPR-HS with SACS as a stable and regulator-aligned approach for forward-looking SVaR and SES estimation in CCAR and ICAAP applications.

Ujjwala Vadrevu

Quantitative Finance Gaussian Process Regression

May 26, 2026stat.ML

Gaussian Process-based learning with new MCMC-based implementation of Wishart prior on correlation matrix

In probabilstic supervised learning of an input-output relationship - as a sample function of a Gaussian Process (GP) - priors are typically specified for the hyperparameters of the kernel that parametrises the covariance function of the GP, where the induced covariance matrix of the (resulting multivariate Normal) likelihood, governs the learning and prediction. When the sought function is highly multivariate, multiple lengthscale parameters must be learnt simultaneously, making inference difficult. We develop a ``self-assembled'' Wishart prior for the covariance matrix, while undertaking Bayesian inference on the kernel hyperparameters using MCMC. The construction uses a look-back window over recent MCMC iterations to define a time-step dependent scale matrix, thereby introducing adaptiveness to the chain. Results suggest that direct prior specification on the covariance matrix can be useful for diagnosing weakly informative inputs within the GP-based learning paradigm. We support our prior development with two distinct empirical illustrations - one on synthetic data, and another on a real-world dataset.

Kane Warrior, Dalia Chakrabarty

Department of Mathematics University of York

Bayesian Inference Covariance Estimation Gaussian Process Regression

May 22, 2026cs.LG

Onsager-Machlup Posterior Transport for Deep Gaussian Processes

Approximate inference over inducing variables is the central computational bottleneck of Deep Gaussian Processes (DGPs). Existing methods either fit an explicit density

q_φ(\bU)

by an ELBO (DSVI, IPVI, DDVI, DBVI) or sample by MCMC (SGHMC). We instead frame DGP inference as \emph{posterior transport}: learn a deterministic sampler that maps a tractable reference measure to posterior-relevant inducing variables, regularised by a path prior derived from the Doob-bridged reference diffusion. Our realisation, \textbf{OM-Path} (formally FBVI-bridge-Path), uses Song's probability-flow ODE applied to DBVI's Doob-bridged forward SDE; the reference drift is closed-form from the bridge marginal coefficients (no score matching) and the path regulariser is the \textbf{Onsager--Machlup action}. At the finite-

ε

value used at training, the objective is the negative log unnormalised density of a tempered Doob-bridge path posterior, and Theorem 1 identifies it with the same posterior's small-noise MAP path via the Freidlin--Wentzell LDP. Two strict path-space ELBO variants on the same bridge backbone (FFJORD log-det; OM-regularised CNF) are derived as ablations. Under a matched-seed paired Wilcoxon test against DBVI on seven UCI regression benchmarks, OM-Path delivers statistically significant wins on the two largest datasets (\textit{power}:

p\!=\!0.014

, NLL

\mathbf{0.012}

matching the DSVI baseline of

0.017

; \textit{protein}:

p\!=\!0.002

, RMSE

\mathbf{0.716}

vs.\

0.764

, NLL

\mathbf{1.086}

vs.\

1.149

), statistical ties on \textit{yacht} / \textit{qsar}, and concedes \textit{boston} / \textit{energy} / \textit{concrete} to DBVI on small-

N

noisy data. The strict-ELBO variants do not clear DBVI on any UCI metric: in this regime, reducing the variance of the path objective dominates exact-density tracking.

Jian Xu, Delu Zeng, John Paisley +1

RIKEN iTHEMS · RIKEN AIP · South China University of Technology +1

Gaussian Processes Variational Inference Gaussian Process Regression

May 21, 2026cs.LG

Aerodynamic force reconstruction using physics-informed Gaussian processes

Accurate modeling of aerodynamic loads is essential for understanding and predicting the responses of complex structural systems. However, these models often rely on simplifications of the true physical forces, introducing assumptions that can limit their accuracy. Validating such models becomes particularly challenging in the presence of noisy or incomplete data. To address this, we introduce a probabilistic physics-informed machine learning approach designed to reconstruct the underlying aerodynamic loads from noisy measurements of structural dynamic responses. The model avoids overfitting, eliminates the need for regularization schemes, and allows for the use of heterogeneous and multi-fidelity data during the training process. The efficacy of the approach is demonstrated through the reconstruction of aerodynamic loads on the Great Belt East Bridge, simulated under a linear unsteady assumption. Results show a strong agreement between true and predicted loads, particularly related to root mean squared errors, magnitude, phase angle and peak values of the signals. The method for load reconstructing holds broad applicability, such as modeling validation, future load estimation, and structural damage prognosis.

Gledson Rodrigo Tondo, Igor Kavrakov, Guido Morgenthal

Bauhaus-Universität Weimar Weimar, Germany · University of Cambridge Cambridge, United Kingdom

Physics-Informed ML Inverse Problems Gaussian Process Regression

May 20, 2026cs.LG

Three Costs of Amortizing Gaussian Process Inference with Neural Processes

Neural processes amortize Gaussian process inference, replacing the exact

O(n^3)

posterior with a learned

O(n)

map from context sets to predictive distributions. For a class of latent neural processes, we bound the Kullback--Leibler (KL) divergence between the GP and LNP predictives, decomposing it into three interpretable sources, namely label contamination as the neural process uses label values to estimate a quantity that is label-independent in the exact GP, an information bottleneck because the finite-dimensional representation cannot resolve the full context geometry, and amortization error from a single encoder network shared across all contexts. The bottleneck truncation term decays in the representation dimension

d

as

O(e^{-cd^{2/d_x}})

for squared-exponential kernels on

\mathbb{R}^{d_x}

where

c > 0

is a kernel-dependent constant and as

O(d^{-2ν/d_x})

for Matérn-

ν

kernels, directly linking architecture sizing to kernel smoothness and input dimension. The label contamination term is

O(1)

in general, with only the observation-noise component decaying as

O(1/n)

, identifying a persistent cost of routing uncertainty estimation through a label-dependent representation. These results characterize the costs of amortization within the analyzed class and yield architectural recommendations to predict variance from context locations alone in the GP-amortization regime, and replace mean aggregation with second-order pooling to close the dominant amortization gap.

Robin Young

University of Cambridge, Cambridge, UK

Latent Variable Models Amortized Inference Neural Processes Gaussian Process Regression

May 20, 2026cs.LG

Modeling Temporal scRNA-seq Data with Latent Gaussian Process and Optimal Transport

Single-cell RNA sequencing provides insights into gene expression at single-cell resolution, yet inferring temporal processes from these static snapshot measurements remains a fundamental challenge. Current approaches utilizing neural differential equations and flows are sensitive to overfitting and lack careful considerations of biological variability. In this work, we propose a generative framework that models population trends using a latent heteroscedastic Gaussian process (GP) approximated by Hilbert space methods. To address the absence of genuine cell trajectories, we leverage an optimal transport (OT) objective that aligns generated and observed population distributions. Our method explicitly captures biological heterogeneity by incorporating cell-specific latent time and cell type conditioning to disentangle temporal asynchrony and trajectories to different cell types. We demonstrate state-of-the-art performance on complex interpolation and extrapolation benchmarks and introduce a novel gradient-based strategy for inferring perturbation trajectories.

Mehmet Yigit Balik, Harri Lähdesmäki

Department of Computer Science, Aalto University, Espoo, Finland.

Latent Dynamics Modeling Conditional Generative Modeling Single-Cell Transcriptomics Gaussian Process Regression

May 19, 2026cs.LG

DeRegiME: Deep Regime Mixtures for Probabilistic Forecasting under Distribution Shift

We introduce DeRegiME -- Deep Regime Mixture of Experts -- a direct multi-horizon probabilistic forecaster that separates latent uncertainty regimes from the underlying signal and softly assigns each forecast location to learned recurring regimes using a sparse variational Gaussian process (GP) whose nonstationary regime-mixing kernel and Student-t likelihood combine per-regime sub-kernels and noise processes via a shared gate. This yields a single sparse-GP posterior, not a mixture of GP experts. DeRegiME addresses a key limitation of neural forecasters: point forecasts discard residual uncertainty, and probabilistic heads -- whether single marginals, uninterpreted mixtures, quantile sets, or diffusion samples -- rarely expose the regime structure of the residual. Yet distribution shift in noisy heteroskedastic time series may be abrupt, gradual, or horizon-dependent and often appears in residual uncertainty rather than the conditional mean. DeRegiME yields an interpretable mean-residual-noise decomposition with a direct-sum feature-space representation that anchors regimes as clusters of residual similarity whose transitions surface as implicit changepoints. The effective number of regimes is pruned by the stick-breaking gate. We prove kernel validity and predictive-density propriety, and across ten benchmarks and three encoder grids DeRegiME improves negative log predictive density (NLPD) by 20.3% over the strongest encoder-matched baseline, a DeepAR/GluonTS-style dynamic Student-t head, with parallel gains on CRPS (3.0%) and MSE (4.7%). Improvements are consistent across all datasets, which span abrupt, gradual, and seasonal shifts.

Kieran Wood, Stefan Zohren, Stephen J. Roberts

Machine Learning Research Group, University of Oxford · Oxford-Man Institute, University of Oxford

Distribution Shift Robustness Time Series Forecasting Non-Stationary Time Series Forecasting Probabilistic Time Series Forecasting Temporal Distribution Shift Gaussian Process Regression

May 18, 2026cs.LG

Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization

Gaussian Process (GP) kernels are central to Bayesian optimization (BO), yet designing effective kernels for high-dimensional problems still relies on extensive manual engineering. Existing automated approaches struggle in high dimensions for two bottlenecks: their kernel search space is limited to additions and multiplications of base kernels, and LLM-based approaches require conditioning on raw observations, which becomes infeasible due to context-length limits and the difficulty of extracting meaningful patterns. We introduce \textbf{Kernel Discovery}, a LLM-driven evolutionary framework for high-dimensional BO that searches a broader kernel space beyond predefined composition rules and does not require conditioning on observations. Motivated by the observation that directly prompting an LLM to generate kernel code yields syntactically varied but functionally identical kernels, we adopt a two-stage approach: an LLM first proposes novel mathematical forms, then a second LLM call converts each form into validated, executable code. We also propose a leave-one-out continuous ranked probability score (LOO-CRPS) as a selection criterion that penalizes overfitted kernels. On five high-dimensional BO benchmarks, our method achieves an average rank of \textbf{1.2 out of 17}, outperforming competitive baselines. We further analyze the discovered kernels to identify which kernels lead to improvements in high-dimensional BO.

Taeyoung Yun, Woocheol Shin, Inhyuck Song +2

1Korea Advanced Institute of Science and Technology (KAIST)

Bayesian Optimization Large Language Model-Guided Optimization Kernel Methods Gaussian Process Regression

May 18, 2026cs.LG

Lightweight Gaussian Process Inference in C++ on Metal and CUDA

Gaussian process (GP) inference in Python is dominated by libraries such as GPyTorch and GPflow, which are built on deep-learning frameworks and inherit their dispatch overhead and dependency footprint. We present LightGP, a dependency-free C++17 library for GP regression with Python bindings, supporting Apple Metal and NVIDIA CUDA backends alongside tuned CPU paths via Apple Accelerate and OpenBLAS. LightGP provides four inference paths -- exact Cholesky, matrix-free conjugate gradients, sparse variational free energy, and structured kernel interpolation with FFT -- covering problems from

N{=}100

to

N{=}500{,}000

. On an Apple M4, LightGP CPU is 2.6--8.7

\times

faster than GPyTorch CPU for exact GP and

{\sim}1.5\times

faster for sparse GP at every scale tested. On an NVIDIA RTX~3060, LightGP CUDA is 2.3--6.7

\times

faster than GPyTorch CUDA for exact GP up to

N{=}2{,}048

, with GPyTorch closing the gap at

N{=}4{,}096

. A fused matrix-free kernel-vector product on Metal achieves 32

\times

over the explicit path at

N{=}20{,}000

with

O(N)

memory, and an FFT-accelerated SKI matvec via Accelerate vDSP runs in sub-millisecond time at

N{=}200{,}000

. LightGP compiles as a single static library with zero external dependencies and is installable via \texttt{pip install lightgp

Yu-Hsueh Fang

Department of Information Management, National Taiwan University · 2H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology

GPU Kernel Optimization GPU Acceleration High-Performance Computing Gaussian Processes Gaussian Process Regression

May 17, 2026q-fin.RM

A Hybrid Gaussian Process Regression Framework for Stable Volatility-Covariance Estimation: Evidence from Global Equity Indices

Accurate forecasting of the Volatility-Covariance Matrix (VCV) is central to regulatory capital adequacy processes such as the Internal Capital Adequacy Assessment Process (ICAAP) and the Comprehensive Capital Analysis and Review (CCAR). Traditional econometric models, including GARCH-family and Exponentially Weighted Moving Average (EWMA) approaches, suffer from parametric rigidity, distributional assumptions, and numerical instability under stress, leading to systematic underestimation of tail risk. This paper proposes and validates a novel Hybrid Gaussian Process Regression-Historical Simulation (GPR-HS) framework for estimating Value-at-Risk (VaR) and Expected Shortfall (ES) across a diversified portfolio of seven major global equity indices. The framework decouples the VCV estimation problem: individual asset volatilities are modelled dynamically using Univariate GPR with a Matern 5/2 kernel, while inter-asset correlations are estimated via stable historical covariance. A key methodological contribution is the Aggressive Noise Initialization (ANI) strategy, which sets the initial White Noise kernel variance equal to the empirical variance of the training returns, ensuring Gram matrix positive-definiteness, regularization, and conservative, regulatory-compliant forecasts. Evaluated using an expanding window forward-chaining cross-validation scheme over June 2020 -June 2025, the GPR-HS framework achieves regulatory compliance in the majority of test splits; including a 100% ES pass rate at the portfolio level, while outperforming the static Historical VaR benchmark in 71.4% of univariate cases by Quadratic Loss and 100% of cases by violation count.

Ujjwala Vadrevu

Quantitative Finance Financial Time Series Covariance Estimation Gaussian Process Regression

May 11, 2026stat.ML

Interpretable Machine Learning for Spatial Science: A Lie-Algebraic Kernel for Rotationally Anisotropic Gaussian Processes

Many three-dimensional spatial fields are anisotropic, with directions of rapid and slow variation that need not align with the coordinate axes. Standard Gaussian process kernels with Automatic Relevance Determination (ARD) capture only axis-aligned anisotropy, while generic full symmetric positive definite (SPD) metrics can represent rotated anisotropy but do not parameterise principal length-scales and directions directly. We introduce an interpretable rotationally anisotropic GP kernel that parameterises a three-dimensional SPD covariance metric using three principal length-scales and an explicit SO(3) rotation. The rotation is represented by an axis-angle vector and mapped to SO(3) via the Lie-algebra exponential map, giving unconstrained Euclidean coordinates for inference while always inducing a valid SPD metric. The construction spans the same family of three-dimensional SPD covariance metrics as a generic full-SPD parameterisation, but exposes the geometry differently: length-scales and orientation are explicit, interpretable, and directly available for prior specification and posterior summaries. We perform Bayesian inference on these quantities using Markov Chain Monte Carlo (MCMC), and characterise the resulting symmetries and weakly identified regimes. On synthetic data with rotated anisotropy, the posterior recovers the generating metric and improves prediction relative to an axis-aligned ARD baseline, while matching the predictive performance of a generic full SPD baseline. When the ground truth is axis-aligned, posterior mass concentrates near the identity rotation and predictive performance matches ARD. On a material-density dataset from a laboratory-fabricated nano-brick, the inferred metric reveals rotated anisotropy that is not captured by axis-aligned kernels.

Kane Warrior, Dalia Chakrabarty

University of York, Department of Mathematics, York, United Kingdom

Gaussian Processes Lie Group Methods Gaussian Process Regression

May 11, 2026cs.LG

Active Learning for Gaussian Process Regression Under Self-Induced Boltzmann Weights

We consider the active learning problem where the goal is to learn an unknown function with low prediction error under an unknown Boltzmann distribution induced by the function itself. This self-induced weighting arises naturally in problems such as potential energy surface (PES) modeling in computational chemistry, yet poses unique challenges as the target distribution is unknown and its partition function is intractable. We propose \texttt{AB-SID-iVAR}, a Gaussian Process-based acquisition function that approximates the intractable Bayesian target distribution in closed form while avoiding partition function estimation, and is applicable to both discrete and continuous input domains. We also analyze a Thompson sampling alternative (\texttt{TS-SID-iVAR}) as a higher variance Monte Carlo variant. Despite the unknown target, under mild conditions, we establish that the terminal prediction error vanishes with high probability, and provide a tighter average-case guarantee. We demonstrate consistent improvements over existing approaches in this setting on synthetic benchmarks and real-world PES modeling and drug discovery tasks.

Jixiang Qing, Henry Moss, Matthias Sachs

MARS: Mathematics for AI in Real-world Systems, School of Mathematical Sciences, Lancaster University, Lancaster, LA1 4YF, United Kingdom.

Active Learning Gaussian Process Regression

May 11, 2026cs.LG

Online Sharp-Calibrated Bayesian Optimization

Bayesian optimization (BO) is a widely used framework for optimizing expensive black-box functions, commonly based on Gaussian process (GP) surrogate models. Its effectiveness relies on uncertainty quantification that is both sharp (informative) and well-calibrated along the BO trajectory. In practice, GP kernel hyperparameters are unknown and are refit online from sequentially collected (non-i.i.d.) data, which can yield miscalibrated or overly conservative uncertainty and lies outside the fixed-kernel assumptions of standard BO regret theory. We propose Online Sharp-Calibrated Bayesian Optimization (OSCBO), a BO algorithm that adaptively balances GP sharpness and calibration by casting hyperparameter selection as a constrained online-learning problem. We also show that OSCBO preserves sublinear regret bounds by leveraging the theoretical guarantees of the underlying online learning algorithm. Empirically, OSCBO performs competitively across synthetic and real-world benchmarks, ranking among the strongest methods in final simple regret while maintaining robust cumulative-regret behavior.

Marshal Arijona Sinaga, Julien Martinelli, Teemu Turpeinen +1

ELLIS Institute Finland · Aalto University · University of Manchester

Bayesian Optimization Uncertainty Calibration Gaussian Process Regression

May 11, 2026stat.ML

Multifidelity Gaussian process regression for solving nonlinear partial differential equations

Solving nonlinear partial differential equations (PDEs) using kernel methods offers a compelling alternative to traditional numerical solvers. However, the performance of these methods strongly depends on the choice of kernel. In this work, as the available information is inherently multifidelity, we propose a kernel learning approach based on cokriging, leveraging empirical information from multifidelity simulations. In the first step, we fit a differentiable non-stationary kernel to an empirical kernel obtained from low-fidelity simulations. In the second step, we derive a high-fidelity kernel with estimated hyperparameters, and construct a corresponding high-fidelity mean using the multifidelity framework. These components can then be used within a Gaussian process framework for solving PDEs. Finally, we demonstrate the performance of the proposed physics-informed method on the Burgers' equation.

Fatima-Zahrae El-Boukkouri, Josselin Garnier, Olivier Roustant

Institut de Mathématiques de Toulouse, Université de Toulouse, INSA, Toulouse, 31077, France · CMAP, CNRS, Ecole polytechnique, Institut Polytechnique de Paris, Palaiseau, 91120, France

Surrogate Modeling PDE Solving Multifidelity Modeling Physics-Informed ML Gaussian Process Regression

May 11, 2026stat.ML

Scalable Gaussian process inference via neural feature maps

We present a theoretically grounded Gaussian process framework that leverages neural feature maps to construct expressive kernels. We show that the learned feature map can be interpreted as an optimal low-rank approximation to a Gram matrix derived from an implied RKHS, from which we establish consistency of the GP posterior. We further analyse the spectral properties of the induced kernels and introduce product feature-map kernels to address oversmoothing. This simple yet powerful approach enables fast, scalable, and accurate exact GP inference with minimal upfront work. The flexibility of kernel design supports seamless application to both regression and classification tasks across diverse data modalities, including tabular inputs and structured domains such as images. On benchmark datasets, this approach surpasses pre-existing methods in terms of accuracy and training and prediction efficiency.

Anthony Stephenson

School of Mathematics University of Bristol

Gaussian Processes Kernel Methods Gaussian Process Regression

May 8, 2026cs.LG

Don't Get Your Kroneckers in a Twist: Gaussian Processes on High-Dimensional Incomplete Grids

We introduce CUTS-GPR, a new method for performing numerically exact GPR in high-dimensional settings. The key component of CUTS-GPR is an extremely fast kernel matrix-vector product, which exhibits near-linear or even linear scaling with the amount of training data,

N

, and low-order polynomial scaling with dimensionality,

D

. This is obtained by combining an additive kernel with an incomplete grid and exploiting the resulting structure of the kernel matrix. The scalability of the matrix-vector product is verified by benchmarks with billions of data points and thousands of dimensions. We demonstrate the end-to-end scalability of CUTS-GPR by running full GPR calculations, including hyperparameter optimization, on synthetic datasets with up to

N = 4\,494\,001

and

D = 500

. As a realistic and challenging test, we finally apply CUTS-GPR to a set of ten potential energy surfaces (PESs) with

N = 447\,265

and

D = 24

. The calculations are completed in a matter of hours, showing that CUTS-GPR enables Bayesian modelling of high-dimensional PESs - a longstanding challenge in computational chemistry.

Mads Greisen Højlund, August Smart Lykke-Møller, Henry Moss +1

Department of Chemistry Aarhus University · School of Mathematical Sciences Lancaster University

Kernel Methods Gaussian Process Regression

May 7, 2026cs.LG

Structure-Preserving Gaussian Processes Via Discrete Euler-Lagrange Equations

In this paper, we propose Lagrangian Gaussian Processes (LGPs) for probabilistic and data-efficient learning of dynamics via discrete forced Euler-Lagrange equations. Importantly, the geometric structure of the Lagrange-d'Alembert principle, which governs the motion of dynamical systems, is preserved by construction in the absence of external forces. This allows learning physically consistent models that overcome erroneous drift in the system's energy, thereby providing stable long-term predictions. At the core of our approach lie linear operators for Gaussian process conditioning, constructed from discrete forced Euler-Lagrange equations and variational discretization schemes. Thereby and unlike prior work, the method enables learning dynamics from discrete position snapshots, i.e., without access to a system's velocities or momenta. This is particularly relevant for a large class of practical scenarios where only position measurements are available, for instance, in motion capture or visual servoing applications. We demonstrate the data-efficiency and generalization capabilities of the LGPs in various synthetic and real-world case studies, including a real-world soft robot with hysteresis. The experimental results underscore that the LGPs learn physically consistent dynamics with uncertainty quantification solely from sparse positional data and enable stable long-term predictions.

Jan-Hendrik Ewering, Kathrin Flaßkamp, Niklas Wahlström +2

Leibniz Universität Hannover · Saarland University · Uppsala University

Latent Dynamics Modeling Physics-Informed ML Gaussian Process Regression Data-Efficient Learning

May 1, 2026cs.LG

Deep Kernel Learning for Stratifying Glaucoma Trajectories

Effectively stratifying patient risk in chronic diseases like glaucoma is a major clinical challenge. Clinicians need tools to identify patients at high risk of progression from sparse and irregularly-sampled electronic health records (EHRs). We propose a novel deep kernel learning (DKL) architecture that leverages a Gaussian Process (GP) backend. The GP's kernel is defined by a transformer-based feature extractor applied to clinical-BERT embeddings to model glaucoma patient trajectories from multimodal EHR data. Our method successfully identifies three clinically distinct patient subgroups. Crucially, the model learns to decouple disease progression from current severity, identifying a high-risk group with a worsening trajectory despite having better average visual acuity than a second, stably poor group. This reveals that the model learns to identify progression risk rather than just the current disease state. This ability to stratify patients based on their risk trajectory progression offers a powerful tool for clinical decision support, enabling targeted interventions for high-risk individuals and improving the management of glaucoma care.

Bruce Rushing, Angela Danquah, Alireza Namazi +2

Research Computing, University of Virginia · School of Data Science, University of Virginia · School of Medicine, University of Virginia

Clinical Outcome Prediction Kernel Methods Longitudinal EHR Modeling Clinical Risk Prediction Gaussian Process Regression

Apr 30, 2026quant-ph

Provable and scalable quantum Gaussian processes for quantum learning

Despite rapid recent advances in quantum machine learning, the field is in many ways stuck. Existing approaches can exhibit serious limitations, and we still lack learning frameworks that are simple, interpretable, scalable, and naturally suited to quantum data. To address this, here we introduce quantum Gaussian processes, a Bayesian framework for learning from quantum systems through priors over unknown quantum transformations. We show that, under suitable conditions, unitary quantum stochastic processes define Gaussian processes, thereby enabling regression, classification, and Bayesian optimization directly on quantum data. The key ingredient in this framework is sufficient knowledge of a quantum process's structure and symmetries to define an informative prior through its corresponding quantum kernel, effectively injecting a strong, physics-informed inductive bias into the learning model. We then prove that matchgate, or free-fermionic, evolutions give rise to provable and scalable quantum Gaussian processes, providing the first family in our framework where the unknown unitary acts non-trivially on all qubits. Finally, we demonstrate accurate long-range extrapolation, phase-diagram learning in many-body systems, and sample-efficient Bayesian optimization in a quantum sensing task. Our results identify quantum Gaussian processes as a promising route toward simpler and more structured forms of quantum learning.

Jonas Jäger, Paolo Braccia, Pablo Bermejo +3

Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, USA · Department of Computer Science and Institute of Applied Mathematics, University of British Columbia, Vancouver, V6T 1Z4 B.C., Canada · Stewart Blusson Quantum Matter Institute, Vancouver, V6T 1Z4 B.C., Canada +8

Quantum Machine Learning Quantum Kernel Methods Gaussian Process Regression

Apr 30, 2026eess.SP

Sequential Inference for Gaussian Processes: A Signal Processing Perspective

The proliferation of capable and efficient machine learning (ML) models marks one of the strongest methodological shifts in signal processing (SP) in its nearly 100-year history. ML models support the development of SP systems that represent complex, nonlinear relationships with high predictive accuracy. Adapting these models often requires sequential inference, which differs both theoretically and methodologically from the usual paradigm of ML, where data are often assumed independent and identically distributed. Gaussian processes (GPs) are a flexible yet principled framework for modeling random functions, and they have become increasingly relevant to SP as statistical and ML methods assume a more prominent role. We provide a self-contained, tutorial-style overview of GPs, with a particular focus on recent methodological advances in sequential, incremental, or streaming inference. We introduce these techniques from a signal-processing perspective while bridging them to recent advances in ML. Many of the developments we survey have direct applications to state-space modeling, sequential regression and forecasting, anomaly detection in time series, sequential Bayesian optimization, adaptive and active sensing, and sequential detection and decision-making. By organizing these advances from a signal-processing perspective, we intend to equip practitioners with practical tools and a coherent roadmap for deploying sequential GP models in real-world systems.

Daniel Waxman, Fernando Llorente, Petar M. Djurić

1Basis Research Institute · 3Computing and Data Sciences Directorate, Brookhaven National Laboratory · Department of Electrical and Computer Engineering, Stony Brook University

Gaussian Processes Gaussian Process Regression

Apr 30, 2026cs.LG

Differentiable latent structure discovery for interpretable forecasting in clinical time series

Background: Timely, uncertainty-aware forecasting from irregular electronic health records (EHR) can support critical-care decisions, yet most approaches either impute to a grid or sacrifice interpretability. We introduce StructGP, a continuous-time multi-task Gaussian process that couples process convolutions with differentiable structure learning to uncover a sparse, ordered directed acyclic graph (DAG) of inter-variable dependencies while preserving principled uncertainty. We further propose LP-StructGP, which augments StructGP with latent pathways-shared, temporally shifted trajectories inferred via subject-specific coupling filters and a softmax gating mechanism-to capture cross-patient progression patterns. Both models are trained under sparsity and acyclicity constraints (augmented Lagrangian, Adam) using scalable low-rank updates. Results: In simulations, the approach reliably recovers ground-truth graphs (Structural Hamming Distance approaching 0 as cohorts grow) and pathway assignments (high Adjusted Rand Index). On a MIMIC-IV septic shock cohort (n=1,008; norepinephrine, creatinine, mean arterial pressure), StructGP improves short-horizon (6 h) forecasting over independent-task baselines (average RMSE 0.68 [95%CI: 0.63--0.74] vs. 0.88 [0.83-0.94]) and, with 15 additional inputs, markedly outperforms unstructured kernels (0.63 [0.58-0.69] vs. 3.02 [2.85-3.18]) with superior calibration (coverage 0.96 vs. 0.84). On the PhysioNet Challenge (12k patients, 41 variables), StructGP attains competitive accuracy (MAE 3.72e-2) relative to a state-of-the-art graph neural model while maintaining calibrated uncertainty. Conclusion: These results show that structured process convolutions with latent pathways deliver interpretable, scalable, and well-calibrated forecasting for irregular clinical time series.

Ivan Lerner, Jean Feydy, Alexandre Kalimouttou +2

Universit´e Paris Cit´e and Universit´e Sorbonne Paris Nord, INRIA, Inserm, HEKA, Paris, 75015, France. · Department of Medical Informatics, Assistance Publique Hˆopitaux de Paris (AP-HP), Georges Pompidou European Hospital, Paris, France. · SIERRA, Inria Paris, Paris, 75015, France. +2

Irregular Time-Series Modeling Healthcare Causal Structure Learning Longitudinal EHR Modeling Probabilistic Time Series Forecasting Gaussian Process Regression