math.NANov 9, 2026

Low-rank Low Precision Training of Gaussian Processes

Authors: Moritz Poguntke, Eda Oktay, Yuanzhe Xi, Erin Carson, Martin Stoll

Organizations: Department of Mathematics, TU Chemnitz, Chemnitz, Germany · Max Planck Institute for the Dynamics of Complex Technical Systems, Magdeburg, Germany · Department of Mathematics, Emory University, USA · Department of Numerical Mathematics, Faculty of Mathematics and Physics, Charles University

Abstract

Training Gaussian processes (GPs) at large scale is computationally challenging because evaluating the log-marginal likelihood and its gradients involves large, dense kernel matrices. We develop a scalable low-rank GP training approach that combines AnchorNet landmark selection with a fully independent training conditional (FITC) correction. The resulting objective and its analytic gradients can be evaluated using low-rank factors, diagonal quantities, and small dense matrices, without forming the full kernel matrix or its derivatives. The complete training procedure is implemented in single precision and evaluated on nine large-scale regression datasets. Across these problems, the method reaches the reference predictive accuracy while substantially reducing training time, although the required approximation rank varies across datasets. We also study the numerical behavior of the full-kernel predictive solve and show that single precision can become limiting for ill-conditioned problems. Finally, we investigate further reducing selected matrix operations to fp16 or TF32. These lower-precision variants provide only modest additional speedups and can become unreliable when the reduced system is poorly conditioned.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Adaptive multi-resolution Gaussian processes: Scalable exact inference with naturally data-sparse covariance matrices

    Sep 24, 2026Yanchuang Cao, Jun Liu, Tengchao Yu +1Gaussian ProcessesGaussian Process Regression

  2. Scalable Gaussian process inference via neural feature maps

    May 11, 2026Anthony StephensonGaussian ProcessesKernel Methods