Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functions

Yang Xu; Chiwoo Park

cs.LGOct 24, 2025

Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functions

Authors: Yang Xu, Chiwoo Park

Organizations: Industrial & Systems Engineering, University of Washington, Seattle, WA, USA

Abstract

We introduce Deep Jump Gaussian Processes (DJGP), a novel method for surrogate modeling of a piecewise continuous function on a high-dimensional domain. DJGP addresses the limitations of conventional Jump Gaussian Processes (JGP) in high-dimensional input spaces by integrating region-specific, locally linear projections with JGP modeling. These projections employ region-dependent matrices to capture local low-dimensional subspace structures, making them well suited to the inherently localized modeling behavior of JGPs, a variant of local Gaussian processes. To control model complexity, we place a Gaussian Process prior on the projection matrices, allowing them to evolve smoothly across the input space. The projected inputs are then modeled with a JGP to capture piecewise continuous relationships with the response. This yields a distinctive two-layer deep learning of GP/JGP. We further develop a scalable variational inference algorithm to jointly learn the projection matrices and JGP hyperparameters. Rigorous theoretical analysis and extensive empirical studies are provided to justify the proposed approach. In particular, we derive an oracle error bound for DJGP and decompose it into four distinct sources of error, which are then linked to practical implications. Experiments on synthetic and benchmark datasets demonstrate that DJGP achieves superior predictive accuracy and more reliable uncertainty quantification compared with existing methods.

Figures & tables

Panel of Figure 1 — Figure 1: Illustrations of response surfaces over 2D latent spaces. (a) A noiseless ground-truth regression surface with sharp transitions from the L2 dataset. (b) An example realization from the high-dimensional LH dataset, projected onto the 2D latent domain $[-0.5,0.5]^{2}$ .

Dataset				DJGP(Ours)	JGP	JGP-PCA	JGP-SIR	DGP	NGBoost	BART
	D	K	Feat.	R/C/C90	R/C/C90	R/C/C90	R/C/C90	R/C/C90	R/C/C90	R/C/C90
L2	20	2	AE	1.35/ 0.50 /0.87	1.43/0.53/0.78	1.42/0.52/0.77	1.83/0.78/0.79	1.40/0.68/0.95	1.37/0.58/0.79	1.32 /0.58/ 0.90
	20	2	ME	1.30 / 0.50 /0.86	1.47/0.53/0.78	1.52/0.53/0.76	2.07/0.83/0.81	1.38/0.67/0.95	1.31/0.57/0.77	1.38/0.60/ 0.90
	20	2	PE	1.42 / 0.52 /0.86	1.55/0.58/0.75	2.30/0.97/0.76	2.09/0.96/0.80	1.42 /0.69/0.94	1.50/0.61/0.78	1.46/0.64/ 0.91
	20	2	RP	1.19 / 0.48 /0.86	1.48/0.53/0.80	1.53/0.53/0.77	1.52/0.53/0.80	1.81/0.93/0.96	1.45/0.61/0.78	1.38/0.60/ 0.90
	20	2	RF	1.58/ 0.51 /0.84	1.46/0.54/0.77	1.92/0.74/0.78	2.12/0.90/0.80	1.67/0.85/0.94	1.42 /0.60/0.77	1.43/0.61/ 0.91

Table 1: Per-configuration results on RMSE, CRPS, and 90% coverage.

Method	NGBoost	BART	DJGP	JGP-PCA	DGP
Time (s)	10.0	29.5	36.9	70.5	93.7

Table 2: Wall-clock time for a single model-training run on the LH benchmark with

D=50

N=5000

, and

J=500

test points. For DJGP, the reported runtime includes PLS initialization and subsequent model fitting with a fixed configuration, but excludes the additional computational cost of evaluating multiple initialization and model selection procedures used in the main benchmark experiments.

Dataset	$N$	$D$	$Q$	$G_{a}$	$G_{m}$	$\mathrm{TV}_{2}$
Wine Quality	6,497	11	3	0.327	9.90	8,798
Parkinson’s Telemonitoring	5,875	18	5	2.685	56.83	60,872
Appliances Energy Prediction	19,735	28	5	6.459	355.17	2,849,220

Table 3: Dataset statistics and smoothness metrics.

N

is the total size of the available dataset; the training and test subsets used in the experiments are described in the text.

Dataset	Method	Train=1000, Test=200			Train=5000, Test=500
Dataset	Method	RMSE $\downarrow$	CRPS $\downarrow$	Cov90 $\rightarrow 0.90$	RMSE $\downarrow$	CRPS $\downarrow$	Cov90 $\rightarrow 0.90$
Wine Quality	DJGP	0.69	0.39	0.91	0.64	0.34	0.90
	JGP	0.85	0.51	0.57	0.75	0.39	0.69
	JGP-PCA	0.82	0.47	0.77	0.80	0.46	0.74
	JGP-SIR	0.80	0.46	0.78	0.78	0.45	0.77
	DGP	0.70	0.39	0.91	0.69	0.38	0.89

Table 4: UCI benchmark results under two train/test sizes.

Deep Gaussian Processes

See all 8 papers

Feb 27, 2026

Scalable Gaussian process modeling of parametrized spatio-temporal fields

We introduce a scalable Gaussian process (GP) framework with deep product kernels for data-driven learning of parametrized spatio-temporal fields over fixed or parameter-dependent domains. The proposed framework learns a continuous representation, enabling predictions at arbitrary spatio-temporal coordinates, independent of the training data resolution. We leverage Kronecker matrix algebra to formulate a computationally efficient training procedure with complexity that scales nearly linearly with the total number of spatio-temporal grid points. A key feature of our approach is the efficient computation of the posterior variance at essentially the same computational cost as the posterior mean (exactly for Cartesian grids and via rigorous bounds for unstructured grids), thereby enabling scalable uncertainty quantification. Numerical studies on a range of benchmark problems demonstrate that the proposed method achieves accuracy competitive with operator learning methods such as Fourier neural operators and deep operator networks. On the one-dimensional unsteady Burgers' equation, our method surpasses the accuracy of projection-based reduced-order models. These results establish the proposed framework as an effective tool for data-driven surrogate modeling, particularly when uncertainty estimates are required for downstream tasks.

Srinath Dama, Prasanth B. Nair

University of Toronto Institute for Aerospace Studies, 4925 Dufferin Street, Toronto, M3H 5T6, Ontario, Canada

Gaussian Processes Gaussian Process Regression PDE Surrogate Modeling Uncertainty Quantification

Ask PDF

Feb 24, 2026

Generative Bayesian Computation as a Scalable Alternative to Gaussian Process Surrogates

Gaussian process (GP) surrogates are the default tool for emulating expensive computer experiments, but cubic cost, stationarity assumptions, and Gaussian predictive distributions limit their reach. We propose Generative Bayesian Computation (GBC) via Implicit Quantile Networks (IQNs) as a surrogate framework that targets all three limitations. GBC learns the full conditional quantile function from input--output pairs; at test time, a single forward pass per quantile level produces draws from the predictive distribution. Across fourteen benchmarks we compare GBC to four GP-based methods. GBC improves CRPS by 11--26% on piecewise jump-process benchmarks, by 14% on a ten-dimensional Friedman function, and scales linearly to 90,000 training points where dense-covariance GPs are infeasible. A boundary-augmented variant matches or outperforms Modular Jump GPs on two-dimensional jump datasets (up to 46% CRPS improvement). In active learning, a randomized-prior IQN ensemble achieves nearly three times lower RMSE than deep GP active learning on Rocket LGBB. Overall, GBC records a favorable point estimate in 12 of 14 comparisons. GPs retain an edge on smooth surfaces where their smoothness prior provides effective regularization.

Nick Polson, Vadim Sokolov

Booth School of Business, University of Chicago. · Department of Systems Engineering and Operations Research, George Mason University.

Surrogate Modeling Gaussian Process Regression

Ask PDF

Jun 1, 2026

Scalable Derivative Gaussian Processes via Exact Gradient Reduction

Gradient observations can substantially improve Gaussian process (GP) surrogates, particularly in high-dimensional settings where function evaluations are expensive. However, exact inference with

n

function values and

n

full gradients in

d

dimensions scales cubically in the joint state size, imposing an intractable

\mathcal{O}(n^3 d^3)

computational bottleneck. We introduce TERA, a highly scalable derivative GP method based on target-specific exact gradient reduction. We prove that for stationary kernels, the gradient components orthogonal to the directions connecting the target and conditioning points are conditionally independent of the target function value; consequently, the exact conditional density is fully characterized by at most

m^2

directional derivatives once a conditioning set of size

m

is specified. By using these reduced, dimension-free conditionals as local factors in a Vecchia approximation, TERA effectively decouples

n

and

d

from the dense matrix inversion. This reduces the per-target evaluation cost to

\mathcal{O}(dm^2 + m^6)

time and

\mathcal{O}(dm^2 + m^4)

memory, leaving the underlying derivative GP model mathematically unchanged. Empirical evaluations demonstrate that TERA achieves state-of-the-art predictive accuracy while operating orders of magnitude faster than standard derivative GPs. Crucially, both computation time and peak GPU memory remain essentially flat with respect to

d

, enabling highly scalable inference in high-dimensional spaces.

Hyunseok Seung, Matthias Katzfuss

Department of Statistics · University of Wisconsin–Madison · Madison, WI, 53706

Surrogate Modeling Gaussian Processes Gaussian Process Regression

Ask PDF

Abstract

Deep Gaussian Processes

Explore similar work

Scalable Gaussian process modeling of parametrized spatio-temporal fields

Generative Bayesian Computation as a Scalable Alternative to Gaussian Process Surrogates

Scalable Derivative Gaussian Processes via Exact Gradient Reduction