Direct Inference on Admissible Sets (Diona)

Lorenzo Sabug; Eric Kerrigan

math.OCOct 22, 2026

Direct Inference on Admissible Sets (Diona)

Authors: Lorenzo Sabug, Eric Kerrigan

Abstract

We propose a new regression method named Direct Inference on Admissible Sets (Diona) that directly uses samples to estimate the state and its derivatives at a query point, along with their admissible bounds, without long global training. By casting the values and derivatives at the query and neighbouring samples as latent variables in a constrained optimisation problem, the burden of long training loops becomes one of on-demand local solves. We establish Diona's theoretical properties including finite-sample coverage and, in the noiseless case, convergence to the true state and derivatives under dense sampling. In a nonlinear regression benchmark on six functions of three to eight dimensions, Diona was more accurate than a Gaussian process (GP) on noiseless data especially with denser data sets. Furthermore, as with GP, it produced uncertainty bounds that contained the true value at more than 95% of query points in four functions. Diona's time to a first prediction after a data set update was three to four orders of magnitude shorter than that of the GP, and two to three orders shorter than that of a neural network, which was more accurate but offers no uncertainty quantification. This low latency is paid for by on-demand predictions, so this latency advantage applies when relatively few predictions are needed between data updates.

Figures & tables

Figure 1: 1-dimensional visualisation of Hessian Lipschitz-based Diona uncertainty bounds from noiseless samples, with $u(x)=\sin(2x)+0.3x^{2}$ , in comparison with a value Lipschitz-based NSM bounds.

Name	Details
Neural network (NN) [ 1 ]	Implemented with Lux.jl . Fully connected network with $\left\lceil D/2\right\rceil+1$ hidden layers (64 tanh units). Trained to minimise MSE with Adam (25000 epochs with learning rate of $1\times 10^{-3}$ ). Used CUDA via LuxCUDA.jl .
Gaussian process (GP) [ 11 ]	Sparse variational GP using AbstractGPs.jl , ApproximateGPs.jl , and KernelFunctions.jl , Matérn-5/2 kernel, and $100D$ inducing points.
Nonlinear set membership (NSM) [ 8 ]	Implemented in Julia with estimated noise bound $\tilde{\epsilon}$ and value Lipschitz constant $\tilde{\gamma}$ computed as in [ 10 ] . $\tilde{\gamma}$ is then scaled by 2.5.
Radial basis functions (RBF) [ 2 ]	Using Surrogates.jl RadialBasis with the default linear radial kernel, with all default values.
Inverse distance weighting (IDW)	Using Surrogates.jl InverseDistanceSurrogate , using all training samples. Used inverse-distance exponent $p=D+2$ , otherwise all default settings.
Diona (ours)	As described in Section 4 - 5 , and with $\tilde{L}_{H}$ scaled by 2.5. LPs are computed with HiGHS.jl .

Table 1: Compared data-driven regressors

Function name	$D$	Domain	Properties
Ishigami	3	$[-\pi,\pi]^{3}$	Oscillatory, with growing amplitude, variable interactions
Park	4	$[0,1]^{4}$	Nonseparable, spatially varying curvature
Friedman	5	$[0,1]^{5}$	Nonlinear, nonseparable, with linear and quadratic additive terms
Hartmann	6	$[0,1]^{6}$	Multimodal
Piston (simulation)	7	(see [ 14 ] )	Nonlinear physical model, input variable coupling
Rosenbrock	8	$[-5,10]^{8}$	Narrow curved valley, strong input coupling

Table 2: Benchmark functions

Figure 2: Results: relative

L_{2}

error (a) Ishigami, (b) Park, (c) Friedman, (d) Hartmann, (e) Piston, (f) Rosenbrock.

Figure 3: Results: relative uncertainty interval vs truth coverage (a) Ishigami, (b) Park, (c) Friedman, (d) Hartmann, (e) Piston, (f) Rosenbrock.

Figure 4: Results: median per-query times vs training times (a) Ishigami, (b) Park, (c) Friedman, (d) Hartmann, (e) Piston, (f) Rosenbrock. Diona (with zero training times) is placed to the left for comparative visualisation.

Regression Prediction Intervals

Regression prediction intervals quantify plausible ranges for outcomes while balancing reliable coverage against interval width.

See all 102 papers

Efficient Deep Gaussian Process Surrogates via Mini-Batch Latent MAP Estimation
Oct 27, 2026Chun-Yi Chang, Chih-Li SungDeep Gaussian Processes Surrogate Modeling
Michigan State University
Scalable Derivative Gaussian Processes via Exact Gradient Reduction
Jun 1, 2026Hyunseok Seung, Matthias KatzfussSurrogate Modeling Gaussian Processes
Department of Statistics · University of Wisconsin–Madison · Madison, WI, 53706
Variational meta-learning inference for low dimensional neural system identification
Jul 21, 2026Matteo Rufolo, Dario Piga, Marco ForgioneUncertainty Quantification Meta-Learning
SUPSI-DTI-IDSIA, Dalle Molle Institute for Artificial Intelligence, Lugano, Switzerland.

Oct 27, 2026J/K move · Enter open · S save

Efficient Deep Gaussian Process Surrogates via Mini-Batch Latent MAP Estimation

Chun-Yi Chang, Chih-Li Sung

Michigan State University

Deep Gaussian Processes Surrogate Modeling MAP Estimation

Open paper AskPDF arXiv

Deep Gaussian processes (DGPs) are flexible surrogate models for computer experiments with nonstationary response surfaces, but inference over their latent representations remains computationally challenging. Existing methods typically rely on variational inference or Markov chain Monte Carlo (MCMC), which impose variational-family assumptions or require costly posterior sampling. We propose the latent maximum-a-posteriori deep Gaussian process (LaMP-DGP), which uses gradient-based optimization to obtain MAP estimates of the latent representations and enables efficient probabilistic prediction through linked-GP calculations. To scale LaMP-DGP to large datasets, we develop a mini-batch optimization algorithm and, under a polynomial mini-batch bias-decay condition, establish a stationarity bound for latent-state optimization that motivates a growing batch-size schedule. Across numerical experiments and real-data analyses, LaMP-DGP provides consistently competitive predictive performance relative to doubly stochastic variational inference, stochastic gradient Hamiltonian Monte Carlo, and fully Bayesian Vecchia DGP inference. Computationally, LaMP-DGP is substantially faster than the Vecchia MCMC approach while remaining broadly competitive with the stochastic-gradient alternatives. These results position LaMP-DGP as a competitive and computationally efficient alternative for DGP inference. An R package, "LaMPDGP", is available for implementing the proposed method.

Abstract

Regression Prediction Intervals

Explore similar work

Efficient Deep Gaussian Process Surrogates via Mini-Batch Latent MAP Estimation

Scalable Derivative Gaussian Processes via Exact Gradient Reduction

Variational meta-learning inference for low dimensional neural system identification