Ghost tasking for parametrized Gaussian Processes solving linear differential equations

Johanna Moser; Christopher Albert; Sascha Ranftl

physics.comp-phOct 8, 2026

Ghost tasking for parametrized Gaussian Processes solving linear differential equations

Authors: Johanna Moser, Christopher Albert, Sascha Ranftl

Abstract

Physics-informed machine learning has gained significant attention in recent years. In regimes of limited data, parametrized Gaussian processes have become popular. Existing approaches, however, often face limitations, such as requiring parametrizable (also called controllable) systems or a large number of output tasks. In this work, we introduce a systematic procedure we call "ghost tasking", using auxiliary tasks to circumvent these limitations. We prove that such ghost tasks can render any non-parametrizable system effectively parametrizable, enabling algorithmic construction of parametrized Gaussian Processes while keeping the number of required tasks (i.e. output dimensions) and latent functions low. We find that ghost tasking performs especially well in an inverse problem setting, even with very few available data. We show the usage and power of ghost tasking in three experiments, providing systematic comparisons to the only other currently available method applicable to all experiments. We provide necessary syntax and explications for two computer algebra programs that compute parametrizations for systems with polynomial or rational coefficients. Our theoretical results extend to systems with meromorphic functions.

Figures & tables

Figure 2: Overview over parametrizability in operator space

D

and function space

{\cal F}

. Every combination can be read as "Systems represented by modules in operator space that are [i.e. torsion-free] can parametrize [i.e. all] solutions over the function space

{\cal F}

if [i.e.

{\cal F}

is injective]"

Figure 3: Overview over the idea behind ghost tasking and its connection to physics-informed Gaussian Processes (PIGP) as in [ 43 ] . A non parametrizable system can be rendered parametrizable by adding extra tasks, and subsequently conditioning them away again. The differences between ghost tasking and PIGPs lie in (a) how many tasks we add and (b) how they are coupled to the system (

\Lambda

versus the identity matrix

\mathbb{I}_{q}

Figure 4: Inferred solutions of the forward problem of experiment 1 for the GT method (left), GT+ with extra collocation points (middle), and the PIGP method (right). Each subplot shows the solution of one task, the last three are the derivative of the angles in the first three tasks, necessary to constrain a second order initial condition problem. The true solution is marked in red, the training data with black stars, and the inferred solutions in blue with a lightblue

2\sigma

uncertainty band. The RMSE on the test points is written in the corner of each task.

Figure 5: Results of the systematic comparison in experiment 1 of GT (green), GT+ (orange), and PIGP (blue) using the mean absolute error of the learned physical parameters (maximum a posteriori, MAP value)

\ell

(first row),

g

(middle row), as well as their ratio

g/\ell

(last row) relative to their true values

(\ell,g)=(2.5,9.81)

as metric. Note that

g/\ell

is not an independent parameter, but does dictate much of the dynamics of the system and can as such be most precisely estimated by GT+. The experiments have been conducted for different numbers of training data

n\in\{3,5,10\}

for the tasks, as well as different noise levels

\sigma\in\{0.001,0.01,0.1,0.2\}

, shown in different subplots. We show the distinct runs as scatterplots, as well as their box-plots for each combination

(\sigma

, n).

Panel of Figure 6 — Figure 6: Re-scaled log posteriors $-\log(\max(\log(p(\ell,g)))-\log(p(\ell,g))+1)$ in the $(g,\ell)$ -plane marginalized over the calibration parameters $A$ and $l$ for all combinations of $n$ and $\sigma$ . The reference Bayesian estimate for the known solution (upper left plot) is compared to GT (upper right), GT+ (lower right), and PIGP (lower left). The elongated structures show a strong correlation between $g$ and $\ell$ , and we see multimodality in the reference and GT posteriors. We do not show colorbars because the likelihood scales vary substantially between panels; the comparison is qualitative and focuses on the relative structure of each likelihood rather than absolute values compared across panels.

Figure 7: Inferred solution (uneven rows) and absolute error (even rows) of the forward problem given training data for the ghost task/input tasks and the initial conditions. The torsion-free GT version is depicted in the first two rows, the projective GT version in the middle two rows, and PIGP in the lowest two rows. The left column shows the results for $u$ , the middle column for $v$ and on the right we show the ghost task resp. input tasks for the inhomogeneity necessary. Conditioned points are marked with blue. The RMSE is computed within the region of interest, marked by a red box.

Figure 8: Relative error as performance metric of the inverse problem of experiment 2, analogously to Fig. 5 of experiment 1. The results for the torsion-free GT system are shown in green, the results for the projective GT version are shown in orange and the results for PIGP are shown in blue.

Figure 9: Quiver plot of the magnetic flux density $\vec{B}$ inferred with GT in the forward problem of experiment 3 given Dirichlet boundary conditions for $\vec{A}$ and $6\times 6\times 6$ training points for $J_{x}$ and the ghost task $\xi_{GT}$ . Direction of $\vec{B}$ is indicated with arrows, the strength is represented with the length of the arrows as well as their color.

Figure 10: Visualization of the conditioned tasks in the forward problem of Experiment 3. The left column shows two dimensional slices though the

y-z

plane at

x=0.6

of the analytic ground truth, the middle column shows the ghost tasking method, and the right column shows PIGP. The rows depict the (x,y,z) components of the vector potential

\vec{A}

, the

x

component of the current density

J_{x}

, and the ghost task

\xi_{GT}

for GT, and

J_{y}

J_{z}

respectively for PIGP. Training data is marked with blue points and the RMSE over the full volume is written in the corner of each task.

Figure 11: Visualization of the solution of the forward problem of Experiment 3 for the magnetic flux density components. The left column shows two dimensional slices though the $y-z$ plane at $x=0.6$ of the analytic ground truth, the middle column shows the ghost tasking method, and the right column shows PIGP. The rows depict the (x,y,z) components of the inferred quantity of interest, the magnetic flux density $\vec{B}$ . No training data on $\vec{B}$ is used. The RMSE over the full volume is written in the corner of each task.

Figure 12: Relative error of the MAP estimate of the parameter

\nu_{0}

as performance metric of the inverse problem of experiment 3, analogously to Fig. 5 and 8 . We see a clear advantage of the GT method (green) as compared to PIGP (blue) in all experiments.

Figure 13: Re-scaled components

p

-\log(\max(p)-p+1)

of the log marginal likelihood (Eq. 2 ) for GT (marked with x) and PIGP (marked with circles) as a function of

\nu_{0}

, conditioned on the respective MAP estimates of the calibration parameters

A

and

l

for the

n=3

training data set and all values

\sigma

evaluated on 50 points

\nu_{0}\in[-1,3]

. The first row shows the complexity term, the second row shows the data-fit term, and the last row shows the total log marginal likelihood. Curves are not normalized due to re-scaling; the relevant information is conveyed by the position and width of the peaks.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

$\frac{\Delta\ell}{\ell}/\frac{\Delta g}{g}/\frac{\Delta g/\ell}{g/\ell}$		measured $f(t)$			known $f(t)$
		GT		PIGP	GT	PIGP
n	$\sigma$	standard	+	standard
3	0.001	$0.596\,268$ / $0.231\,035$ / $0.228\,804$	$0.055\,175\,6$ / $0.055\,165\,5$ / $4.490\,98\text{\times}{10}^{-05}$	$0.066\,349\,6$ / $0.862\,723$ / $0.852\,972$	$0.000\,308\,705$ / $0.000\,353\,493$ / $3.527\,98\text{\times}{10}^{-05}$	$0.000\,328\,083$ / $0.000\,372\,496$ / $3.3207\text{\times}{10}^{-05}$
3	0.01	$0.596\,267$ / $0.231\,478$ / $0.228\,526$	$0.263\,984$ / $0.263\,757$ / $0.000\,765\,588$	$0.065\,815\,1$ / $0.862\,947$ / $0.853\,053$	$0.005\,615\,53$ / $0.005\,132\,56$ / $0.000\,447\,494$	$0.005\,626\,78$ / $0.005\,100\,16$ / $0.000\,404\,498$
3	0.1	$0.596\,229$ / $0.231\,431$ / $0.228\,536$	$0.574\,47$ / $0.575\,996$ / $0.002\,196\,45$	$0.067\,586\,2$ / $0.8603$ / $0.851\,068$	$0.111\,937$ / $0.102\,848$ / $0.006\,967\,08$	$0.096\,474\,9$ / $0.089\,278\,2$ / $0.004\,341\,57$
3	0.2	$0.596\,577$ / $0.236\,963$ / $0.225\,217$	$0.594\,656$ / $0.596\,028$ / $0.007\,831\,46$	$0.064\,340\,1$ / $0.870\,774$ / $0.860\,306$	$0.231\,128$ / $0.210\,09$ / $0.018\,385\,4$	$0.219\,964$ / $0.200\,549$ / $0.015\,383\,7$

Appendix

Table 1: Ablation study experiment 1. Median absolute error of the physical parameters (each cell:

\frac{\Delta\ell}{\ell}

\frac{\Delta g}{g}

\frac{\Delta g/\ell}{g/\ell}

) relative to their true values

(\ell,g)=(2.5,9.81)

across 10 runs. We compare the variants "standard", where every task has the same amount of training data

n

, and "+", where the ghost task is constrained on

25

points and the standard deviation of the likelihood in these tasks is fixed to be

\sigma=0.001

. We compare two settings "measured

f(t)

", where

f(t)

is provided with

n

measurements, and "known

f(t)

", where

f(t)

is provided with 25 noise-free input points for all settings. For PIGP, we cannot provide a "+" variant for "measured

f(t)

" per definition. Overall performance is calculated using the geometric mean of the above results and shown in the last row. We see similar performances between GT and PIGP, while "GT+" outperforms both drastically especially for

g/\ell

in the measured case. For known

f(t)

, performances are similar.

Figure 14: Re-scaled log posteriors of

g,\ell

analogously to Fig. 6 assuming

f(t)

is known. We recover similar multimodal structures as we did in the original experiment, but this time for both methods.

$\frac{\Delta\alpha}{\alpha}$		GT (torsion-free)		GT (projective)		PIGP
n	$\sigma$	standard	+	standard	+	standard	+
3	0.001	$0.025\,008\,9$	0.000663676	$0.007\,629\,62$	0.000510805	$0.005\,829\,49$	0.000355886
3	0.01	$0.025\,117\,2$	0.00308351	0.0103383	$0.012\,536\,1$	$0.008\,65$	0.00339176
3	0.1	0.0360237	$0.043\,599\,6$	0.0333033	$0.083\,099\,9$	0.044	$0.109\,045$
3	0.2	0.0953646	$0.255\,076$	0.168597	$0.725\,525$	0.109904	$0.641\,046$
5	0.001	$0.000\,476\,128$	0.000189513	$0.000\,152\,691$	0.000141994	$0.000\,345\,346$	0.000185464

Appendix

Table 2: Ablation study experiment 2. Median absolute error of the physical parameter

\alpha

relative to its true value

\alpha_{true}=2

, analogously to 1 . For the "+" variant, the ghost task resp. the inhomogeneities (for GT resp. PIGP) are constrained on

15\times 15

points and the standard deviation of the likelihood in these tasks is fixed to be

\sigma=0.001

. Contrary to experiment 1 ( 1 ), we do not see significant differences between variants in the results.

$\frac{\Delta\nu_{0}}{\nu_{0}}$		GT		PIGP
n	$\sigma$	standard	+	standard	+
3	0.001	$0.101\,25$	0.0809063	0.342445	$0.483\,184$
3	0.01	$0.096\,035\,8$	0.0731214	0.345774	$0.494\,206$
3	0.1	$0.083\,456\,5$	0.0539069	0.386825	$0.525\,768$
3	0.2	$0.077\,833\,1$	0.0469981	0.434896	$0.551\,056$
5	0.001	$0.000\,944\,797$	0.000884615	0.32439	$0.377\,715$

Appendix

Table 3: Ablation study experiment 3. Median absolute error of the physical parameter

\nu_{0}

relative to its true value

\nu_{0true}=1

, analogously to 1 . For the "+" variant, the ghost task resp. the inhomogeneities (for GT resp. PIGP) are constrained on

10\times 10\times 10

points and the standard deviation of the likelihood in these tasks is fixed to be

\sigma=0.001

. GT outperforms PIGP in every setting. Contrary to experiment 1 ( 1 ), we do not see significant differences between variants in the results.

Explore similar work

May 20, 2026stat.ML

Conditioning Gaussian Processes on Almost Anything

Gaussian processes (GPs) offer a principled probabilistic model over functions, but exact inference is restricted to the linear-Gaussian regime. We establish an explicit equivalence between GPs and a class of linear diffusion models, recasting predictive sampling as an ODE with closed-form Gaussian dynamics and a likelihood-dependent guidance term that admits a simple Monte Carlo approximation. In the linear-Gaussian setting, we recover standard GP conditioning exactly; beyond conjugacy, the same machinery handles any conditioning statement admitting point-wise likelihood evaluation -- including non-linear physics, and, for the first time, natural language via large language models. Whitening isolates the irreducible non-Gaussian dynamics, minimising Wasserstein-2 transport cost and eliminating numerical stiffness. The result is a general-purpose GP inference scheme requiring no bespoke derivations. Together, these results provide a general mechanism for incorporating the full richness of real-world knowledge as conditioning information, opening a new frontier for the probabilistic modelling of real-world problems.

Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite +5

Lancaster University · University of New South Wales · University of Cambridge +1

Gaussian Processes Conditional Distribution Estimation

Ask PDF

May 18, 2026cs.LG

Lightweight Gaussian Process Inference in C++ on Metal and CUDA

Gaussian process (GP) inference in Python is dominated by libraries such as GPyTorch and GPflow, which are built on deep-learning frameworks and inherit their dispatch overhead and dependency footprint. We present LightGP, a dependency-free C++17 library for GP regression with Python bindings, supporting Apple Metal and NVIDIA CUDA backends alongside tuned CPU paths via Apple Accelerate and OpenBLAS. LightGP provides four inference paths -- exact Cholesky, matrix-free conjugate gradients, sparse variational free energy, and structured kernel interpolation with FFT -- covering problems from

N{=}100

N{=}500{,}000

. On an Apple M4, LightGP CPU is 2.6--8.7

\times

faster than GPyTorch CPU for exact GP and

{\sim}1.5\times

faster for sparse GP at every scale tested. On an NVIDIA RTX~3060, LightGP CUDA is 2.3--6.7

\times

faster than GPyTorch CUDA for exact GP up to

N{=}2{,}048

, with GPyTorch closing the gap at

N{=}4{,}096

. A fused matrix-free kernel-vector product on Metal achieves 32

\times

over the explicit path at

N{=}20{,}000

with

O(N)

memory, and an FFT-accelerated SKI matvec via Accelerate vDSP runs in sub-millisecond time at

N{=}200{,}000

. LightGP compiles as a single static library with zero external dependencies and is installable via \texttt{pip install lightgp

Yu-Hsueh Fang

Department of Information Management, National Taiwan University · 2H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology

GPU Kernel Optimization GPU Acceleration High-Performance Computing Gaussian Processes Gaussian Process Regression

Ask PDF

May 11, 2026stat.ML

Scalable Gaussian process inference via neural feature maps

We present a theoretically grounded Gaussian process framework that leverages neural feature maps to construct expressive kernels. We show that the learned feature map can be interpreted as an optimal low-rank approximation to a Gram matrix derived from an implied RKHS, from which we establish consistency of the GP posterior. We further analyse the spectral properties of the induced kernels and introduce product feature-map kernels to address oversmoothing. This simple yet powerful approach enables fast, scalable, and accurate exact GP inference with minimal upfront work. The flexibility of kernel design supports seamless application to both regression and classification tasks across diverse data modalities, including tabular inputs and structured domains such as images. On benchmark datasets, this approach surpasses pre-existing methods in terms of accuracy and training and prediction efficiency.

Anthony Stephenson

School of Mathematics University of Bristol

Gaussian Processes Kernel Methods Gaussian Process Regression

Ask PDF