stat.MLOct 2, 2026
SaveAmortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models
Organizations: LMU Munich · Munich Center for Machine Learning (MCML) · Imperial College London · University of Technology Nuremberg (UTN) · Helmholtz AI · Xyme
Abstract
Many machine learning methods aim to approximate the lower-dimensional manifold on which the data lives. A desirable feature of such methods is that they should capture the epistemic uncertainty of this learned manifold. One model that achieves this is the Gaussian Process Latent Variable Model, in which a Gaussian Process (GP) mapping from the latent space provides an estimate of the uncertainty of the manifold. However, the effectiveness of this uncertainty estimation is limited by the mean-field variational approximation between the GP inducing points and the latent variables. In this work, we apply Amortized Structured Stochastic Variational Inference to allow the variational posterior for the latent space to be conditionally dependent on the value of the inducing points. We demonstrate that this more flexible variational posterior improves several metrics relating to the reconstruction of points on the data manifold.
Figures & tables
| Dataset | GPLVM type | Train RMSE | Test RMSE | Train NLPD | Test NLPD | Final neg. ELBO |
|---|---|---|---|---|---|---|
| Oilflow | MF | |||||
| A-MF | ||||||
| IWVI-GPLVM | – | |||||
| A-S (Ours) | ||||||
| qPCR | MF | |||||
| A-MF |
Table 1 : Performance comparison on the Oilflow, qPCR, MNIST, and FMNIST datasets under the standard configuration. Values are reported as mean standard deviation over 5 runs. The final negative ELBO is not reported for IWVI-GPLVM because it optimizes a different importance-weighted objective that is not directly comparable to the standard ELBO.
Figure 1 : The training NLPD progression during training across all datasets. The A-S-GPLVM and A-MF-GPLVM converge at similar rates, but the A-S-GPLVM converges to a better optimum.
Figure 2 : The three latent spaces and reciprocal length scales for the oil flow dataset. The A-S-GPLVM has found a slightly more tightly clustered latent space and has been more aggressive about turning off non-informative dimensions.
Figure 3 : The distribution of training points in latent space for FMNIST for the A-S-GPLVM and A-MF-GPLVM. The A-S-GPLVM latent space has learned substantially more structure than the A-MF-GPLVM. In fact, the A-S-GPLVM appears to have collapsed the latent predictions into a semantically meaningful curve within the latent space, capturing the relative similarity of the classes with sandals, sneakers, and ankle boots at one end and trousers and dresses at the other. The A-MF-GPLVM seems to try to capture a similar shape, but achieves a much less clear separation between the groupings.
| Dataset | GPLVM type | Train CRPS | Test CRPS | Train coverage | Test coverage |
|---|---|---|---|---|---|
| Oilflow | MF | ||||
| A-MF | |||||
| IWVI-GPLVM | |||||
| A-S (Ours) | |||||
| qPCR | MF | ||||
| A-MF |
Table 2 : Predictive uncertainty evaluation on the Oilflow, qPCR, MNIST, and FMNIST datasets. Values are reported as mean standard deviation over 5 runs. The best result for each dataset and metric is shown in bold.
Figure 4 : Sample reconstructions using the A-S-GPLVM. Each image shows the posterior predictive mean of the reconstructed observation. The model is able to produce a reasonable quality of reconstruction, although the difference in quality to the A-MF-GPLVM (see Figure 13 , Appendix I ) is not immediately obvious.
Figure 5 : Latent slice through the most informative latent dimension and a histogram of the learned posterior distribution of the training data along the horizontal axis. Curves show predictive means; shaded areas denote predictive uncertainty for each output. The epistemic uncertainty at points far from the training data clearly increases more rapidly with the A-S-GPLVM than with the MF models.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6 : Graphical models comparing the GPLVM variants. Left: Amortized mean-field (A-MF-GPLVM), where the variational posteriors of the latent variables, , are independent of the inducing point values, . Right: Amortized Structured (A-S-GPLVM, ours ), which conditions the value of the latent variable variational posterior on the global inducing variables, capturing dependencies between local and global variational factors. The dashed line indicates a variable used for amortization.
| Dataset | RMSE (train) | RMSE (test) | NLPD (train) | NLPD (test) | ELBO |
|---|---|---|---|---|---|
| Oilflow | |||||
| qPCR | |||||
| MNIST | |||||
| FMNIST |
Table 3: Two-sided Welch’s -test -values over independent runs comparing A-MF-GPLVM and A-S-GPLVM.
| Metric | A-MF-GPLVM | A-S-GPLVM |
|---|---|---|
| Wall-clock time (s) | ||
| Peak GPU memory (MB) |
Table 4: Empirical runtime and peak GPU memory over training iterations on Oilflow, using , , , and float64 precision.
Figure 7 : Ablation over encoder architecture on the Oilflow dataset. We compare A-MF-GPLVM and A-S-GPLVM across different two-hidden-layer MLP encoder widths.
Figure 8 : Ablation over the number of inducing points on the Oilflow dataset. We compare A-MF-GPLVM and A-S-GPLVM for different values of the number of inducing points.
Figure 9 : Ablation over the number of sampled inducing-variable states on the Oilflow dataset.
Figure 10 : Latent slice through the most informative latent dimension. Curves show predictive means; shaded areas denote predictive uncertainty for each output. The A-S-GPLVM GP mapping has significantly larger predictive uncertainty away from the data.
Figure 11 : Latent space comparison on the qPCR dataset, for (a) MF-GPLVM, (b) A-MF-GPLVM, (c) A-S-GPLVM.
Figure 12 : Training dynamics comparison across inference methods on the qPCR dataset. The panels show the ELBO, RMSE, and NLPD trajectories over training iterations. A-S-GPLVM converges to a lower final NLPD than MF-GPLVM and A-MF-GPLVM.
Figure 13 : Sample reconstructions using the A-MF-GPLVM. Each image shows the posterior predictive mean of the reconstructed observation.
Figure 14 : Latent spaces for the MNIST dataset.
Figure 15 : Training dynamics comparison across inference methods on the MNIST dataset. The panels show the ELBO, RMSE, and NLPD trajectories over training iterations.
Figure 16 : Training dynamics comparison across inference methods on the Fashion-MNIST dataset. The panels show the ELBO, RMSE, and NLPD trajectories over training iterations.