DE-VAE: Revealing Uncertainty in Parametric and Inverse Projections with Variational Autoencoders using Differential Entropy

Frederik L. Dennig; Daniel A. Keim

cs.LGAug 16, 2025

DE-VAE: Revealing Uncertainty in Parametric and Inverse Projections with Variational Autoencoders using Differential Entropy

Authors: Frederik L. Dennig, Daniel A. Keim

Organizations: University of Konstanz

Abstract

Recently, autoencoders (AEs) have gained interest for creating parametric and invertible projections of multidimensional data. Parametric projections make it possible to embed new, unseen samples without recalculating the entire projection, while invertible projections allow the synthesis of new data instances. However, existing methods perform poorly when dealing with out-of-distribution samples in either the data or embedding space. Thus, we propose DE-VAE, an uncertainty-aware variational AE using differential entropy (DE) to improve the learned parametric and invertible projections. Given a fixed projection, we train DE-VAE to learn a mapping into 2D space and an inverse mapping back to the original space. We conduct quantitative and qualitative evaluations on four well-known datasets, using UMAP and t-SNE as baseline projection methods. Our findings show that DE-VAE can create parametric and inverse projections with comparable accuracy to other current AE-based approaches while enabling the analysis of embedding uncertainty.

Figures & tables

Panel of Figure 1 — Figure 1: We show the encoder output $\mu$ for all test data of the learned UMAP projection of MNIST and the t-SNE projection of KMNIST. The uncertainty modeling, expressed by three Gaussian distribution types, allows for varying levels of expressiveness through their differing degrees of freedom. These are shown as 1st, 2nd, and 3rd standard deviations, depicted as ellipses around the class medoids.

Parametric projection: Average projection loss $\altmathcal{L}_{\text{proj}}$ (lower is better)
	None	Isotropic	Diagonal	Full
HAR (UMAP)	$.675\pm.685$	$.525\pm.257$	$.447\pm 0.166$	$\mathbf{.404\pm.079}$
MNIST (UMAP)	$.402\pm.064$	$\mathbf{.400\pm.077}$	$.418\pm.162$	$.428\pm 0.07$
Fashion-MNIST (UMAP)	$\mathbf{.263\pm.034}$	$.293\pm.069$	$.306\pm.059$	$.298\pm 0.05$
KMNIST (t-SNE)	$\mathbf{30.4\pm 4.68}$	$32.3\pm 3.98$	$31.6\pm 6.23$	$33.0\pm 5.27$
Inverse Projection: Average reconstruction loss $\altmathcal{L}_{\text{recon}}$ (lower is better)

Table 1: Average losses and standard deviations (after

\pm

) of the parametric and inverse projections on test data for 10 runs each, as well as average number of training epochs to compare training time.

Panel of Figure 2 — Figure 2: For each approach (a-e), we inverse project 25 samples from an evenly spaced $5\times 5$ grid on a UMAP projection of the MNIST dataset and show the results of the inverse projections ( $P^{-1}$ ). Our models (c-e) show higher quality and more diverse outputs.

Explore similar work

ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generation
Aug 11, 2026Ge WangEvidential Uncertainty
Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, New York, USA
Entropic Auto-Encoding via Implicit Free-Energy Minimization
May 15, 2026Hazhir Aliahmadi, Irina Babayan, Greg van AndersConditional Variational Autoencoder Autoencoder Architectures
Department of Physics, Engineering Physics & Astronomy, Queen’s University, Kingston, ON, Canada
Deep Invertible Autoencoders for Dimensionality Reduction of Dynamical Systems
Mar 13, 2026Nicolò Botteghi, Silke Glas, Christoph BruneReduced-Order Models Autoencoder Architectures
1MOX Laboratory, Department of Mathematics, Politecnico di Milano, Milano, Italy · Department of Applied Mathematics, University of Twente, Enschede, Netherlands

Aug 11, 2026, cs.LGJ/K move · Enter open · S save

ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generation

Ge Wang

Biomedical Imaging Center, Rensselaer Polytechnic Institute, Troy, New York, USA

Evidential Uncertainty

Open paper AskPDF arXiv

Variational autoencoders generate samples from probabilistic latent representations but do not distinguish uncertainty about the latent location from variability around it. We formulate ELVAE, an evidential learning-based VAE in which each latent coordinate is governed by an input-dependent normal-inverse-gamma posterior. This hierarchy yields an explicit latent-location uncertainty that can be used during generation, not merely reported after inference: low-uncertainty anchors support more reliable synthetic samples, while high-uncertainty anchors can be deliberately exploited for stress testing. The objective is an exact evidence lower bound, and we show that direct regularization of the full hierarchy is required, since the marginalized latent law alone cannot identify the uncertainty decomposition. In an MNIST generation pilot with a frozen external classifier, this uncertainty clearly stratified the semantic reliability of generated digits. A zero-displacement control revealed that most of the effect reflects how reliably an anchor can be re-generated, while a smaller but distinct component is attributable to uncertainty-scaled perturbation itself. The effect holds only under within-class uncertainty ranking, and its magnitude varies across seeds. These findings support the learned latent-location uncertainty as a practical control variable for uncertainty-aware generation, separating anchor reliability from perturbation-induced failure.

Abstract

Explore similar work

ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generation

Entropic Auto-Encoding via Implicit Free-Energy Minimization

Deep Invertible Autoencoders for Dimensionality Reduction of Dynamical Systems