cs.LGNov 15, 2026

The structure of variational autoencoder latent spaces and its consequences for representation and optimization

Authors: Sergei V. Kalinin

Organizations: Department of Materials Science and Engineering, University of Tennessee, Knoxville, Tennessee, USA

Abstract

Variational autoencoders (VAEs) are widely used to build low-dimensional representations of scientific data, with the expectation that regularization yields smooth coordinates for interpolation, generation and optimization. However, applications in microscopy reveal domains, narrow transitions and rare-state islands in VAE latent spaces that smoothness alone does not explain. We show that, for prototype data with squared-error reconstruction and an optimal decoder, the latent-space objective has a direct correspondence with the Flory-Huggins theory of a regular solution, and use it to analyze how the objective, posterior distributions and latent dimensionality organize the representation. Data families occupy domains separated by finite-width walls that broaden with increasing regularization until class separation is lost through posterior collapse. Class centroids behave like particles in a harmonic trap: the prior confines them, and structural dissimilarity sets a soft repulsion that determines which classes become neighbors and how far apart they sit, with separations growing only logarithmically with dissimilarity. Rare states can remain unresolved, form small islands, accumulate within walls or develop intervening domains, depending on their frequency and similarity to common states. The posterior covariance can align domains with the latent axes, but axis alignment does not imply separation of physical factors. Rotations can produce rings and encoder cuts that reflect data symmetry, while insufficient dimensionality can lead to suppression, mixing or folding of additional factors. Comparisons with flow models, deep kernel learning (DKL) and hybrid VAE-DKL models show how these structures change when reconstruction and prediction play different roles. We discuss implications for interpreting data, discovering rare states, generating intermediate states and optimizing properties.

Explore similar work

CardsList
  1. Entropic Auto-Encoding via Implicit Free-Energy Minimization

    May 15, 2026Hazhir Aliahmadi, Irina Babayan, Greg van AndersVariational AutoencodersPosterior Collapse

  2. Soft-Constrained Optimization of Latent Space in Variational Autoencoders

    Jul 26, 2026Ye ShiVariational AutoencodersDisentangled Representation Learning

  3. λ\mathbfλ-VAE: Variance Equalization for Posterior Collapse

    Jul 6, 2026Girum DemisseVariational AutoencodersPosterior Collapse