Variational Atlas Autoencoders and Self-Supervised Chart Selection

Arno Coomans; Merlin Nimier-David; Eugene d'Eon; Boris Bonev

cs.LGOct 24, 2026

Variational Atlas Autoencoders and Self-Supervised Chart Selection

Authors: Arno Coomans, Merlin Nimier-David, Eugene d'Eon, Boris Bonev

Organizations: NVIDIA · Graz University of Technology, Austria

Abstract

Complex, high-dimensional data lie on manifolds that a single global latent coordinate system cannot represent without distortion. We introduce Variational Atlas Autoencoders, a mixture-of-charts framework that learns an atlas of local latent representations, each responsible for one region of the data manifold. The central difficulty in learning an atlas is deciding which chart should be responsible for which data point without labels. We show that this decision need not be a heuristic. By treating the chart index as a latent variable, the evidence lower bound yields the training objective and casts chart assignment as posterior inference, valid for any routing distribution. Within this framework, we propose latent-evidence routing, which assigns a point to a chart in proportion to the evidence of its encoding under that chart's latent prior. This has a simple interpretation: the closer a point lands to the center of a chart's latent space, and the more confidently it is placed there, the more likely it is to belong to that chart. Chart assignment thus emerges as posterior inference, depends on the encoder alone, and is fully self-supervised, so the charts discover structure in the data on their own, separating classes and subclasses such as distinct types of shoes or faces without access to labels. We further recover an atlas from the trained charts by identifying their overlaps and transition maps, enabling continuous interpolation within and across charts that stays close to the data manifold rather than traversing unsupported regions of latent space. Compared to capacity-matched baselines, Variational Atlas Autoencoders yields better-structured latent spaces and higher sample quality on image datasets including CelebA, while sampling requires evaluating only a single chart's decoder.

Figures & tables

Figure 1: Variational Atlas Autoencoders specialize to meaningful subsets of the data manifold with no label supervision. Left : a genus-2 surface covered by four $2$ D charts, colored by the chart claiming each region, shows that multiple charts avoid the distortion of a single global 2D latent space. Right : generated samples per chart on three image datasets. On MNIST ( LeCun et al., 1998 ) and Fashion-MNIST ( Xiao et al., 2017 ) ( $16$ -D latents), a chart typically covers one class or a related pair; on CelebA ( Liu et al., 2015 ) ( $128$ -D latents), charts split by facial features, illumination, and background.

Panel of Figure 2 — Figure 2: Top : we train $C$ encoder–decoder pairs to model the data manifold using independent charts. Our training objective, the ELBO, decomposes into per-chart losses weighted by a routing probability. The KL divergence on the chart selection prior encourages all charts to be used with equal probability. Bottom : sampling from a chart is then as simple as selecting a chart $c$ uniformly at random, sampling a latent $\mathbf{z}_{c}$ from the standard normal prior, and decoding it. Moment matching can optionally be added, see Sec. 3.2 .

Figure 3: Illustration of latent-evidence routing. Two inputs (left) are encoded by every chart; the evidence of each encoded point under its chart’s latent prior induces a distribution over charts, which is used for routing. For the histogram (right), the temperature parameter was artificially raised to $T=10$ to make the distribution easier to visualize.

Figure 4: An atlas of chart autoencoders can create smooth latent interpolation within each chart as well as across multiple charts. After training, we precompute a transition graph (right). Interpolation between any two examples in any two charts can then be achieved by following the shortest path in that graph. Within each chart’s latent space, we interpolate linearly (left).

Figure 5: Sampling a model trained on Fashion-MNIST ( Xiao et al., 2017 ) along the two dominant latent PCA axes generates fewer degenerate samples than a parameter-matched vanilla VAE ( Kingma and Welling, 2014 ) . The charts also specialize without supervision, e.g. by placing bags without handles in a separate chart. For readability, we show 16 out of the 20 charts.

Panel of Figure 6 — Figure 6: Interpolations between sequences of randomly selected training samples (green). Our method may walk through one or more charts via portals ( Sec. 3.3 ), denoted by orange vertical bars. The vanilla VAE traverses its single global latent space in a straight line, which leads to several nonsensical generations along the way. In contrast, our method consistently stays on the data manifold, even if it entails covering more distance in latent space.

	Sample FID			Reconstruction FID
Dataset	Plain VAE	VaDE	VAA (ours)	Plain VAE	VaDE	VAA (ours)
MNIST	19.84 $\pm$ 0.34	14.95 $\pm$ 0.54	13.74 $\pm$ 0.55	16.81 $\pm$ 0.23	20.54 $\pm$ 0.12	18.25 $\pm$ 0.13
Fashion	83.27 $\pm$ 1.62	45.07 $\pm$ 1.62	37.73 $\pm$ 1.12	31.27 $\pm$ 0.32	43.49 $\pm$ 0.15	32.83 $\pm$ 0.38
CelebA	30.43 $\pm$ 0.59	28.01 $\pm$ 0.26	27.84 $\pm$ 0.42	16.66 $\pm$ 0.18	22.09 $\pm$ 0.07	17.24 $\pm$ 0.25

Table 1: Sampling and reconstruction quality for a plain VAE ( Kingma and Welling, 2014 ) , VaDE ( Jiang et al., 2017 ) , and our atlas model (10 charts), all using the same number of parameters and training iterations. Quality is measured by the Fréchet Inception Distance (Inception-v3, Szegedy et al. (2016) ), averaged over three runs (mean

\pm

std). Lower is better.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

	MNIST	Fashion-MNIST	CelebA
Convolutional stages	2	2	4
Channel widths	64, 128	64, 128	64, 128, 256, 512
Bottleneck channels	13	6	51
Bottleneck resolution	$7\times 7$	$7\times 7$	$4\times 4$

Appendix

Table 2: Resolved DCGAN dimensions per dataset: channel widths of the shared convolutional network and the size of the feature map passed to the per-chart linear layers.

Hyperparameter	MNIST	Fashion-MNIST	CelebA
Charts	10	20	10
Latent dimension	16	16	128
Reconstruction loss	Bernoulli (BCE)	Bernoulli (BCE)	L1 + LPIPS, scale $\times 3$
$T$ (routing temperature)	Fixed at 1.0	Fixed at 1.0	Annealed $6.0\to 2.0$
$\beta$ (KL weight)	0.5	0.1	0.1
$\gamma$ (load-balancing weight)	50, entire run	50, entire run	50, entire run

Appendix

Table 3: Tuned hyperparameters for MNIST, Fashion-MNIST, and CelebA.

Hyperparameter	Value
Backbone	MLP, hidden width 512, depth 5
Charts	4
Latent dimension	2
Reconstruction loss	Squared error, no output squashing, scale $\times 200$
$T$ (routing temperature)	annealed $1.0\to 0.2$
$\beta$ (KL weight)	1.5

Appendix

Table 4: Tuned hyperparameters for the double torus.

		Latent-evidence routing
Dataset	ELBO	$\mu$ -based	Resampling	Expected log-prior	Prior-marginal
MNIST	11.80 $\pm$ 0.10	12.99 $\pm$ 0.22	13.79 $\pm$ 1.00	12.77 $\pm$ 0.33	12.62 $\pm$ 0.53
Fashion-MNIST	43.64 $\pm$ 0.34	38.42 $\pm$ 0.49	39.77 $\pm$ 1.46	38.03 $\pm$ 0.64	38.53 $\pm$ 0.45
CelebA	47.14 $\pm$ 0.19	43.46 $\pm$ 0.53	44.05 $\pm$ 0.24	43.89 $\pm$ 0.55	43.69 $\pm$ 0.24

Appendix

Table 5: Sample FID (mean

\pm

std over seeds) by routing function, DCGAN backbone,

C=10

charts, and iteration counts matched within each dataset. Lower is better; best in bold, second best underlined.

Panel of Figure 7 — Figure 7: 10-chart Variational Atlas Autoencoders (ours) and 1-chart capacity-matched VAE reconstructions of randomly selected held-out inputs from MNIST, Fashion-MNIST and CelebA.

Figure 8: Interpolation trajectories between two charts through a portal on the double-torus geometry. For any two points on the torus, we can create interpolation trajectories in our charts’ latent spaces that remain close to the torus surface (the data manifold). The red line illustrates straight-line interpolation through a global latent space, which leaves the surface and thus passes through implausible points. Optimizing portal placement to select shorter paths is left as future work.

Panel of Figure 9 — Figure 9: Sample images drawn from 10-, 10-, and 20-chart VAAs trained on MNIST, CelebA, and Fashion-MNIST, respectively. Each grid represents samples generated from a distinct chart. Samples from a capacity-matched VAE are shown on the right.

Variational Autoencoders

Variational autoencoders are probabilistic encoder–decoder models for learning latent representations, generating data, and estimating uncertainty.

See all 209 papers

Learning Tangent Bundles and Characteristic Classes with Autoencoder Atlases
Feb 26, 2026Eduardo Paluzo-Hidalgo, Yuichi IkeManifold Learning Autoencoders
Department of Applied Mathematics I, University of Sevilla, Sevilla, Spain · Graduate School of Mathematical Sciences, The University of Tokyo, Tokyo, Japan
Constructing VAE Latent Spaces with Prescribed Topology
Jun 5, 2026Jilles S. van Hulst, Jakub M. Tomczak, W. P. M. H. Heemels +1Variational Autoencoders Manifold Learning
Control Systems Technology Section, Department of Mechanical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands. · The research is carried out as part of the ITEA4 20216 ASIMOV project.
Entropic Auto-Encoding via Implicit Free-Energy Minimization
May 15, 2026Hazhir Aliahmadi, Irina Babayan, Greg van AndersVariational Autoencoders Posterior Collapse
Department of Physics, Engineering Physics & Astronomy, Queen’s University, Kingston, ON, Canada

Feb 26, 2026J/K move · Enter open · S save

Learning Tangent Bundles and Characteristic Classes with Autoencoder Atlases

Eduardo Paluzo-Hidalgo, Yuichi Ike

Department of Applied Mathematics I, University of Sevilla, Sevilla, Spain · Graduate School of Mathematical Sciences, The University of Tokyo, Tokyo, Japan

Manifold Learning Autoencoders

Open paper AskPDF arXiv

We introduce a theoretical framework that connects multi-chart autoencoders in manifold learning with the classical theory of vector bundles and characteristic classes. Rather than viewing autoencoders as producing a single global Euclidean embedding, we treat a collection of locally trained encoder-decoder pairs as a learned atlas on a manifold. We show that any reconstruction-consistent autoencoder atlas canonically defines transition maps satisfying the cocycle condition, and that linearising these transition maps yields a vector bundle coinciding with the tangent bundle when the latent dimension matches the intrinsic dimension of the manifold. This construction provides direct access to differential-topological invariants of the data. In particular, we show that the first Stiefel-Whitney class can be computed from the signs of the Jacobians of learned transition maps, yielding an algorithmic criterion for detecting orientability. We also show that non-trivial characteristic classes provide obstructions to single-chart representations, and that the minimum number of autoencoder charts is determined by the good cover structure of the manifold. Finally, we apply our methodology to low-dimensional orientable and non-orientable manifolds, as well as to a non-orientable high-dimensional image dataset.

Abstract

Variational Autoencoders

Explore similar work

Learning Tangent Bundles and Characteristic Classes with Autoencoder Atlases

Constructing VAE Latent Spaces with Prescribed Topology

Entropic Auto-Encoding via Implicit Free-Energy Minimization