Manifold-Matching Autoencoders

Laurent Cheret; Vincent Létourneau; Isar Nejadgholi; Chris Drummond; Hussein Al Osman; Maia Fraser

cs.LGMar 17, 2026

Manifold-Matching Autoencoders

Authors: Laurent Cheret, Vincent Létourneau, Isar Nejadgholi, Chris Drummond, Hussein Al Osman, Maia Fraser

Organizations: Department of Computer Science, University of Ottawa, Ottawa, Canada · MILA, Montréal, Canada · National Research Council of Canada, Ottawa, Canada

Abstract

We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error. Because alignment occurs on pairwise distances rather than coordinates, it can also be extended to a lower-dimensional representation of the data, adding flexibility to the method. We find that this regularization outperforms similar methods on metrics based on preservation of nearest-neighbor distances and persistent homology-based measures. We also observe that MMAE provides a scalable approximation of Multi-Dimensional Scaling (MDS).

Figures & tables

Figure 1: Left: Overview of the current approach. The Manifold-Matching regularization MM-reg is added to the objective function of the standard AE, forming MMAEs. Top Right: 2D latent spaces of the Nested Spheres dataset ( Moor et al., 2020b ) . Standard AE (Vanilla) using no MM-reg and 9 MMAE models using different number of PCA components in their regularization ( $1\rightarrow 100$ ). Bottom Right: MMAE 2D latent spaces “copying” 2D embeddings from UMAP, t-SNE, and PCA across MNIST, F-MNIST, and CIFAR10 datasets.

Figure 2: 2D latent spaces of synthetic shapes Left: Quantitative metrics. Average over 5 runs (optimized for metric $\text{KL}_{0.1}$ ) Right: 2D Latent representations: a) Standard AE (Vanilla); b) MMAE; c) TopoAE; d) RTD-AE; e) GeomAE; f) GGAE; g) SPAE.

Dataset	Method	Rec $\downarrow$	DC $\uparrow$	TA $\uparrow$	KL 0.1 $\downarrow$	Trust $\uparrow$	Cont $\uparrow$	W 0 $\downarrow$
Paul15	Vanilla AE	0.94	0.42	0.58	0.14	0.67	0.70	272.1
	MMAE (Ours)	0.94	0.85	0.77	0.07	0.69	0.82	269.5
	TopoAE	0.95	0.50	0.67	0.07	0.67	0.80	269.5
	RTD-AE	0.96	0.58	0.69	0.07	0.66	0.79	269.9
	GeomAE	0.94	0.37	0.59	0.12	0.68	0.71	273.2
	GGAE	0.94	0.23	0.57	0.13	0.63	0.65	277.2

Table 1: Representation quality on real-world datasets. Best per dataset in bold . Latent space dimensionality for each dataset: Paul15 (2D), MNIST (16D), FMNIST (64D), PBMC3K (64D), CIFAR10 (128D).

Figure 3: Training time versus batch size. MMAE scales similarly to the standard AE (Vanilla). RTD-AE limited to batch size of 80.

Multidimensional Scaling

See all 11 papers

MEDAL: Manifold Embedding Distillation via Autoencoder Learning
May 22, 2026Irene Chang, Tarek M. Zikry, Genevera I. AllenAutoencoders Manifold Learning
Department of Statistics, Columbia University, New York, NY 10027 · School of Data and Information Sciences, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 · Irving Institute for Cancer Dynamics, Columbia University, New York, NY 10027 +1
Partially Linear Autoencoders for Manifold Learning and Dimensionality Reduction
Aug 30, 2026Louen Pottier, Louis Lesueur, Anders ThorinAutoencoders Manifold Learning
CEA List, Université Paris-Saclay · Palaiseau, F-91120, France · LMS, École Polytechnique, Institut Polytechnique de Paris +1
Bi-Lipschitz Autoencoder With Injectivity Guarantee
Apr 8, 2026Qipeng Zhan, Zhuoping Zhou, Zexuan Wang +2Autoencoders Representation Learning
University of Pennsylvania

May 22, 2026J/K move · Enter open · S save

MEDAL: Manifold Embedding Distillation via Autoencoder Learning

Irene Chang, Tarek M. Zikry, Genevera I. Allen

Department of Statistics, Columbia University, New York, NY 10027 · School of Data and Information Sciences, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 · Irving Institute for Cancer Dynamics, Columbia University, New York, NY 10027+1

Autoencoders Manifold Learning Dimensionality Reduction

Open paper AskPDF arXiv

Low-dimensional embeddings are widely used as visual summaries of high-dimensional data and to enable downstream scientific discoveries. Yet, popular nonlinear dimension reduction methods, such as t-SNE and UMAP, are often selected based on visual appeal alone and without rigorous quantitative validation. A major reason is that manifold embeddings typically do not provide an out-of-sample map nor an inverse back to the original feature space; this makes held-out validation, the gold standard in supervised learning, all but impossible. To address these challenges, we develop a novel framework, MEDAL (Manifold Embedding Distillation via Autoencoder Learning), which distills a fitted manifold embedding into a reusable encoder--decoder model. MEDAL trains a constrained autoencoder whose bottleneck exactly matches any teacher embedding while the decoder reconstructs the original input; this yields an explicit map for new samples, an approximate inverse, and a pointwise reconstruction-based measure of distortion in the manifold space. This converts static manifold embeddings into models that can be evaluated on held-out data, enabling quantitative validation including comparing different dimension reduction methods as well as hyperparameter tuning. Across multiple benchmark and scientific case studies, we show that MEDAL enables held-out validation to determine optimal manifold embeddings and hyperparameters, reveals biologically coherent regions that are difficult to preserve in two dimensional embeddings, and detects distribution shift when new samples are mapped into a fixed reference manifold. MEDAL provides a general validation wrapper to any existing dimension reduction technique that will improve the rigor and reliability of dimension reduction in scientific workflows.

Abstract

Multidimensional Scaling

Explore similar work

MEDAL: Manifold Embedding Distillation via Autoencoder Learning

Partially Linear Autoencoders for Manifold Learning and Dimensionality Reduction

Bi-Lipschitz Autoencoder With Injectivity Guarantee