stat.MLNov 14, 2026

A Geometric Understanding of Denoising Autoencoder with Applications to Image Denoising

Authors: Haoran Yan, Rongjie Lai, Wenjing Liao

Organizations: School of Mathematics, Georgia Institute of Technology, Atlanta, GA, USA. · Department of Mathematics, Purdue University, West Lafayette, IN, USA.

Abstract

Denoising autoencoders have shown strong empirical performance in recovering clean data from their noisy observations, yet their underlying statistical behavior remains underexplored. In this paper, we study denoising autoencoders under the manifold hypothesis, assuming that clean data are supported on low-dimensional smooth manifolds embedded in a high-dimensional ambient space, while noisy observations are generated by isotropic Gaussian noise. We establish generalization guarantees in three geometric scenarios: a single manifold, a pairwise separated finite union of manifolds, and a finite union of intersecting manifolds. Using local chart representations, partitions of unity, and neural network approximation, we derive upper bounds for the expected squared reconstruction error for denoising autoencoder. In the single-manifold and separated-union scenarios, the error consists of a sample-dependent statistical term of order n−2/(d+2)log⁡4nn^{-2/(d+2)}\log^4 n, a residual term of order dσ2/Ddσ^2/D caused by tangential noise, and an exponentially small tail term associated with failure of the manifold projection. Here DD is the data ambient dimension, dd is the intrinsic dimension of the clean-data manifold, and σ2σ^2 is the expected squared noise magnitude. In the scenario of intersecting manifolds, we further identify an additional ambiguity term determined by the probability mass near the intersection neighborhood. Numerical experiments on synthetic manifolds and real image datasets support the predicted sample-size scaling law and illustrate the additional reconstruction ambiguity near manifold intersections.

Figures & tables

Explore similar work

Sep 17, 2026

Sharp Reconstruction Bounds for Autoencoders Using the Same Forward Map

We study reconstruction in autoencoders that apply the same forward map before and after setting the observed coordinates to zero. For equal odd input and hidden dimensions d≥3d\geq 3, among orientation-preserving diffeomorphisms whose Jacobian singular values lie in [m,M][m,M], we show that the least uniform reconstruction-derivative error is max⁡{1−M(M−m)/2,0}\max\{1-M(M-m)/2,0\}, with affine maps attaining this sharp bound at every prescribed depth. A translated radial rotation can nevertheless reconstruct any prescribed ball exactly with singular values arbitrarily close to one, motivating additional conditions for a finite-data bound. We test this prediction on a 798,452-point terrestrial LiDAR forest scan. At input scale 0.050.05, the mean theoretical bound is 0.1550.155, about 84%84\% of the mean normalized training error 0.1850.185 across four spatial regions, two depths, and three seeds. At this scale, adding one hidden coordinate reduces the mean reconstruction error below 6×10−66\times10^{-6}.
May 4, 2026

Denoising data using convex relaxations

We study the problem of denoising observations Yi=Xi+ZiY_i=X_i+Z_i, where the latent variables XiX_i are sampled from a low-dimensional manifold in Rn\mathbb{R}^n and the noise variables ZiZ_i are isotropic Gaussian. We propose a convex-relaxation estimator that first reduces dimension by principal component analysis and then projects the observations onto the convex hull of the projected latent manifold. We construct a statistical oracle that estimates its supporting hyperplanes from empirical Gaussian tail probabilities of the noisy sample. Under a lower-mass condition on the latent distribution, we prove finite-sample guarantees for the oracle and derive error bounds for the resulting denoiser. The analysis combines risk bounds for least-squares projection under convex constraints (due to Chatterjee) with entropy bounds for the manifold. We also verify the assumptions of the framework for a Cryo-Electron Microscopy observation model by establishing suitable covering number and Lipschitz estimates for the associated group action and imaging operators.
Mar 17, 2026

Manifold-Matching Autoencoders: Embedding Data into Chosen Geometric Structures

Manipulating the structure of data in the latent space of an autoencoder is a long-standing goal of representation learning. We propose a simple regularization, Manifold Matching, that imposes an explicit geometric structure on that latent space. The idea is to take a reference representation R of the data, which need not be the raw input X fed to the network, and to minimize, batch-wise, the mean squared error between the pairwise distances in R and those in the latent space Z. Although the autoencoder still learns to reconstruct X, the shape of its latent space comes to mimic that of R. The objective requires no labels, only reference distances, and because those are scalars, the dimensionality of R is decoupled from that of Z: any reference can guide any bottleneck. Using this single instrument we study how different geometric structures can be imposed on a latent space, comparing against autoencoders built on geometric or topological regularizers. Across fifteen datasets under a shared protocol with statistical testing, the resulting Manifold-Matching Autoencoder (MMAE) matches the global-structure quality of persistence-based autoencoders, statistically tying RTD-AE on distance correlation and triplet accuracy, and stays competitive on the local and topological metrics that other methods lead, at the computational cost of a plain autoencoder: it uses only a batched distance loss, with no persistence computation. We frame MMAE not as a universal winner but as an efficient and controllable regularizer whose reference selects which structure the latent inherits, and which turns any fixed embedding into a reusable out-of-sample encoder. All experiments use a two-dimensional bottleneck.