Manifold Learning

Ask about this topic

Overview

Manifold learning seeks compact representations of high-dimensional data by modeling it as lying near a lower-dimensional, possibly curved space, enabling visualization, denoising, scientific analysis, and learning with fewer effective degrees of freedom. Common approaches build neighborhood graphs and optimize UMAP- or t-SNE-style embeddings, recover coordinates or operators with spectral and diffusion methods, or use autoencoders with geometric constraints; Riemannian and optimal-transport methods handle structured objects such as covariance matrices and probability distributions. Work increasingly targets properties beyond nearest-neighbor preservation—including density, geodesic distances, topology, tangent spaces, symmetry, and out-of-sample mapping—in settings from images and single-cell data to networks and dynamical systems. A central challenge is balancing local fidelity against global geometry and computational scale, while noise, sparse or anisotropic sampling, and inconsistent evaluation metrics make it difficult to tell when an embedding reflects the underlying manifold.

Latest papers

CardsList

115 papers10 papers in the last four weeks, up 43% on the four weeks before. 0.1% of all new papers.

Apr 1, 2026

Differentially Private Manifold Denoising

We introduce a differentially private manifold denoising framework that allows users to exploit sensitive reference datasets to correct noisy, non-private query points without compromising privacy. The method follows an iterative procedure that (i) privately estimates local means and tangent geometry using the reference data under calibrated sensitivity, (ii) projects query points along the privately estimated subspace toward the local mean via corrective steps at each iteration, and (iii) performs rigorous privacy accounting across iterations and queries using (ε,δ)(\varepsilon,δ)-differential privacy (DP). Conceptually, this framework brings differential privacy to manifold methods, retaining sufficient geometric signal for downstream tasks such as embedding, clustering, and visualization, while providing formal DP guarantees for the reference data. Practically, the procedure is modular and scalable, separating DP-protected local geometry (means and tangents) from budgeted query-point updates, with a simple scheduler allocating privacy budget across iterations and queries. Under standard assumptions on manifold regularity, sampling density, and measurement noise, we establish high-probability utility guarantees showing that corrected queries converge toward the manifold at a non-asymptotic rate governed by sample size, noise level, bandwidth, and the privacy budget. Simulations and case studies demonstrate accurate signal recovery under moderate privacy budgets, illustrating clear utility-privacy trade-offs and providing a deployable DP component for manifold-based workflows in regulated environments without reengineering privacy systems.

Jiaqi Wu, Yiqing Sun, Zhigang Yao

Department of Statistics and Data Science, National University of Singapore

Privacy-Preserving MLDifferential PrivacyManifold Learning
AskPDF
Apr 1, 2026

Denoising distances beyond the volumetric barrier

We study the problem of reconstructing the latent geometry of a dd-dimensional Riemannian manifold from a random geometric graph. While recent works have made significant progress in manifold recovery from random geometric graphs, and more generally from noisy distances, the precision of pairwise distance estimation has been fundamentally constrained by the volumetric barrier, namely the natural sample-spacing scale n−1/dn^{-1/d} coming from the fact that a generic point of the manifold typically lies at distance of order n−1/dn^{-1/d} from the nearest sampled point. In this paper, we introduce a novel approach, Orthogonal Ring Distance Estimation Routine (ORDER), which achieves a pointwise distance estimation precision of order n−2/(d+5)n^{-2/(d+5)} up to polylogarithmic factors in nn in polynomial time. This strictly beats the volumetric barrier for dimensions d>5d > 5. As a consequence of obtaining pointwise precision better than n−1/dn^{-1/d}, we prove that the Gromov--Wasserstein distance between the reconstructed metric measure space and the true latent manifold is of order n−1/dn^{-1/d}. This matches the Wasserstein convergence rate of empirical measures, demonstrating that our reconstructed graph metric is asymptotically as good as having access to the full pairwise distance matrix of the sampled points. Our results are proven in a very general setting which includes general models of noisy pairwise distances, sparse random geometric graphs, and unknown connection probability functions.

Han Huang, Pakawut Jiradilok, Elchanan Mossel

Manifold LearningLatent Space Geometry
AskPDF
Mar 30, 2026

Diffusion Maps is not Dimensionality Reduction

Diffusion maps (DMAP) are often used as a dimensionality-reduction tool, but more precisely they provide a spectral representation of the intrinsic geometry rather than a complete charting method. To illustrate this distinction, we study a Swiss roll with known isometric coordinates and compare DMAP, Isomap, and UMAP across latent dimensions. For each representation, we fit an oracle affine readout to the ground-truth chart and measure reconstruction error. Isomap most efficiently recovers the low-dimensional chart, UMAP provides an intermediate tradeoff, and DMAP becomes accurate only after combining multiple diffusion modes. Thus the correct chart lies in the span of diffusion coordinates, but standard DMAP do not by themselves identify the appropriate combination.

Julio Candanedo, Alejandro Patiño

SparseTrace.ai. Appleton, WI, USA. · Universidad Nacional de Colombia sede Manizales. Manizales, Caldas, Colombia

Manifold LearningDimensionality Reduction
AskPDF
Mar 22, 2026

Accelerate Vector Diffusion Maps by Landmarks

We propose a landmark-constrained algorithm, LA-VDM (Landmark Accelerated Vector Diffusion Maps), to accelerate the Vector Diffusion Maps (VDM) framework built upon the Graph Connection Laplacian (GCL), which captures pairwise connection relationships within complex datasets. LA-VDM introduces a novel two-stage normalization that effectively address nonuniform sampling densities in both the data and the landmark sets. Under a manifold model with the frame bundle structure, we show that we can accurately recover the parallel transport with landmark-constrained diffusion from a point cloud, and hence asymptotically LA-VDM converges to the connection Laplacian. The performance and accuracy of LA-VDM are demonstrated through experiments on simulated datasets and an application to nonlocal image denoising.

Sing-Yuan Yeh, Yi-An Wu, Hau-Tieng Wu +1

Manifold Learning
AskPDF
Mar 20, 2026

Continual Learning as Shared-Manifold Continuation Under Compatible Shift

Continual learning methods usually preserve old behavior by regularizing parameters, matching old outputs, or replaying previous examples. These strategies can reduce forgetting, but they do not directly specify how the latent representation should evolve. We study a narrower geometric alternative for the regime where old and new data should remain on the same latent support: continual learning as continuation of a shared manifold. We instantiate this view within Support-Preserving Manifold Assimilation (SPMA) and evaluate a geometry-preserving variant, SPMA-OG, that combines sparse replay, output distillation, relational geometry preservation, local smoothing, and chart-assignment regularization on old anchors. On representative compatible-shift CIFAR10 and Tiny-ImageNet runs, SPMA-OG improves over sparse replay baselines in old-task retention and representation-preservation metrics while remaining competitive on new-task accuracy. On a controlled synthetic atlas-manifold benchmark, it achieves near-perfect anchor-geometry preservation while also improving new-task accuracy over replay. These results provide evidence that geometry-aware anchor regularization is a useful inductive bias when continual learning should preserve a shared latent support rather than create a new one.

Henry J. Kobs

Continual LearningGeometric Representation LearningManifold Learning
AskPDF
Mar 17, 2026

GeMA: Learning Latent Manifold Frontiers for Benchmarking Complex Systems

Benchmarking the performance of complex systems such as rail networks, renewable generation assets and national economies is central to transport planning, regulation and macroeconomic analysis. Classical frontier methods, notably Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA), estimate an efficient frontier in the observed input-output space and define efficiency as distance to this frontier, but rely on restrictive assumptions on the production set and only indirectly address heterogeneity and scale effects. We propose Geometric Manifold Analysis (GeMA), a latent manifold frontier framework implemented via a productivity-manifold variational autoencoder (ProMan-VAE). Instead of specifying a frontier function in the observed space, GeMA represents the production set as the boundary of a low-dimensional manifold embedded in the joint input-output space. A split-head encoder learns latent variables that capture technological structure and operational inefficiency. Efficiency is evaluated with respect to the learned manifold, endogenous peer groups arise as clusters in latent technology space, a quotient construction supports scale-invariant benchmarking, and a local certification radius, derived from the decoder Jacobian and a Lipschitz bound, quantifies the geometric robustness of efficiency scores. We validate GeMA on synthetic data with non-convex frontiers, heterogeneous technologies and scale bias, and on four real-world case studies: global urban rail systems (COMET), British rail operators (ORR), national economies (Penn World Table) and a high-frequency wind-farm dataset. Across these domains GeMA behaves comparably to established methods when classical assumptions hold, and provides additional insight in settings with pronounced heterogeneity, non-convexity or size-related bias.

Jia Ming Li, Anupriya, Daniel J. Graham

Transport Strategy Centre, Imperial College London, London, SW7 2AZ, United Kingdom.

Manifold LearningVariational Autoencoders
AskPDF
Mar 17, 2026

Manifold-Matching Autoencoders

We study a simple unsupervised regularization scheme for autoencoders called Manifold-Matching (MMAE): we align the pairwise distances in the latent space to those of the input data space by minimizing mean squared error. Because alignment occurs on pairwise distances rather than coordinates, it can also be extended to a lower-dimensional representation of the data, adding flexibility to the method. We find that this regularization outperforms similar methods on metrics based on preservation of nearest-neighbor distances and persistent homology-based measures. We also observe that MMAE provides a scalable approximation of Multi-Dimensional Scaling (MDS).

Laurent Cheret, Vincent Létourneau, Isar Nejadgholi +3

Department of Computer Science, University of Ottawa, Ottawa, Canada · MILA, Montréal, Canada · National Research Council of Canada, Ottawa, Canada

Multidimensional ScalingAutoencodersLatent-Space RegularizationDimensionality ReductionLatent Space GeometryLatent Space Alignment+2
AskPDF
Mar 12, 2026

Uncovering Locally Low-dimensional Structure in Networks by Locally Optimal Spectral Embedding

Standard Adjacency Spectral Embedding (ASE) relies on a global low-rank assumption often incompatible with the sparse, transitive structure of real-world networks, causing local geometric features to be 'smeared'. To address this, we introduce Local Adjacency Spectral Embedding (LASE), which uncovers locally low-dimensional structure via weighted spectral decomposition. Under a latent position model with a kernel feature map, we treat the image of latent positions as a locally low-dimensional set in infinite-dimensional feature space. We establish finite-sample bounds quantifying the trade-off between the statistical cost of localisation and the reduced truncation error achieved by targeting a locally low-dimensional region of the embedding. Furthermore, we prove that sufficient localisation induces rapid spectral decay and the emergence of a distinct spectral gap, theoretically justifying low-dimensional local embeddings. Experiments on synthetic and real networks show that LASE improves local reconstruction and visualisation over global and subgraph baselines, and we introduce UMAP-LASE for assembling overlapping local embeddings into high-fidelity global visualisations.

Hannah Sansford, Nick Whiteley, Patrick Rubin-Delanchy

School of Mathematics, University of Bristol, UK · School of Mathematics, University of Edinburgh, UK

Spectral Graph EmbeddingGraph EmbeddingSpectral MethodsManifold Learning
AskPDF
Mar 12, 2026

Context-dependent manifold learning: A neuromodulated constrained autoencoder approach

Many physical systems exhibit a low-dimensional structure that varies with external parameters: link lengths in a robot, forcing constants in a fluid, or Reynolds numbers in a flow shift the underlying manifold while preserving its intrinsic dimension. Constrained AutoEncoders (cAEs) learn such manifolds through an idempotent encoder-decoder projection, a property that unconstrained autoencoders cannot match and that is essential whenever the model is applied iteratively. However, the standard strategies for making a cAE context-dependent, namely concatenating the context to the input or affinely modulating hidden activations, break the encoder-decoder idempotency, sacrificing the projection guarantee precisely in the setting where it would be most valuable. To restore this guarantee under context variation, we developed the Neuromodulated Constrained Autoencoder (NcAE), which modulates the activation slope and bias of a cAE through a context-driven hyper-network. This paper presents the NcAE, its theoretical foundation, and its empirical validation. We prove that for every context, including contexts unseen at training time, the reconstruction map remains an idempotent projection, the topology of the learned manifold is invariant, and context perturbations induce smooth changes in the manifold. We evaluated our approach on a 16-DoF pendulum with context-dependent coupling and the Lorenz96 system across a bifurcation. The NcAE matched or exceeded the best of six baselines on reconstruction, idempotency, and latent-geometry metrics, while being the only architecture that preserves geometric consistency by construction. The NcAE thereby provides a stable, geometry-preserving coordinate system across families of physical regimes.

Jérôme Adriaens, Gustave Bainier, Guillaume Drion +1

Neuroengineering Lab, Department of Electrical Engineering and Computer Science, University of Liège · University of Liège, Liège, Belgium

AutoencodersManifold LearningNeural Representation Geometry
AskPDF
Mar 12, 2026

Harnessing Data Asymmetry: Manifold Learning in the Finsler World

Manifold learning is a fundamental task at the core of data analysis and visualisation. It aims to capture the simple underlying structure of complex high-dimensional data by preserving pairwise dissimilarities in low-dimensional embeddings. Traditional methods rely on symmetric Riemannian geometry, thus forcing symmetric dissimilarities and embedding spaces, e.g. Euclidean. However, this discards in practice valuable asymmetric information inherent to the non-uniformity of data samples. We suggest to harness this asymmetry by switching to Finsler geometry, an asymmetric generalisation of Riemannian geometry, and propose a Finsler manifold learning pipeline that constructs asymmetric dissimilarities and embeds in a Finsler space. This greatly broadens the applicability of existing asymmetric embedders beyond traditionally directed data to any data. We also modernise asymmetric embedders by generalising current reference methods to asymmetry, like Finsler t-SNE and Finsler Umap. On controlled synthetic and large real datasets, we show that our asymmetric pipeline reveals valuable information lost in the traditional pipeline, e.g. density hierarchies, and consistently provides superior quality embeddings than their Euclidean counterparts.

Thomas Dagès, Simon Weber, Daniel Cremers +1

Technion – Israel Institute of Technology · Technical University of Munich · Munich Center for Machine Learning +1

Finsler GeometryManifold LearningDimensionality Reduction
AskPDF
Mar 2, 2026

The Theory behind UMAP?

In 2018, McInnes et al. introduced a dimensionality reduction algorithm called UMAP, which enjoys wide popularity among data scientists. Their work introduces a finite variant of a functor called the metric realization, based on an unpublished draft by Spivak. This draft contains many errors, most of which are reproduced by McInnes et al. and subsequent publications. This article aims to repair these errors and provide a self-contained document with the full derivation of Spivak's functors and McInnes et al.'s finite variant. We contribute an explicit description of the metric realization and related functors. At the end, we discuss the UMAP algorithm, as well as claims about properties of the algorithm and the correspondence of McInnes et al.'s finite variant to the UMAP algorithm.

David Wegmann

University of Erlangen-Nuremberg

Category TheoryDimensionality ReductionManifold Learning
AskPDF
Mar 2, 2026

Manifold Aware Denoising Score Matching (MAD)

A major focus in designing methods for learning distributions defined on manifolds is to alleviate the need to implicitly learn the manifold so that learning can concentrate on the data distribution within the manifold. However, accomplishing this often leads to compute-intensive solutions. In this work, we propose a simple modification to denoising score-matching in the ambient space to implicitly account for the manifold, thereby reducing the burden of learning the manifold while maintaining computational efficiency. Specifically, we propose a simple decomposition of the score function into a known component sbases^{base} and a remainder component s−sbases-s^{base} (the learning target), with the former implicitly including information on where the data manifold resides. We derive known components sbases^{base} in analytical form for several important cases, including distributions over rotation matrices and discrete distributions, and use them to demonstrate the utility of this approach in those cases.

Alona Levy-Jurgenson, Alvaro Prat, James Cuin +1

Department of Statistics University of Oxford · Department of Mathematics, Imperial College London, London, United Kingdom

Score-Based Generative ModelingGenerative ModelingManifold Learning
AskPDF
Mar 1, 2026

A Comparative Study of UMAP and Other Dimensionality Reduction Methods

Uniform Manifold Approximation and Projection (UMAP) is a widely used manifold learning technique for dimensionality reduction. This paper studies UMAP, supervised UMAP, and several competing dimensionality reduction methods, including Principal Component Analysis (PCA), Kernel PCA, Sliced Inverse Regression (SIR), Kernel SIR, and t-distributed Stochastic Neighbor Embedding, through a comprehensive comparative analysis. Although UMAP has attracted substantial attention for preserving local and global structures, its supervised extensions, particularly for regression settings, remain rather underexplored. We provide a systematic evaluation of supervised UMAP for both regression and classification using simulated and real datasets, with performance assessed via predictive accuracy on low-dimensional embeddings. Our results show that supervised UMAP performs well for classification but exhibits limitations in effectively incorporating response information for regression, highlighting an important direction for future development.

Guanzhe Zhang, Shanshan Ding, Zhezhen Jin

Department of Applied Economics and Statistics, University of Delaware · Department of Biostatistics, Columbia University

Dimensionality ReductionManifold LearningClassification
AskPDF
Feb 27, 2026

Histopathology Image Normalization via Latent Manifold Compaction

Batch effects arising from technical variations in histopathology staining protocols, scanners, and acquisition pipelines pose a persistent challenge for computational pathology, hindering cross-batch generalization and limiting reliable deployment of models across clinical sites. In this work, we introduce Latent Manifold Compaction (LMC), an unsupervised representation learning framework that performs image harmonization by learning batch-invariant embeddings from a single source dataset through explicit compaction of stain-induced latent manifolds. This allows LMC to generalize to target domain data unseen during training. Evaluated on three challenging public and in-house benchmarks, LMC substantially reduces batch-induced separations across multiple datasets and consistently outperforms state-of-the-art normalization methods in downstream cross-batch classification and detection tasks, enabling superior generalization.

Xiaolong Zhang, Jianwei Zhang, Selim Sevim +3

Knight Cancer Institute, Oregon Health and Science University, Oregon, USA · Brenden-Colson Center for Pancreatic Care, Oregon Health and Science University, Oregon, USA

Computational PathologyDomain GeneralizationUnsupervised Representation LearningImage EnhancementManifold Learning
AskPDF
Feb 27, 2026

Manifold-Preserving Superpixel Hierarchies and Embeddings for the Exploration of High-Dimensional Images

High-dimensional images, or images with a high-dimensional attribute vector per pixel, are commonly explored with coordinated views of a low-dimensional embedding of the attribute space and a conventional image representation. Nowadays, such images can easily contain several million pixels. For such large datasets, hierarchical embedding techniques are better suited to represent the high-dimensional attribute space than flat dimensionality reduction methods. However, available hierarchical dimensionality reduction methods construct the hierarchy purely based on the attribute information and ignore the spatial layout of pixels in the images. This impedes the exploration of regions of interest in the image space, since there is no congruence between a region of interest in image space and the associated attribute abstractions in the hierarchy. In this paper, we present a superpixel hierarchy for high-dimensional images that takes the high-dimensional attribute manifold into account during construction. Through this, our method enables consistent exploration of high-dimensional images in both image and attribute space. We show the effectiveness of this new image-guided hierarchy in the context of embedding exploration by comparing it with classical hierarchical embedding-based image exploration in two use cases.

Alexander Vieth, Boudewijn Lelieveldt, Elmar Eisemann +2

Leiden University Medical Center, Leiden, The Netherlands · Delft University of Technology, Delft, The Netherlands · Eindhoven University of Technology, Eindhoven, The Netherlands

Hierarchical Representation LearningDimensionality ReductionManifold Learning
AskPDF
Feb 26, 2026

RepSPD: Enhancing SPD Manifold Representation in EEGs via Dynamic Graphs

Decoding brain activity from electroencephalography (EEG) is crucial for neuroscience and clinical applications. Among recent advances in deep learning for EEG, geometric learning stands out as its theoretical underpinnings on symmetric positive definite (SPD) allows revealing structural connectivity analysis in a physics-grounded manner. However, current SPD-based methods focus predominantly on statistical aggregation of EEGs, with frequency-specific synchronization and local topological structures of brain regions neglected. Given this, we propose RepSPD, a novel geometric deep learning (GDL)-based model. RepSPD implements a cross-attention mechanism on the Riemannian manifold to modulate the geometric attributes of SPD with graph-derived functional connectivity features. On top of this, we introduce a global bidirectional alignment strategy to reshape tangent-space embeddings, mitigating geometric distortions caused by curvature and thereby enhancing geometric consistency. Extensive experiments demonstrate that our proposed framework significantly outperforms existing EEG representation methods, exhibiting superior robustness and generalization capabilities.

Haohui Jia, Zheng Chen, Lingwei Zhu +4

Information Science and Techinology, Hokkaido University, Japan · SANKEN, The University of Osaka, Japan · Great Bay University, China +1

EEG DecodingGeometric Representation LearningManifold Learning
AskPDF
Feb 26, 2026

Learning Tangent Bundles and Characteristic Classes with Autoencoder Atlases

We introduce a theoretical framework that connects multi-chart autoencoders in manifold learning with the classical theory of vector bundles and characteristic classes. Rather than viewing autoencoders as producing a single global Euclidean embedding, we treat a collection of locally trained encoder-decoder pairs as a learned atlas on a manifold. We show that any reconstruction-consistent autoencoder atlas canonically defines transition maps satisfying the cocycle condition, and that linearising these transition maps yields a vector bundle coinciding with the tangent bundle when the latent dimension matches the intrinsic dimension of the manifold. This construction provides direct access to differential-topological invariants of the data. In particular, we show that the first Stiefel-Whitney class can be computed from the signs of the Jacobians of learned transition maps, yielding an algorithmic criterion for detecting orientability. We also show that non-trivial characteristic classes provide obstructions to single-chart representations, and that the minimum number of autoencoder charts is determined by the good cover structure of the manifold. Finally, we apply our methodology to low-dimensional orientable and non-orientable manifolds, as well as to a non-orientable high-dimensional image dataset.

Eduardo Paluzo-Hidalgo, Yuichi Ike

Department of Applied Mathematics I, University of Sevilla, Sevilla, Spain · Graduate School of Mathematical Sciences, The University of Tokyo, Tokyo, Japan

Manifold LearningAutoencoders
AskPDF
Feb 20, 2026

Learning Without Training

Machine learning is at the heart of managing the real-world problems associated with massive data. With the success of neural networks on such large-scale problems, more research in machine learning is being conducted now than ever before. This dissertation focuses on three different projects rooted in mathematical theory for machine learning applications. The first project deals with supervised learning and manifold learning. In theory, one of the main problems in supervised learning is that of function approximation: that is, given some data set D={(xj,f(xj))}j=1M\mathcal{D}=\{(x_j,f(x_j))\}_{j=1}^M, can one build a model F≈fF\approx f? We introduce a method which aims to remedy several of the theoretical shortcomings of the current paradigm for supervised learning. The second project deals with transfer learning, which is the study of how an approximation process or model learned on one domain can be leveraged to improve the approximation on another domain. We study such liftings of functions when the data is assumed to be known only on a part of the whole domain. We are interested in determining subsets of the target data space on which the lifting can be defined, and how the local smoothness of the function and its lifting are related. The third project is concerned with the classification task in machine learning, particularly in the active learning paradigm. Classification has often been treated as an approximation problem as well, but we propose an alternative approach leveraging techniques originally introduced for signal separation problems. We introduce theory to unify signal separation with classification and a new algorithm which yields competitive accuracy to other recent active learning algorithms while providing results much faster.

Ryan O'Dowd

Claremont Graduate University

Function ApproximationSupervised LearningTransfer LearningActive LearningManifold Learning
AskPDF
Feb 2, 2026

Discovering Data Manifold Geometry through Geometric Properties

A prevailing paradigm in modern representation learning is the map-first approach, in which a representation map is learned from reconstruction, embedding, or task objectives. At the optimum, when the learned map accurately recovers a global coordinate chart, it should exhibit three structural properties whose geometric meaning can be illustrated through a face-editing example: Commutativity requires that changing pose and then expression gives the same result as applying them in the reverse order; Time Coherence requires that the same variation along one coordinate induces the same expression change across faces; Common-Reference requires that all faces are organized relative to a common reference face. However, small approximation errors in the learned map need not translate into small errors in these structural properties, and can therefore disrupt the global organization of the representation. Based on this observation, we consider the converse of the map-first formulation and ask whether a global representation can instead emerge by directly learning these properties. We represent variations along individual coordinates through vector fields defined in the ambient space and introduce a non-contraction condition preventing one transformation from destroying directions associated with the others. We derive an unsupervised objective that learns these structural properties and establish theoretical results connecting its minimization to tangent-space recovery. Experiments on controlled manifolds validate the predicted tangent-space recovery and global structure, while an autoencoder baseline shows that small map-first errors can still produce substantial violations of the targeted properties.

David Vigouroux, Lucas Drumetz, Ronan Fablet +1

ANITI, IMT Atlantique - DSD, LaTIM · IRT Saint Exupery, ANITI, Toulouse, France · IMT Atlantique, LaTIM UMR 1101 INSERM, Brest, France +5

Unsupervised Representation LearningGeometric Representation LearningManifold Learning
AskPDF
Nov 24, 2025

Geometry-Aware Deep Congruence Networks for Manifold Learning in Cross-Subject Motor Imagery

Cross-subject motor imagery decoding remains a fundamental challenge in EEG-based brain-computer interfaces due to substantial inter-subject variability. Recent approaches have leveraged Riemannian geometry by representing EEG signals as covariance matrices on the symmetric positive definite (SPD) manifold. However, existing methods primarily focus on manifold-based representations while largely overlooking subject-specific variations in covariance dispersion and orientation. In this work, we address these challenges through geometry-aware congruence transformations and propose three complementary models: (i) Discriminative Congruence Transform (DCT), (ii) Deep Linear DCT (DLDCT), and (iii) Deep DCT-UNet (DDCT-UNet). The proposed models are evaluated both as manifold alignment modules for downstream classifiers and as end-to-end discriminative architectures optimized via cross-entropy with a custom logistic regression head. Experiments on challenging cross-subject motor imagery benchmarks demonstrate consistent improvements in transductive decoding performance, achieving 2-3% higher accuracy than strong baselines. These results highlight the effectiveness of geometry-aware congruence learning for mitigating inter-subject variability in EEG decoding.

Sanjeev Manivannan, Chandra Shekar Lakshminarayan

Department of Biotechnology · Department of Data Science and Artificial Intelligence

Cross-Subject EEG DecodingMI ClassificationBrain-Computer InterfacesNeural DecodingEEG DecodingGeometric Representation Learning+1
AskPDF
Oct 2, 2025

Robust Tangent Space Estimation via Laplacian Eigenvector Gradient Orthogonalization

Estimating the tangent spaces of a data manifold is a fundamental problem in geometric data analysis. The standard approach, Local Principal Component Analysis (LPCA), struggles in high-noise settings due to a critical trade-off in choosing the neighborhood size. Selecting an optimal size requires prior knowledge of the geometric and noise characteristics of the data that are often unavailable. In this paper, we propose a spectral method, Laplacian Eigenvector Gradient Orthogonalization (LEGO), that utilizes the global structure of the data to guide local tangent space estimation. Instead of relying solely on local neighborhoods, LEGO estimates the tangent space at each data point by orthogonalizing the gradients of low-frequency eigenvectors of the graph Laplacian. We provide two theoretical justifications of our method. First, a differential geometric analysis on the tubular neighborhood of a manifold shows that gradients of the low-frequency Neumann eigenfunctions of the tube align closely with the manifold's tangent bundle, while an eigenfunction with high gradient in directions orthogonal to the manifold lies deeper in the spectrum. Second, a random matrix theoretic analysis also demonstrates that low-frequency eigenvectors are robust to sub-Gaussian noise. These results allow us to derive the asymptotic scaling and stability of the estimated eigenvector gradients. Numerical experiments demonstrate that LEGO yields tangent space estimates that are significantly more robust to noise than those from LPCA, resulting in marked improvements in downstream tasks such as manifold learning, boundary detection, and local intrinsic dimension estimation.

Dhruv Kohli, Sawyer J. Robertson, Gal Mishne +1

Manifold LearningSpectral Methods
AskPDF
Oct 1, 2025

HomID : Benchmarking Intrinsic Dimension Estimators on Homogenous Manifolds with Anisotropic Embeddings

The manifold hypothesis suggests that data lies on manifolds with smaller intrinsic dimension (ID) than their ambient dimension. However there is no empirical agreement on the estimates for ID from different estimators for realistic datasets. Thus it is important to test ID estimators (IDEs) with targeted stressors. In this work, we consider the role of anisotropy. To this end, we propose HomID, a collection of homogeneous spaces with anisotropic embedding, for benchmarking ID estimators. We observe that methods that perform well on standard benchmarks systematically degrade on HomID under identical resource allocation. We further observe that anisotropic distortion of such benchmarks also results in performance degradation. Finally, we demonstrate that controlled anisotropic distortions induce systematic shifts in the distributions on which these methods rely, providing a concrete mechanism for the resulting estimation errors in two particular IDEs.

Aritra Das, Joseph T. Iosue, Victor V. Albert

Joint Center for Quantum Information and Computer Science (QuICS), NIST & UMD College Park. · Joint Quantum Institute (JQI), NIST & UMD College Park.

Intrinsic DimensionalityManifold Learning
AskPDF
May 30, 2025

Equilibrium Distribution for t-Distributed Stochastic Neighbor Embedding with Generalized Kernels

We extend the results by Auffinger and Fletcher in 2023 on the large-sample limit of t-SNE to a broader class of input and output kernels. We prove that under natural assumptions on the input distribution and the decay of the output kernel, if perplexity scales linearly with data size, the empirical measures and values of minimizers converge to an equilibrium distribution. We conclude with numerical examples comparing different kernels.

Yi Gu, Antonio Auffinger

Department of Mathematics, Northwestern University

Dimensionality ReductionManifold Learning
AskPDF
Apr 16, 2025

Manifold meta-learning for reduced-complexity neural system identification

System identification has greatly benefited from deep learning techniques, particularly for modeling complex, nonlinear dynamical systems with partially unknown physics where traditional approaches may not be feasible. However, deep learning models often require large datasets and significant computational resources for both training and inference due to their high-dimensional parameterizations. To address this challenge, we propose a meta-learning framework that discovers a low-dimensional manifold within the parameter space of an over-parameterized neural network architecture. This manifold is learned from an abundant meta-dataset of input-output sequences generated by a simulator of the class of systems of interest, and thereafter enables efficient model training while preserving the network's expressive power for the considered system class. Unlike bilevel meta-learning approaches, our method employs an auxiliary neural network to map datasets directly onto the learned manifold, eliminating the need for costly second-order gradient computations during meta-training and generally reducing the number of first-order updates required in inference, which could be expensive for large models. Finally, we derive a probabilistic bound that quantifies the performance gap between training on the learned low-dimensional manifold and the full parameterization. We validate our approach on a family of Bouc-Wen oscillators, which is a well-studied nonlinear system identification benchmark. We demonstrate that we are able to learn accurate models even in small-data scenarios.

Marco Forgione, Ankush Chakrabarty, Dario Piga +2

SUPSI, IDSIA - Dalle Molle Institute for Artificial Intelligence, Lugano, Switzerland. · Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA, USA. · IMT School for Advanced Studies Lucca, Lucca, Italy.

System IdentificationMeta-LearningNonlinear System IdentificationData-Efficient LearningManifold Learning
AskPDF
Feb 14, 2023

Linearized Wasserstein dimensionality reduction with approximation guarantees

We introduce LOT Wassmap, a computationally feasible algorithm to uncover low-dimensional structures in the Wasserstein space. The algorithm is motivated by the observation that many datasets are naturally interpreted as probability measures rather than points in Rn\mathbb{R}^n, and that finding low-dimensional descriptions of such datasets requires manifold learning algorithms in the Wasserstein space. Most available algorithms are based on computing the pairwise Wasserstein distance matrix, which can be computationally challenging for large datasets in high dimensions. Our algorithm leverages approximation schemes such as Sinkhorn distances and linearized optimal transport to speed-up computations, and in particular, avoids computing a pairwise distance matrix. We provide guarantees on the embedding quality under such approximations, including when explicit descriptions of the probability measures are not available and one must deal with finite samples instead. Experiments demonstrate that LOT Wassmap attains correct embeddings and that the quality improves with increased sample size. We also show how LOT Wassmap significantly reduces the computational cost when compared to algorithms that depend on pairwise distance computations.

Alexander Cloninger, Keaton Hamm, Varun Khurana +1

DEPARTMENT OF MATHEMATICS, UNIVERSITY OF CALIFORNIA, SAN DIEGO, CA · HALICIOĦGLU DATA SCIENCE INSTITUTE, UNIVERSITY OF CALIFORNIA, SAN DIEGO, CA · DEPARTMENT OF MATHEMATICS, UNIVERSITY OF TEXAS AT ARLINGTON, ARLINGTON, TX +1

Dimensionality ReductionManifold LearningWasserstein DistanceOptimal Transport
AskPDF