Manifold Learning

Ask about this topic

Overview

Manifold learning seeks compact representations of high-dimensional data by modeling it as lying near a lower-dimensional, possibly curved space, enabling visualization, denoising, scientific analysis, and learning with fewer effective degrees of freedom. Common approaches build neighborhood graphs and optimize UMAP- or t-SNE-style embeddings, recover coordinates or operators with spectral and diffusion methods, or use autoencoders with geometric constraints; Riemannian and optimal-transport methods handle structured objects such as covariance matrices and probability distributions. Work increasingly targets properties beyond nearest-neighbor preservation—including density, geodesic distances, topology, tangent spaces, symmetry, and out-of-sample mapping—in settings from images and single-cell data to networks and dynamical systems. A central challenge is balancing local fidelity against global geometry and computational scale, while noise, sparse or anisotropic sampling, and inconsistent evaluation metrics make it difficult to tell when an embedding reflects the underlying manifold.

Latest papers

CardsList

115 papers10 papers in the last four weeks, up 43% on the four weeks before. 0.1% of all new papers.

May 16, 2026

Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine

Diffusion models generate high-dimensional data with remarkable quality, yet how their training efficiently learns the score function, bypassing the curse of dimensionality when data is supported on low-dimensional manifolds, remains theoretically unexplained. We identify a collapse-and-refine mechanism driven by the geometry of the score function itself: at small noise scales, the diverging singularity of the score drives a rapid dimensional collapse of the induced denoising map onto the data manifold projection; at moderate noise scales, training refines the intrinsic density on the learned manifold. We instantiate this principle as Score-induced Latent Diffusion (SiLD), a two-stage framework in which both manifold learning and density estimation emerge from a single denoising score matching objective, replacing the heuristic KL regularization of VAE-based latent diffusion models. We prove that the resulting sample complexity depends on the intrinsic dimension rather than the ambient dimension. Experiments on Stacked MNIST, CelebA variants, and molecular generation benchmarks show that SiLD matches or outperforms VAE-based LDMs in generation quality and consistently improves reconstruction, validating our theoretical predictions.

Wei Huang, Andi Han, Mingyuan Bai +4

RIKEN AIP & The Institute of Statistical Mathematics · University of Sydney · Agency for Science, Technology and Research & The Institute of Statistical Mathematics +4

Latent Diffusion ModelsManifold LearningScore-Based Generative ModelingDiffusion Models
AskPDF
May 16, 2026

Topo-GS: Continuous Volumetric Embedding of High-Dimensional Data via Topological Gaussian Splatting

Dimensionality reduction algorithms map high-dimensional data into visualizable 2D or 3D spaces, but traditionally rely on a discrete point-cloud paradigm. This discrete abstraction is susceptible to visual occlusion and artificial discontinuities, often failing to represent the continuous density of the underlying manifold. To address these limitations, we introduce Topo-GS, a framework that repurposes 3D Gaussian Splatting (3DGS) to cast multidimensional projection as a meshless volumetric reconstruction process. Instead of standard photometric losses, Topo-GS is driven by local geometric constraints. By solving orthogonal Procrustes targets, the optimization enforces an As-Rigid-As-Possible prior while explicitly aligning the spatial covariance of each Gaussian to the local tangent space. Recognizing that unrolling data of varying intrinsic dimensionalities requires distinct spatial treatments, we utilize a topology-aware strategy that tailors the loss formulation to preserve either continuous 1D trajectories or cohesive 2D surfaces. Quantitative and visual evaluations demonstrate that Topo-GS successfully transforms discrete scatter plots into continuous volumetric representations, where inherent projection distortions explicitly manifest as observable geometric variations, while preserving local topological fidelity comparable to discrete baselines.

João Paulo Gois, Luis Gustavo Nonato

Universidade Federal do ABC (UFABC), Santo André, Brazil · Instituto de Ciências Matemáticas e de Computação (ICMC), Universidade de São Paulo (USP), São Carlos, Brazil

3D Gaussian SplattingManifold LearningDimensionality Reduction
AskPDF
May 15, 2026

Neural Point-Forms

Point cloud learning often rests on the premise that observed samples are noisy traces of an underlying geometric object, such as a manifold embedded in a high-dimensional feature space. Yet much of this geometry is not captured directly by coordinates, pairwise distances, or learned graph neighborhoods alone. In the smooth setting, differential forms are devices to encode higher order tangency information. In this work, we introduce a new family of principled learnable geometric features for point clouds called neural point-forms (NPFs). In the absence of a natural tangency structure, we instead use Laplacian-based techniques from Diffusion Geometry to build a discrete model for comparing differential forms on point clouds via inner products. In the continuum, submanifolds of a shared ambient feature space are represented as comparison matrices, whose entries describe how pairs of feature forms interact with extrinsic tangency information. We make this intuition precise by proving the long-run consistency of comparison matrices under standard sampling, bandwidth, density, and manifold-hypothesis assumptions. This yields a compact, efficient and permutation-invariant neural layer whose output is a learned form-comparison matrix. Across synthetic and biologically relevant experiments, we show that NPFs provide a competitive, and interpretable representation, with the strongest benefits appearing when labels depend on sampling density, manifold-like structure, or response-relevant population geometry.

Bruno Trentini, Jacob Hume, Vincenzo Antonio Isoldi +3

University of Oxford Oxford, UK · School of Engineering and Applied Sciences Harvard University Cambridge, MA, US · Max Planck Institute of Molecular Cell Biology and Genetics Dresden, Germany

Point Cloud LearningGeometric Representation LearningManifold Learning
AskPDF
May 13, 2026

Beyond Explained Variance: A Cautionary Tale of PCA

We address shortcomings of principal component analysis (PCA) for visualizing high-dimensional data lying on a nonlinear low-dimensional manifold via two-dimensional scatterplots, focusing on a fossil teeth dataset from the early mammalian insectivore Kuehneotherium. While the PCA scatterplot reported by Jolliffe and Cadima (Philosophical Transactions of the Royal Society A, 2016) shows clustering in the region where PC2 < 0, our analysis based on t-SNE and persistent homology (PH) reveals a ring-like structure with no evident clustering and intrinsic dimensionality equal to one. We further propose a generative probabilistic-geometric model in which the data are sampled uniformly from a unit circle. Under this model, pairwise cosine distances follow an arcsine distribution, in qualitative agreement with the observed U-shaped distribution, thereby independently supporting the analysis based on t-SNE and persistent homology.

Gionni Marchetti

Barcelona, Spain

Data VisualizationPrincipal Component AnalysisManifold LearningDimensionality Reduction
AskPDF
May 12, 2026

FastUMAP: Scalable Dimensionality Reduction via Bipartite Landmark Sampling

Exploratory analysis of high-dimensional data rarely stops at a single embedding. In practice, analysts rerun dimensionality reduction after changing preprocessing, subsets, or hyperparameters, and standard nonlinear methods can quickly become the bottleneck. We introduce FastUMAP (Bipartite Manifold Approximation and Projection), a landmark-based method designed for this repeated-use setting. FastUMAP builds a sparse point-landmark fuzzy graph, computes a Nystrom spectral warm start from the induced landmark affinity, and then refines all sample coordinates with a UMAP-style objective on the bipartite graph. The landmark ratio r = m/n provides a direct way to trade runtime against fidelity. On 9 benchmark datasets spanning 178 to 70,000 samples, FastUMAP has the lowest runtime on 7 datasets in our reported default-implementation comparison on one workstation. On MNIST and Fashion-MNIST (n=70000), it runs in about 4.6 seconds, compared with about 73--75 seconds for Barnes--Hut t-SNE, while reaching 91.4% mean kNN accuracy versus 94.6% for the strongest accuracy baseline. FastUMAP is therefore best viewed as a fast option for repeated exploratory embedding, rather than as a replacement for accuracy-first methods.

Hongmin Li

School of Life Science and Technology, Institute of Science Tokyo · 2-12-1 Ookayama, Meguro-ku, Tokyo 152-8550, Japan 2Department of Computational Biology and Medical Sciences Graduate School of Frontier Sciences, The University of Tokyo

Unsupervised LearningNyström MethodManifold LearningDimensionality Reduction
AskPDF
May 11, 2026

Learning Point Cloud Geometry as a Statistical Manifold: Theory and Practice

Point clouds are a fundamental representation for robotic perception tasks such as localization, mapping, and object pose estimation. However, LiDAR-acquired point clouds are inherently sparse and non-uniform, providing incomplete observations of the underlying scene geometry. This makes reliable geometric reasoning challenging and degrades downstream perception performance. Existing approaches attempt to compensate for these limitations by estimating local geometry, but often rely on hand-crafted statistics or end-to-end supervised learning, which can suffer from limited scalability or require large amounts of accurately labeled data. To address these challenges, we explicitly model point cloud geometry under a principled mathematical formulation. We represent local geometry as a statistical manifold induced by a family of Gaussian distributions, where each point is associated with a Gaussian capturing its local geometric structure. Based on this formulation, we introduce Point-to-Ellipsoid (POLI), a deep neural estimator that predicts per-point Gaussian geometry. POLI learns a mapping from point cloud observations to their underlying geometry in a self-supervised manner, removing the need for labeled data while preserving strong geometric inductive biases. The resulting representation integrates seamlessly into existing robotic perception pipelines without architectural modifications. Extensive experiments show that POLI enables accurate and robust geometry estimation and consistently improves performance across diverse robotic perception tasks.

Jinwoo Lee, Jiwoo Kim, Woojae Shin +2

Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu, Republic of Korea

Geometric Representation LearningManifold LearningPoint Cloud UnderstandingRobotic Perception
AskPDF
May 9, 2026

SMIXAE: Towards Unsupervised Manifold Discovery in Language Models

Sparse autoencoders (SAEs) have been used widely to decompose and interpret neural network activations, especially those of transformer language models. One key issue with SAEs is their inability to directly model multidimensional features. Instead, SAEs may tile such features by a set of independent directions that must be grouped together after the SAE training phase, impeding discoverability and interpretation of learned feature representations. We begin to address this issue by introducing the Sparse MIXture of Autoencoders (SMIXAE) architecture. Empirically, we provide evidence that SMIXAE models have success both in directly learning previously identified manifold structures, as well as finding novel structures, within the open source Gemma 2 2B and 9B models. Finally, we discuss several limitations and point towards areas for future work.

Collin Francel

Department of Arts and Sciences, University of Alabama, Tuscaloosa, AL, United States.

AutoencodersLLM InterpretabilitySparse AutoencodersManifold Learning
AskPDF
May 9, 2026

Bilinear autoencoders find interpretable manifolds

Sparse autoencoders have become a standard tool for uncovering interpretable latent representations in neural networks. Yet salient concepts often span manifolds that current linear methods cannot capture without post hoc analysis. This paper uses quadratic latents to close this gap: we implement these with bilinear autoencoders, which decompose activations into low-rank quadratic forms, compose linearly in weight space, and admit input-independent geometric analysis. This qualitative difference in what concepts quadratic latents can detect challenges the standard linear representation hypothesis. Our experiments and visualisations show that multi-dimensional geometries are highly prevalent and that composite latents capture them well, systematically improving reconstruction error in language models. Furthermore, we show that autoencoders with varying geometric priors recover the same input subspace despite their dictionary entries being distinct. Practically, these models serve as an unsupervised tool for manifold discovery, which we demonstrate through an interactive online visualizer for Qwen 3.5. This is a step toward nonlinear but mathematically tractable latent representations whose composition is expressive and interpretable by design.

Thomas Dooms, Ward Gauderis, Geraint Wiggins +1

University of Antwerp · Vrije Universiteit Brussel · University of Antwerp, sqIRL

Neural Representation GeometryAutoencodersLLM InterpretabilityManifold Learning
AskPDF
May 7, 2026

Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves

Modern deep learning architectures increasingly contend with sophisticated signals that are natively infinite-dimensional, such as time series, probability distributions, or operators, and are defined over irregular domains. Yet, a unified learning theory for these settings has been lacking. To start addressing this gap, we introduce a novel convolutional learning framework for possibly infinite-dimensional signals supported on a manifold. Namely, we use the connection Laplacian associated with a Hilbert bundle as a convolutional operator, and we derive filters and neural networks, dubbed as HilbNets. We make HilbNets and, more generally, the convolution operation, implementable via a two-stage sampling procedure. First, we show that sampling the manifold induces a Hilbert Cellular Sheaf, a generalized graph structure with Hilbert feature spaces and edge-wise coupling rules, and we prove that its sheaf Laplacian converges in probability to the underlying connection Laplacian as the sampling density increases. Notably, this result is a generalization to the infinite-dimensional bundle setting of the Belkin & Niyogi \cite{BELKIN20081289} convergence result for the graph Laplacian to the manifold Laplacian, a theoretical cornerstone of geometric learning methods. Second, we discretize the signals and prove that the discretized (implementable) HilbNets converge to the underlying continuous architectures and are transferable across different samplings of the same bundle, providing consistency for learning. Finally, we validate our framework on synthetic and real-world tasks. Overall, our results broaden the scope of geometric learning as a whole by lifting classical Laplacian-based frameworks to settings where the signal at each point lives in its own Hilbert space.

Kartik Tandon, Julian Gould, Tanishq Bhatia +3

University of Pennsylvania · Sakana AI · Northeastern University +1

Graph Neural NetworksGeometric Representation LearningManifold LearningSheaf Neural NetworksConvolutional Neural Networks
AskPDF
May 7, 2026

Beyond Rigid Alignment: Graph Federated Learning via Dual Manifold Calibration

Graph Federated Learning (GFL) enables collaborative representation learning across distributed subgraphs while preserving privacy. However, heterogeneity remains a critical challenge, as subgraphs across clients typically differ significantly in both semantics and structures. Existing methods address heterogeneity by enforcing the rigid alignment of model parameters or prototypes between clients and the server. However, these alignments implicitly rely on a restrictive global linearity assumption that summarizes local data distributions using a single and globally consistent representation space. This severely compresses the personalized representation space of clients and fails to preserve diverse local graph distributions. To overcome these limitations, we propose Federated Graph Manifold Calibration (FedGMC), a novel paradigm that tackles semantic heterogeneity and structural heterogeneity from a unified manifold perspective. Instead of enforcing rigid alignment, FedGMC introduces a dual manifold calibration mechanism that preserves global commonalities while maximizing the personalized representation space of local clients. Specifically, for semantic heterogeneity, the server constructs a geometrically optimal semantic manifold via equidistant semantic anchors, so as to guide the calibration of local semantic manifolds. For structural heterogeneity, the server constructs a global structural manifold by building global structural templates, so as to guide the calibration of local structural manifolds. Finally, the server dynamically refines both global semantic manifolds and structural manifolds by aggregating local manifolds. Extensive experiments on eleven homophilic and heterophilic graphs demonstrate that FedGMC effectively balances global commonality and local personalization, thereby significantly outperforming state-of-the-art baseline methods.

Wentao Yu, Bo Han, Jie Yang +1

School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing, China · Department of Computer Science Hong Kong Baptist University Hong Kong, China · School of Automation and Intelligent Sensing Shanghai Jiao Tong University Shanghai, China

Federated Graph LearningManifold LearningOnline Federated LearningFederated LearningHeterogeneous Federated Learning
AskPDF
May 6, 2026

Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior

Neural representations carry rich geometric structure; but does that structure causally shape behavior? To address this question, we intervene along paths through activation space defined by different geometries, and measure the behavioral trajectories they induce. In particular, we test whether interventions that respect the geometry of activation space will yield behaviors close to those the model exhibits naturally. Concretely, we first fit an activation manifold MhM_h to representations and a behavior manifold MyM_y to output probability distributions. We then test the link Mh↔MyM_h \leftrightarrow M_y via interventions: we find that steering along MhM_h, which we term manifold steering, yields behavioral trajectories that follow MyM_y, while linear steering -- which assumes a Euclidean geometry -- cuts through off-manifold regions and hence produces unnatural outputs. Moreover, optimizing interventions in activation space to produce paths along MyM_y recovers activation trajectories that trace the curvature of MhM_h. We demonstrate this bidirectional relationship between the geometry of representation and behavior across tasks and modalities. In language models, we use reasoning tasks with cyclic and sequential geometries as well as in-context learning tasks with more complex graph geometries. In a video world model, we use a task with geometry corresponding to physical dynamics. Overall, our work shows that geometry in neural representation is not merely incidental, but is in fact the proper object for enabling principled control via intervention on internals. This recasts the core problem of steering from finding the right direction to finding the right geometry.

Daniel Wurgaft, Can Rager, Matthew Kowal +13

Stanford University · University College London · Northeastern University +2

Neural Representation GeometryManifold LearningCausal Interventions in Language ModelsActivation Steering
AskPDF
May 3, 2026

Complex Diffusion Maps with ωω-Parameterized Kernels Revealing Inherent Harmonic Representations

In this paper, we propose Complex Diffusion Maps (CDM), a novel diffusion mapping framework that aims to reveal the dominant complex harmonics of high-dimensional data. Inspired by the local Gaussian kernel relevant to the heat equation and the nonlocal Schrödinger kernel relevant to the Schrödinger equation, we propose a unified family of ωω-parameterized complex-valued kernels for the trade-off between local and nonlocal connections. We establish the theoretical foundation based on the operator spectrum theory, where the corresponding diffusion operator, diffusion distance, and complex harmonic maps are well-defined. An optimization-based interpretation of the maps is also developed, aiming to preserve angular structure in the complex diffusion space rather than relying solely on real-valued magnitude. We extensively evaluate CDM on both synthetic and real-world datasets. The complex-valued kernel amplifies differences among easily confusable samples, improving discriminative power over both linear and nonlinear methods based on real-valued kernels. CDM remains robust in high-noise settings, yielding a clearer eigengap that enhances spectral separation. For resting-state fMRI data, CDM captures more strongly correlated and nonlocal spatiotemporal dynamics. Without task-specific tuning, CDM achieves competitive performance on a public EEG sleep dataset, while maintaining high computational efficiency compared with both traditional machine learning and deep neural network approaches, highlighting its generality and practical value.

Tongzhen Dang, Weiyang Ding, Michael K. Ng

Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China. · Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China; Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence (Fudan University), Ministry of Education, China. · Department of Mathematics, Hong Kong Baptist University, Hong Kong, China.

Manifold LearningKernel MethodsSpectral Methods
AskPDF
May 1, 2026

Controlled Paraphrase Geometry in Sentence Embedding Space: Local Manifold Modeling and Latent Probing

The paper studies the local geometry of embedding clouds induced by controlled local classes of semantically close sentences. The central question is how controlled paraphrase-like semantic variation is organized in sentence embedding space and whether this local structure can be explicitly modeled by low-degree fitted carriers. We introduce a local geometric modeling scheme based on affine, quadratic, and cubic fitted models. We also use a surface-based latent probing procedure that constructs synthetic latent points in a reduced local PCA space with respect to the fitted carrier. The procedure is intended as an offline method for representation-space analysis, local manifold modeling, and geometry-aware latent probing. Generated latent points are evaluated using criteria that measure consistency with the fitted surface, preservation of neighborhood structure, agreement with the empirical distribution, stability of Hessian-based second-order shape descriptors, and stability of fitted-model coefficients. Experiments on controlled sets of semantically close sentences show that nonlinear local models describe embedding clouds more accurately than affine models. Surface-based generation provides strong fitted-geometry fidelity, including surface consistency, Hessian-based shape consistency, and coefficient consistency. Downstream experiments show that geometric validity of synthetic latent points does not automatically translate into improved classification performance. The results support explicit local geometric modeling of sentence embedding space and highlight the need to distinguish geometric validity from discriminative utility. As a resource contribution, we introduce CoPaGE-300K, a controlled template-based dataset of semantically close sentence variants with slot-level annotations and precomputed sentence embeddings.

Leonid Bedratyuk

Representation LearningNeural Representation GeometrySentence EmbeddingsManifold Learning
AskPDF
Apr 30, 2026

Do Sparse Autoencoders Capture Concept Manifolds?

Sparse autoencoders (SAEs) are widely used to extract interpretable features from neural network representations, often under the implicit assumption that concepts correspond to independent linear directions. However, a growing body of evidence suggests that many concepts are instead organized along low-dimensional manifolds encoding continuous geometric relationships. This raises three basic questions: what does it mean for an SAE to capture a manifold, when do existing SAE architectures do so, and how? We develop a theoretical framework that answers these questions and show that SAEs can capture manifolds in two fundamentally different ways: globally, by allocating a compact group of atoms whose linear span contains the entire manifold, or locally, by distributing it across features that each selectively tile a restricted region of the underlying geometry. Empirically, we find that SAEs suboptimally recover continuous structures, mixing the global subspace and local tiling solutions in a fragmented regime we call dilution. This explains why manifold structure is rarely visible at the level of individual concepts and motivates post-hoc unsupervised discovery methods that search for coherent groups of atoms rather than isolated directions. More broadly, our results suggest that future representation learning methods should treat geometric objects, not just individual directions, as the basic units of interpretability.

Usha Bhalla, Thomas Fel, Can Rager +9

aHarvard University · bNortheastern University · cTechnion IIT +1

Neural Representation GeometrySparse AutoencodersNeural Network InterpretabilityManifold LearningSparse Coding
AskPDF
Apr 29, 2026

High-Dimensional Noise to Low-Dimensional Manifolds: A Manifold-Space Diffusion Framework for Degraded Hyperspectral Image Classification

Recently, Hyperspectral Image (HSI) classification has attracted increasing attention in remote sensing. However, HSI data are inherently high-dimensional but low-rank, with discriminative information concentrated on a low-dimensional latent manifold. In real-world remote sensing scenarios, the superposition of multiple degradation factors disrupts this intrinsic manifold structure, driving samples away from their original low-dimensional distribution and introducing substantial redundant and non-discriminative variations. To better handle this challenge, this paper proposes a manifold-space diffusion framework (MSDiff) for robust hyperspectral classification under complex degradation conditions. Specifically, the proposed method first maps high-dimensional, degradation-affected HSI data into a compact low-dimensional manifold through a discriminative spectral-spatial reconstruction task, preserving class semantics and reducing redundant variations. A diffusion-based generative model is then applied to regularize the spectral-spatial distribution within the manifold, enabling progressive refinement and stabilization of latent features against residual degradations. The key advantage of the proposed framework lies in performing diffusion-based distribution modeling directly on the low-dimensional manifold, effectively decoupling degradation-induced disturbances from intrinsic discriminative structures and enhancing representation stability under complex degradations. Experimental results on multiple hyperspectral benchmarks demonstrate consistent performance improvements over state-of-the-art methods under diverse composite degradation settings. The code will be available at https://github.com/yangboxiang1207/MSDiff

Boxiang Yang, Ning Chen, Xia Yue +6

Institute of Remote Sensing and Geographic Information System, Peking University, Beijing 100871, China · Beijing Key Laboratory of Spatio-temporal Perception and Urban Resilience, Peking University, Beijing 100871, China · School of Computer Science and Engineering, Central South University, Changsha 410083, China +4

Manifold LearningHyperspectral Image ClassificationDiffusion Models
AskPDF
Apr 28, 2026

DiRe-RAPIDS: Topology-faithful dimensionality reduction at scale

Dimensionality reduction methods such as UMAP and t-SNE are central tools for visualising high-dimensional data, but their local-neighborhood objectives can preserve sampling noise while distorting global topology. We show that standard local metrics reward this noise memorisation: top-performing embeddings invent cycles and disconnected islands absent from the data. We introduce a topology-faithfulness benchmark based on noisy manifolds with known homology, tune DiRe against it, and find Pareto-optimal configurations that match or beat GPU-accelerated UMAP on classification while recovering exact first Betti numbers on stress tests. On 723K arXiv paper embeddings, DiRe preserves 3-4 times more topological structure than UMAP at comparable wall-clock.

Alexander Kolpakov, Igor Rivin

CSTEM, University of Austin, Austin, TX, USA · Department of Mathematics, Temple University, Philadelphia, PA, USA

Data VisualizationManifold LearningDimensionality ReductionTopological Representation LearningTopological MappingTopological Data Analysis
AskPDF
Apr 22, 2026

Sheaf Neural Networks on SPD Manifolds: Second-Order Geometric Representation Learning

Graph neural networks face two fundamental challenges rooted in the linear structure of Euclidean vector spaces: (1) Current architectures represent geometry through vectors (directions, gradients), yet many tasks require matrix-valued representations that capture relationships between directions-such as how atomic orientations covary in a molecule. These second-order representations are naturally captured by points on the symmetric positive definite matrices (SPD) manifold; (2) Standard message passing applies shared transformations across edges. Sheaf neural networks address this via edge-specific transformations, but existing formulations remain confined to vector spaces and therefore cannot propagate matrix-valued features. We address both challenges by developing the first sheaf neural network operates natively on the SPD manifold. Our key insight is that the SPD manifold admits a Lie group structure, enabling well-posed analogs of sheaf operators without projecting to Euclidean space. Theoretically, we prove that SPD-valued sheaves are strictly more expressive than Euclidean sheaves: they admit consistent configurations (global sections) that vector-valued sheaves cannot represent, directly translating to richer learned representations. Empirically, our sheaf convolution transforms effectively rank-1 directional inputs into full-rank matrices encoding local geometric structure. Our dual-stream architecture achieves SOTA on 6/7 MoleculeNet benchmarks, with the sheaf framework providing consistent depth robustness.

Yuhan Peng, Junwen Dong, Yuzhi Zeng +6

School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore · Chern Institute of Mathematics, Nankai University, Tianjin, China · Department of Mathematics, Faculty of Science, National University of Singapore, Singapore +3

Graph Neural NetworksGeometric Representation LearningManifold LearningSheaf Neural NetworksSheaf TheoryMolecular Representation Learning
AskPDF
Apr 21, 2026

Towards Modality-Agnostic Medical Image Anomaly Detection: A Training-Free Manifold Refinement Approach

Deploying AI-based anomaly detection across diverse clinical imaging settings remains challenging because most existing methods rely on modality-specific architectures, anatomical priors, or extensive retraining, limiting their use as general-purpose screening tools. One-class classification (OCC) offers a label-efficient alternative by training exclusively on normal data, but conventional two-stage pipelines fit a density estimator directly on raw pretrained embeddings, leaving substantial discriminative structure in the latent space unexploited. We introduce a training-free, modality-agnostic framework that inserts an explicit manifold-refinement stage between feature extraction and anomaly scoring. Empirical density weights, estimated via a UMAP-derived neighborhood graph, guide an iterative shift of embeddings toward locally dense regions, compacting normal samples, leaving anomalies relatively isolated prior to Gaussian density estimation and Mahalanobis-based scoring. This refinement introduces no additional trainable parameters and no architectural modification, allowing it to be layered onto any pretrained encoder. Evaluated on the MedIAnomaly benchmark across seven datasets spanning five imaging modalities (X-ray, MRI, fundus, dermatoscopy, histopathology), the framework achieves the best AUC on four datasets and the best Average Precision on five datasets among methods evaluated in the benchmark, outperforming specialized reconstruction and diffusion-based methods with a single fixed hyperparameter configuration across all modalities. These results demonstrate that meaningful gains can be achieved through post-hoc geometric refinement of existing representations rather than bespoke encoders, offering a practical and scalable AI screening framework for real-world, multi-modality clinical workflows where retraining and abnormal-case annotation are costly or infeasible.

Pritam Kar, Gouri Lakshmi S, Saptarshi Bej

aSchool of Data Science, Indian Institute of Science Education and Research, Thiruvananthapuram, 695551, Kerala, India

Medical Image Anomaly DetectionManifold LearningUnsupervised Anomaly DetectionOne-Class Classification
AskPDF
Apr 21, 2026

S2MAM: Semi-supervised Meta Additive Model for Robust Estimation and Variable Selection

Semi-supervised learning with manifold regularization is a classical framework for jointly learning from both labeled and unlabeled data, where the key requirement is that the support of the unknown marginal distribution has the geometric structure of a Riemannian manifold. Typically, the Laplace-Beltrami operator-based manifold regularization can be approximated empirically by the Laplacian regularization associated with the entire training data and its corresponding graph Laplacian matrix. However, the graph Laplacian matrix depends heavily on the prespecified similarity metric and may lead to inappropriate penalties when dealing with redundant or noisy input variables. To address the above issues, this paper proposes a new Semi-Supervised Meta Additive Model (S2^2MAM) based on a bilevel optimization scheme that automatically identifies informative variables, updates the similarity matrix, and simultaneously achieves interpretable predictions. Theoretical guarantees are provided for S2^2MAM, including the computing convergence and the statistical generalization bound. Experimental assessments across 4 synthetic and 12 real-world datasets, with varying levels and categories of corruption, validate the robustness and interpretability of the proposed approach.

Xuelin Zhang, Hong Chen, Yingjie Wang +2

Huazhong Agricultural University · China University of Petroleum (East China) · Xi’an Jiaotong University +1

Manifold LearningFeature SelectionSemi-Supervised LearningRobust Estimation
AskPDF
Apr 20, 2026

Curvature-Aware PCA with Geodesic Tangent Space Aggregation for Semi-Supervised Learning

Principal Component Analysis (PCA) is a fundamental tool for representation learning, but its global linear formulation fails to capture the structure of data supported on curved manifolds. In contrast, manifold learning methods model nonlinearity but often sacrifice the spectral structure and stability of PCA. We propose Geodesic Tangent Space Aggregation PCA (GTSA-PCA), a geometric extension of PCA that integrates curvature awareness and geodesic consistency within a unified spectral framework. Our approach replaces the global covariance operator with curvature-weighted local covariance operators defined over a kk-nearest neighbor graph, yielding local tangent subspaces that adapt to the manifold while suppressing high-curvature distortions. We then introduce a geodesic alignment operator that combines intrinsic graph distances with subspace affinities to globally synchronize these local representations. The resulting operator admits a spectral decomposition whose leading components define a geometry-aware embedding. We further incorporate semi-supervised information to guide the alignment, improving discriminative structure with minimal supervision. Experiments on real datasets show consistent improvements over PCA, Kernel PCA, Supervised PCA and strong graph-based baselines such as UMAP, particularly in small sample size and high-curvature regimes. Our results position GTSA-PCA as a principled bridge between statistical and geometric approaches to dimensionality reduction.

Alexandre L. M. Levada

Computing Department Federal University of São Carlos 13565-905, São Carlos-SP, Brazil

Principal Component AnalysisManifold LearningDimensionality Reduction
AskPDF
Apr 20, 2026

Characterizing AlphaEarth Embedding Geometry for Agentic Environmental Reasoning

Earth observation foundation models encode land surface information into dense embedding vectors, yet the geometric structure of these representations and its implications for downstream reasoning remain underexplored. We characterize the manifold geometry of Google AlphaEarth's 64-dimensional embeddings across 12.1 million Continental United States samples (2017--2023) and develop an agentic system that leverages this geometric understanding for environmental reasoning. The manifold is non-Euclidean: effective dimensionality is 13.3 (participation ratio) from 64 raw dimensions, with local intrinsic dimensionality of approximately 10. Tangent spaces rotate substantially, with 84% of locations exceeding 60\textdegree{} and local-global alignment (mean∣cos⁡θ∣=0.17|\cosθ| = 0.17) approaching the random baseline of 0.125. Supervised linear probes indicate that concept directions rotate across the manifold, and compositional vector arithmetic using both PCA-derived and probe-derived directions yields poor precision. Retrieval instead produces physically coherent results, with local geometry predicting retrieval coherence (R2=0.32R^2 = 0.32). Building on this characterization, we introduce an agentic system with nine specialized tools that decomposes environmental queries into reasoning chains over a FAISS-indexed embedding database. A five-condition ablation (120 queries, three complexity tiers) shows that embedding retrieval dominates response quality (μ=3.79±0.90μ= 3.79 \pm 0.90 vs.\ 3.03±0.773.03 \pm 0.77 parametric-only; scale 1--5), with peak performance on multi-step comparisons (μ=4.28±0.43μ= 4.28 \pm 0.43). A cross-model benchmark show that geometric tools reduce Sonnet 4.5's score by 0.12 points but improve Opus 4.6's by 0.07, with Opus achieving higher geometric grounding (3.38 vs.\ 2.64), suggesting that the value of geometric characterization scales with the reasoning capability of the consuming model.

Mashrekur Rahman, Samuel J. Barrett, Christina Last

Dartmouth Libraries, Dartmouth College, Hanover, 03755, NH, USA. · LGND AI, Canarias, Spain. · TipplyAI, London, UK.

Geospatial Foundation ModelsGeospatial ReasoningManifold LearningLLM Agents
AskPDF
Apr 18, 2026

Neighbor Embedding for High-Dimensional Sparse Poisson Data

Across many scientific fields, measurements often represent the number of times an event occurs. For example, a document can be represented by word occurrence counts, neural activity by spike counts per time window, or online communication by daily email counts. These measurements yield high-dimensional count data that often approximate a Poisson distribution, frequently with low rates that produce substantial sparsity and complicate downstream analysis. A useful approach is to embed the data into a low-dimensional space that preserves meaningful structure, commonly termed dimensionality reduction. Yet existing dimensionality reduction methods, including both linear (e.g., PCA) and nonlinear approaches (e.g., t-SNE), often assume continuous Euclidean geometry, thereby misaligning with the discrete, sparse nature of low-rate count data. Here, we propose p-SNE (Poisson Stochastic Neighbor Embedding), a nonlinear neighbor embedding method designed around the Poisson structure of count data, using KL divergence between Poisson distributions to measure pairwise dissimilarity and Hellinger distance to optimize the embedding. We test p-SNE on synthetic Poisson data and demonstrate its ability to recover meaningful structure in real-world count datasets, including weekday patterns in email communication, research area clusters in OpenReview papers, and temporal drift and stimulus gradients in neural spike recordings.

Noga Mudrik, Adam S. Charles

Department of Biomedical Engineering Center for Imaging Science Kavli Neuroscience Discovery Institute Johns Hopkins University Baltimore, MD 21218, USA

Manifold LearningDimensionality Reduction
AskPDF
Apr 17, 2026

Geometric regularization of autoencoders via observed stochastic dynamics

Stochastic dynamical systems with slow or metastable behavior evolve, on long time scales, on an unknown low-dimensional manifold in high-dimensional ambient space. Building a reduced simulator from short-burst ambient ensembles is a long-standing problem: local-chart methods like ATLAS suffer from exponential landmark scaling and per-step reprojection, while autoencoder alternatives leave tangent-bundle geometry poorly constrained, and the errors propagate into the learned drift and diffusion. We observe that the ambient covariance~ΛΛ already encodes coordinate-invariant tangent-space information, its range spanning the tangent bundle. Using this, we construct a tangent-bundle penalty and an inverse-consistency penalty for a three-stage pipeline (chart learning, latent drift, latent diffusion) that learns a single nonlinear chart and the latent SDE. The penalties induce a function-space metric, the ρρ-metric, strictly weaker than the Sobolev H1H^1 norm yet achieving the same chart-quality generalization rate up to logarithmic factors. For the drift, we derive an encoder-pullback target via Itô's formula on the learned encoder and prove a bias decomposition showing the standard decoder-side formula carries systematic error for any imperfect chart. Under a W2,∞W^{2,\infty} chart-convergence assumption, chart-level error propagates controllably to weak convergence of the ambient dynamics and to convergence of radial mean first-passage times. Experiments on four surfaces embedded in up to 201201 ambient dimensions reduce radial MFPT error by 5050--70%70\% under rotation dynamics and achieve the lowest inter-well MFPT error on most surface--transition pairs under metastable Müller--Brown Langevin dynamics, while reducing end-to-end ambient coefficient errors by up to an order of magnitude relative to an unregularized autoencoder.

Sean Hill, Felix X. -F. Ye

Department of Mathematics and Statistics, University at Albany, SUNY

Latent Dynamics ModelingAutoencodersManifold Learning
AskPDF
Apr 15, 2026

Complex Interpolation of Matrices with an application to Multi-Manifold Learning

Given two symmetric positive-definite matrices A,B∈Rn×nA, B \in \mathbb{R}^{n \times n}, we study the spectral properties of the interpolation A1−xBxA^{1-x} B^x for 0≤x≤10 \leq x \leq 1. The presence of `common structures' in AA and BB, eigenvectors pointing in a similar direction, can be investigated using this interpolation perspective. Generically, exact log-linearity of the operator norm ∥A1−xBx∥\|A^{1-x} B^x\| is equivalent to the existence of a shared eigenvector in the original matrices; stability bounds show that approximate log-linearity forces principal singular vectors to align with leading eigenvectors of both matrices. These results give rise to and provide theoretical justification for a multi-manifold learning framework that identifies common and distinct latent structures in multiview data.

Adi Arbel, Stefan Steinerberger, Ronen Talmon

Viterbi Faculty of Electrical and Computer Engineering, Technion – Israel Institute of Technology, Haifa, Israel · Department of Mathematics and Department of Applied Mathematics, University of Washington, Seattle, WA 98195, USA

Multi-View LearningManifold Learning
AskPDF
Apr 13, 2026

EigenCoin: sassanid coins classification based on Bhattacharyya distance

Solving pattern recognition problems using imbalanced databases is a hot topic, which entices researchers to bring it into focus. Therefore, we consider this problem in the application of Sassanid coins classification. Our focus is not only on proposing EigenCoin manifold with Bhattacharyya distance for the classification task, but also on testing the influence of the holistic and feature-based approaches. EigenCoin consists of three main steps namely manifold construction, mapping test data, and classification. Conducted experiments show EigenCoin outperformed other observed algorithms and achieved the accuracy from 9.45% up to 21.75%, while it has the capability of handling the over-fitting problem.

Rahele Allahverdi, Mohammad Mahdi Dehshibi, Azam Bastanfard +1

Department of Electrical, Computer and IT, Islamic Azad University, Qazvin Branch, Iran · Department of Computer Engineering, Islamic Azad University, Karaj Branch, Iran · National Museum of Iran, Tehran, Iran

Image ClassificationClass-Imbalanced LearningManifold Learning
AskPDF
Apr 11, 2026

MPNet: A Robust and Efficient Manifold Pooling Network for Multi-Rhythm EEG Signal Decoding

Deep Riemannian networks provide a powerful framework for Electroencephalography (EEG) decoding, but their practical applications are severely constrained. Accurately decoding EEG signals requires modeling complex temporal dynamics across multiple rhythms, which results in high-dimensional Riemannian inputs and significant computational costs. To address this, we propose the Manifold Pooling Network (MPNet). MPNet uses a rhythm-adaptive convolutional frontend to extract comprehensive time-frequency representations and generate multi-view Riemannian nodes. A novel manifold node pooling layer is then proposed to aggregate these nodes into a single fusion node with a fixed size, enabling the following deep Riemannian network to process it with greatly reduced costs. Experiments on two public EEG datasets show that MPNet achieves state-of-the-art accuracy, runs up to 10 times faster than the comparable Riemannian model, and maintains robust performance under limited-data conditions. These findings highlight MPNet's practicality and efficiency for real-world EEG applications.

Guoqing Cai, Kai Zeng, Shoulin Huang +1

Harbin Institute of Technology (Shenzhen), Shenzhen, China · Guangxi Normal University, Guilin, China · Peng Cheng Laboratory, Shenzhen, China

Riemannian Neural NetworksEEG DecodingEfficient Neural Network InferenceManifold Learning
AskPDF
Apr 8, 2026

Diffusion Processes on Implicit Manifolds

High-dimensional data are often assumed to lie on lower-dimensional manifolds. We study how to construct diffusion processes on this data manifold using only point cloud samples and without access to charts, projections, or other geometric primitives. Here, we introduce Implicit Manifold-valued Diffusions (IMDs), a data-driven mathematical formalism for defining stochastic differential equations in the original high-dimensional space that describe drifting Brownian particles evolving intrinsically on the underlying manifold. Our construction hinges on approximating the corresponding infinitesimal generator of the diffusion process using a proximity graph over the data and using the carré-du-champ of the generator, which encodes the local tangent spaces of the manifold and lifts the intrinsic process into ambient coordinates. We show that as the number of samples grows, our discrete diffusion process converges in law on the space of probability paths to its smooth manifold counterpart. We further present an Euler-Maruyama scheme for the numerical integration of IMDs. We validate our framework using numerical experiments on synthetic manifolds and the MNIST data manifold, showing that IMDs remain confined over the manifold and enable its guided exploration. Our work provides the mathematical foundation and practical implementations of diffusion processes on data manifolds, opening new avenues for manifold-aware sampling, exploration, and generative modeling.

Victor Kawasaki-Borruat, Clara Grotehans, Pierre Vandergheynst +1

Signal Processing Laboratory 2 EPFL · Institute of Artificial Intelligence Medical University of Vienna

Stochastic Differential EquationsManifold Learning
AskPDF
Apr 8, 2026

Energy-based Tissue Manifolds for Longitudinal Multiparametric MRI Analysis

We propose a geometric framework for longitudinal multi-parametric MRI analysis based on patient-specific energy modelling in sequence space. Rather than operating on images with spatial networks, each voxel is represented by its multi-sequence intensity vector (T1T1, T1cT1c, T2T2, FLAIR, ADC), and a compact implicit neural representation is trained via denoising score matching to learn an energy function Eθ(u)E_θ(\mathbf{u}) over Rd\mathbb{R}^d from a single baseline scan. The learned energy landscape provides a differential-geometric description of tissue regimes without segmentation labels. Local minima define tissue basins, gradient magnitude reflects proximity to regime boundaries, and Laplacian curvature characterises local constraint structure. Importantly, this baseline energy manifold is treated as a fixed geometric reference: it encodes the set of contrast combinations observed at diagnosis and is not retrained at follow-up. Longitudinal assessment is therefore formulated as evaluation of subsequent scans relative to this baseline geometry. Rather than comparing anatomical segmentations, we analyse how the distribution of MRI sequence vectors evolves under the baseline energy function. In a paediatric case with later recurrence, follow-up scans show progressive deviation in energy and directional displacement in sequence space toward the baseline tumour-associated regime before clear radiological reappearance. In a case with stable disease, voxel distributions remain confined to established low-energy basins without systematic drift. The presented cases serve as proof-of-concept that patient-specific energy manifolds can function as geometric reference systems for longitudinal mpMRI analysis without explicit segmentation or supervised classification, providing a foundation for further investigation of manifold-based tissue-at-risk tracking in neuro-oncology.

Kartikay Tehlan, Lukas Förner, Sina Wendrich +5

Dept. of diagnostic and interventional Radiology and Neuroradiology, University Hospital Augsburg, Germany · Digital Medicine, University Hospital Augsburg, Germany · Chair for Computer Aided Medical Procedures and Augmented Reality, Technical University of Munich, Germany +3

Longitudinal Medical Image AnalysisMagnetic Resonance ImagingEnergy-Based ModelsHealthcareManifold Learning
AskPDF
Apr 8, 2026

Bi-Lipschitz Autoencoder With Injectivity Guarantee

Autoencoders are widely used for dimensionality reduction, based on the assumption that high-dimensional data lies on low-dimensional manifolds. Regularized autoencoders aim to preserve manifold geometry during dimensionality reduction, but existing approaches often suffer from non-injective mappings and overly rigid constraints that limit their effectiveness and robustness. In this work, we identify encoder non-injectivity as a core bottleneck that leads to poor convergence and distorted latent representations. To ensure robustness across data distributions, we formalize the concept of admissible regularization and provide sufficient conditions for its satisfaction. In this work, we propose the Bi-Lipschitz Autoencoder (BLAE), which introduces two key innovations: (1) an injective regularization scheme based on a separation criterion to eliminate pathological local minima, and (2) a bi-Lipschitz relaxation that preserves geometry and exhibits robustness to data distribution drift. Empirical results on diverse datasets show that BLAE consistently outperforms existing methods in preserving manifold structure while remaining resilient to sampling sparsity and distribution shifts. Code is available at https://github.com/qipengz/BLAE.

Qipeng Zhan, Zhuoping Zhou, Zexuan Wang +2

University of Pennsylvania

AutoencodersRepresentation LearningDimensionality ReductionManifold Learning
AskPDF