Riemannian Geometry

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

Sep 15, 2026math.OC

Optimization over covariance matrices with a parameterized metric

The choice of Riemannian metric can strongly influence the convergence of gradient-based optimization over covariance matrices. Euclidean, Bures-Wasserstein and affine-invariant metrics are common choices, but their relative effectiveness depends on the objective. We introduce a two-parameter family defined by

X^{p}LX^{q}+X^{q}LX^{p}=U

, solved for

L

at each tangent vector

U

, that contains all three as exact members, at

(0,0)

,

(1,0)

and

(1,1)

, and extends past them. We treat the choice of member as a particular way of preconditioning for a given problem. To this end, we analyze the conditioning of the Riemannian Hessian at the solution. We show that it obeys a lower bound that depends on

(p,q)

only through the exponent

r=p+q

. When the Euclidean Hessian is a pure power that mixes no eigendirections, the member

p=q=r/2

attains that bound, and a closed-form criterion identifies the other members that do. We discuss ways to tune

r

for a given problem. Experiments on real covariance data confirm the predicted conditioning and the benefit of tuning

r

. A task covariance example shows a further gain from tuning the shape.

Yibang Li, Bamdev Mishra, Pratik Jawanpuria +1

Nanyang Technological University, Singapore · Microsoft India · Indian Institute of Technology Bombay, India

Riemannian Optimization Second-Order Optimization Matrix Optimization Riemannian Geometry

Sep 14, 2026cs.LG

Information-Induced Training Geometry: Exact Reduction, Canonical Completion, and Structured Expressivity

Training data constrains optimizer geometry through the covectors visible to a declared information channel. We study how such partial information determines a full positive cometric relative to a reference and which degrees of freedom remain unidentified. Our central result resolves full-column-rank positive-definite compression under affine-invariant Riemannian geometry. The compression map is a split-Hadamard metric submetry and admits an explicit unique completion that is the affine-invariant nearest full geometry realizing a visible target and yields exact full-to-visible variational reduction. When the channel moves, the completions form a gauge-invariant rank stratification of the positive-definite cone. Its closed-form pullback pair metric separates visible-metric motion from subspace rotation through a reference-mismatch weight, yields an explicit positive-semidefinite multi-direction Gram matrix, and exposes the precise singularity of reference-valued modes. The mechanism is explained by a metric theorem equating ball submetry, attained fiber distance, and lossless reduction of every monotone radial visible decision problem. A smooth split-Hadamard theorem supplies coherent information sheets, proximal commutation, and solution-wise gradient-flow lifting. The positive-definite realization also gives closed-form prior-data shrinkage. Diagonal and block optimizer families reduce to relative-interior conic image tests with valid facial certificates, while deterministic and finite-sample bounds quantify recovery of the visible geometry and its subspace. Together these results characterize exact reduction, reference-dependent completion, and structured expressivity for the stated finite-dimensional affine-invariant model.

Zavier Li

Xidian University Xi’an, China

Information Geometry Riemannian Optimization Riemannian Geometry

Aug 3, 2026cs.AI

The Field Knows: Cross-Dimensional Geometry from Navigation to Black Holes

We introduce a continuous metric field framework trained by a single causal contrastive loss. The framework encodes a scene into coefficients of a fixed symmetric matrix basis, assembles them into a Lie algebra element, and exponentiates the result to a Riemannian or Lorentzian metric. Across dimensions, this field discovers the full spectrum of geometric structures: from obstacle-avoiding geodesics in robot navigation across planar and manipulator configuration spaces, to event horizons of black holes in Lorentzian spacetime. Extensive zero-shot generalization studies demonstrate that the field captures transferable geometric structure rather than memorizing specific configurations. In the black hole setting, the causal loss spontaneously evolves genuine black-hole-like structures with the correct Lorentzian signature. The same loss, the same architecture, and the same training protocol produce the full range of geometric phenomena across dimensions. The field knows geometry, and geometry knows physics.

Chenghao Xu

School of Artificial Intelligence and Robotics and National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha, Hunan, China

Geometric Representation Learning Riemannian Geometry

Aug 2, 2026cs.LG

Sphere Retraction Normalizations

Residual connections are the de facto mechanism for training deep neural networks stably. Geodesic Normalization (GeoNorm) recasts them on a Riemannian manifold, orthogonalizing each layer output against the current hidden state and applying the resulting update through the Riemannian exponential map. Every hidden state thus keeps a constant

\ell_{2}

-norm, confining the residual stream to a hypersphere. The exponential map, however, is only one member of a broad family of retraction maps. We show that on the hypersphere this entire family collapses to a single scalar design choice. What distinguishes one retraction from another is only how the magnitude of an update is converted into a rotation angle within the plane spanned by the hidden state and the update. This view places Euclidean residual connections and GeoNorm in one framework. Instantiating it with the metric projection retraction and the Cayley retraction yields Proj-SpheretNorm and Cay-SpheretNorm, which are exactly norm-preserving yet require only algebraic operations. Both prove to be members of a one-parameter family of angular retractions,

p

-SpheretNorm, whose rotation angle saturates rather than growing without bound. The two methods above are recovered exactly at

p = 1

and

p = 2

, while the identity map and GeoNorm arise only as limits at either end. On nanoGPT, all three methods outperform existing lightweight deep connection schemes, and the best validation loss is attained at finite

p

, indicating that the exponential map is not the preferred retraction for spherical residual streams but merely one end of a spectrum.

Jie Zhang, Cheng-Fang Su, Yi-Jui Huang +1

Department of Computer Science and Information Engineering, National Central University, Taiwan · Department of Applied Mathematics, National Yang Ming Chiao Tung University, Taiwan · Department of Mathematics, National Central University, Taiwan

Residual Learning Riemannian Geometry

Jul 30, 2026cs.CV

Landmark shape spaces with induced metrics

We present a unification of Kendall's landmark shape spaces, where rigid motions are factored out and scale fixed on landmark configurations equipped with Euclidean geometry, with landmark configuration spaces carrying Riemannian metrics descending from right-invariant Sobolev metrics on the diffeomorphism group. The resulting new landmark shape spaces achieve the defining properties of both approaches: The regularity of the descending metric prevents landmarks from colliding, the metric is defined in the ambient space independent of the number of landmarks, local rigid transformations are preserved, global rigid motions are removed, and scale fixed. To achieve this, we define a particular Sobolev-type operator, the screened elasticity operator, whose null-space consists exactly of the rigid motions, we show how this operator descends to achieve the desired geometry, and we present approaches to solving matching problems and computing geodesics numerically. The resulting construction allows the use of landmark configuration spaces with sufficiently regular metrics in applications while retaining the shape invariances that are a hallmark of Kendall's shape spaces.

Sarang Joshi, Peter W. Michor, Stefan Sommer

Non-Rigid Shape Correspondence Riemannian Geometry

Jul 28, 2026math.NA

A Riemannian View on Active Subspaces

Active subspaces provide an explainable, eigenvalue-ordered principle for studying how scalar-valued quantities of interest change the most, on average, over a reduced basis of Euclidean domains. Composition with parallel transport generalizes this principle from Euclidean space to quantities of interest defined over Riemannian manifolds, and the resulting intrinsic formulation is contrasted with the extrinsic, embedding-based gradient average of manifold learning. Either strategy is studied in an intrinsically local sense, restricted to mean-centered geodesic-balls, and within that scope the two are not identical: on the central tangent space, eigenvalues agree to second order in the geodesic radius of the sampled domain, while dominant eigenspaces agree at the same order relative to the spectral gap. Extending activity beyond that central space then calls for either recomputed decompositions over changing tangent spaces or, intrinsically, parallel transport of a single central frame. Hyperspheres are emphasized throughout as a particular manifold of interest, motivated by applications over preshape spaces for statistical shape analysis. Numerical examples over the 2-sphere illustrate the formalism, including the derived ridge recovery at a curvature-limited quadratic rate.

Zachary Grey

Dimensionality Reduction Riemannian Geometry

Jul 8, 2026cs.CL

Riemannian Geometry for Pre-trained Language Model Embeddings

Understanding the geometric structure of pre-trained language model embeddings matters for interpretability and safety. We ask whether sentence-level classification signal lives in the Riemannian geometry of contextual token embeddings, and probe it by extracting per-token pullback metrics from a learned encoder's analytical Jacobian and aggregating them with the Fréchet mean on the symmetric positive definite (SPD) manifold; we call this procedure Riemannian Mean Pooling (RMP). Across three datasets with non-trivial linguistic structure (CoLA, CREAK, RTE), RMP outperforms Euclidean mean pooling, while on FEVER-Symmetric, a benchmark constructed to remove annotation-driven lexical artifacts, the method correctly stays at chance. Ablations show that a randomly initialised encoder combined with Fréchet aggregation already beats Euclidean pooling on two of the three signal-bearing datasets, localising the source of the gain to the geometric aggregation rather than to learned manifold structure; the trained encoder contributes additional signal specifically on CREAK, the most knowledge-heavy of the three signal-bearing datasets.

Szczepan Konior, Alexandre Quemy, Przemysław Klocek +2

IBM Automation and AI, Krakow, Poland · Hother, Krakow, Poland · University of Warsaw +1

Sentence Embeddings Geometric Representation Learning Riemannian Geometry Representation Geometry in Language Models

Jul 7, 2026stat.ML

On the convergence of graph Laplacians with a symmetric divergence

When analyzing a manifold learning algorithm for data lying on a smooth, compact, connected Riemannian submanifold

(\mathcal{M}, g)

of

\mathbb{R}^d

, a key estimate for the geodesic distance

d_g

is that there exists

K > 0

such that

0 \leq d_g(p, q)^2 - \|p-q\|^2 \leq K d_g(p, q)^4

for all

p, q \in \mathcal{M}

. We observe that more generally, when

\mathcal{M}

is equipped with a smooth symmetric divergence

D

satisfying a non-degeneracy condition and

g

is given by

g_p := \frac{1}{2}\mathrm{Hess}_p(D(p, \cdot))

for all

p \in \mathcal{M}

, there exists

K > 0

such that

\left| D(p, q) - d_g(p, q)^2 \right| \leq K d_g(p, q)^4

for all

p, q \in \mathcal{M}

. We demonstrate that this is sufficient for the pointwise convergence of graph Laplacians constructed with

D

and discuss examples where

D

is given by the Sinkhorn divergence on a family of probability measures parametrized by a manifold.

Liane Xu

Program in Applied and Computational Mathematics, Princeton University

Manifold Learning Riemannian Geometry

Jun 16, 2026cs.LG

Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds

Gaussian width is a central geometric complexity measure in high-dimensional probability, compressed sensing, convex optimization, and learning theory. It quantifies the average extent of a set along random directions, thereby capturing the effective dimension of constraint sets, hypothesis classes, and descent cones. However, this notion is intrinsically Euclidean. Statistical models instead carry a natural Riemannian geometry induced by the Fisher information metric, where directions are scaled according to statistical distinguishability rather than ambient Euclidean length. We introduce Fisher width, a Fisher-geometric analogue of Gaussian width for statistical manifolds. At a parameter point

θ

, Fisher width replaces the Euclidean identity by the local metric tensor

G(θ)^{1/2}

, measuring the Gaussian width of the Fisher-rescaled set. This makes the resulting quantity sensitive to local statistical curvature and invariant under smooth reparameterizations. We develop the basic theory of Fisher width, showing that it retains key structural features of Gaussian width, including concentration, metric perturbation stability, and spectral comparison bounds with the Euclidean baseline, while also capturing anisotropic geometric effects invisible to Euclidean measures. As an application, we prove a generalization bound for Fisher-Lipschitz hypothesis classes and propose computable estimators, which we evaluate empirically on MNIST across three model classes. Fisher width is to statistical manifolds what Gaussian width is to Euclidean convex bodies. This work lays the foundation for studying complexity and learning on curved statistical manifolds.

Vu Khac Ky

Department of Mathematics, FPT University, Vietnam

Information Geometry Riemannian Geometry

Jun 12, 2026cs.LG

Riemannian Metric Matching for Scalable Geometric Modeling of Distributions

High-dimensional datasets often concentrate near low-dimensional structures, but estimating their geometry from samples typically relies on graphs and kernels that scale poorly with dataset size and dimension. We propose Riemannian metric matching: a denoising probabilistic framework for learning the Riemannian geometry of data using neural networks. Specifically, we learn the carré du champ operator, which, using diffusion geometry, gives us access to the Riemannian geometry toolkit for downstream machine learning and statistical tasks. Our key observation is that the carré du champ operator can be formulated as a conditional expectation over random perturbations of the data, which can be exploited for sample-wise training and constant cost, amortized inference without explicit kernel construction. Empirically, metric matching rivals or improves the accuracy of

k

-NN-based diffusion geometry estimators, while enabling amortized inference that is up to

400\times

faster, and supports graph-free geometric analysis on high-dimensional images where nearest neighbors break down.

Jacob Bamberger, Adam Gosztolai, Pierre Vandergheynst +2

Department of Computer Science, University of Oxford, Oxford, UK · Institute of Artificial Intelligence, Medical University of Vienna, Vienna, Austria · EPFL, Lausanne, Switzerland +1

Geometric Representation Learning Metric Learning Riemannian Geometry

Jun 11, 2026math.NA

Approximating Gaussian Whittle-Matern Fields over Well-Centered Triangulations of Riemannian Manifolds

Markovian Whittle-Matérn fields have been convergently approximated by discrete Gauss Markov Random Fields (GMRFs) with sparse precision matrices using a Finite Element approximation of the two-parameter family,

(κ^2 - Δ)^{α/2} u = \mathcal{W}, \;\; κ\in \mathbb{R}, \; α\in \mathbb{N}.

of SPDEs. Using recent developements in the analysis of Discrete Exterior Calculus (DEC), we present a different, yet closely related, convergent GMRF approximation to these Matérn fields over complete, boundaryless Riemannian manifolds discretized as well-centered simplicial complexes. This convergent method (i) is agnostic to

α, κ

and thus allows a universal approximation scheme for the precision and covariance matrices of the entire

(α, κ)

-family of GMRFs, so they may be inferred rather than guessed. (ii) inherently models pointwise and piecewise-smoothed measurements of a random field and approximates both equally well (iii) is computationally independent of the interpolants used - it suffers no overhead if one convergent interpolant were replaced with another suitable interpolant over the same mesh. Furthermore, we show that, on discretizations that are well-connected in a precise sense, and volume-concentrated, the precision matrices are spectral functions of a graph-laplacian. We provide a low rank approximator to the family of such Matérn GMRFs and mention a use case: reducing the number of measurements needed to model the GMRF by compressed-sensing.

Srinivas Nambirajan

Markov Random Fields Riemannian Geometry

Jun 6, 2026stat.ML

Barycentric Projections of Optimal Transport Plans on Riemannian Manifolds

Optimal transport couplings are probabilistic objects, while many learning pipelines require deterministic maps. In Euclidean space, barycentric projection converts a coupling into a map by taking conditional expectations, but on a Riemannian manifold curvature and cut loci make this operation nontrivial. We develop a framework for barycentric projections of transport couplings on Riemannian manifolds. The intrinsic projection maps each source point to the conditional Fréchet mean of its destination law and is shown to be the best deterministic representative under squared geodesic loss. The corresponding minimum value is an integrated conditional Fréchet variance, which vanishes exactly for map-induced couplings and therefore defines a conditional-variance Monge defect. We also study a tangential log-exp projection, prove its Euclidean exactness, its compatibility with Brenier-McCann maps in the Monge case, and its interpretation as the first unit Riemannian gradient update for the intrinsic objective. For discrete couplings, both constructions decompose row-wise into weighted Fréchet mean and log-exp problems. Experiments on spherical data, synthetic SPD data, and real EEG covariance matrices support the proposed division of roles: the intrinsic projection is the variational representative, while the tangential projection is a useful local displacement surrogate.

Kisung You

Department of Mathematics Baruch College

Riemannian Geometry Optimal Transport

Jun 4, 2026cs.CV

Geodesic Flow Matching on a Riemannian Degradation Manifold for Blind Image Restoration

Blind image restoration requires recovering clean images from observations corrupted by unknown and potentially mixed degradations. While recent deterministic flow-based methods model restoration as transport processes that map degraded images to clean ones, they typically rely on Euclidean interpolation, implicitly assuming linear degradation geometry. In this paper, we explicitly model degradations as points on a low-dimensional Riemannian manifold and formulate restoration as geodesic transport on the joint image-manifold space. Using a geodesic flow matching objective, we learn intrinsic transport dynamics that respect the curvature of degradation space. This framework generalizes linear flow matching, provides a principled treatment of mixed degradations as geodesic compositions, and yields a clean theoretical interpretation for generalization beyond observed degradations.

Akshay Janardan Bankar, Ankita Chatterjee, Sayan Banerjee +3

Samsung Research Institute, Bangalore, India

Flow Matching Riemannian Flow Matching All-in-One Image Restoration Image Restoration Riemannian Geometry

Jun 4, 2026cs.IR

Knowledge Manifold: A Riemannian Geometric Framework for Semantic Mapping and Geodesic Analysis of Scientific Literature

We present the knowledge manifold: a Riemannian geometric space in which a corpus of documents is arranged according to semantic positional relationships derived from character n-gram TF-IDF representations. The framework proceeds in five tightly coupled stages. First, each document is converted to a character-level n-gram TF-IDF vector (4-7 grams, up to 250,000 features, L2-normalized) and embedded in a two-dimensional knowledge map via constrained stress minimization with repulsion, variance, and centering regularizers. Second, knowledge at an arbitrary query point is estimated through Smoothed Particle Hydrodynamics (SPH) interpolation using a cubic-spline kernel, yielding an interpolated TF-IDF feature vector that can be linguistically characterized. Third, directional knowledge gradients at 0, 45, and 90 degrees are computed from the SPH interpolation map, and pairwise directional similarity is quantified via inner product and cosine similarity. Fourth, a Gaussian Process Regression (GPR) model, with a Constant x RBF + White kernel fitted on a 10-dimensional SVD projection, provides a Bayesian posterior mean, uncertainty estimate, and per-document contribution rate at the query point. Fifth, geodesics in the knowledge space are obtained by minimizing a discrete Riemannian path energy derived from the SPH-induced metric tensor, using L-BFGS-B with seven deterministic initial-path candidates. We apply the formulation to a corpus of 20 papers in fiber-reinforced composite materials and aerospace structural mechanics, showing that the semantic map recovers meaningful research clusters, geodesic paths reveal natural conceptual bridges between distant topics, and SPH/GPR interpolation enables the generation of virtual knowledge: hypothetical paper abstracts describing unstudied but geometrically predicted research directions.

Tomonaga Okabe, Kazuhiko Komatsu

Department of Aerospace Engineering, Tohoku University, 6-6-01, Aoba, Aramaki, Aoba-ku, Sendai, Miyagi 980-8579, Japan · Research Center for Green X-Tech, Tohoku University, 6-6-11, Aoba, Aramaki, Aoba-ku, Sendai, Miyagi 980-8579, Japan

Knowledge Representation Manifold Learning Semantic Mapping Riemannian Geometry

Jun 2, 2026cs.LG

Are Common Substructures Transferable? Riemannian Graph Foundation Model with Neural Vector Bundles

Foundation models have sparked a revolution via a pretraining-adaptation paradigm, with recent efforts extending this success to graphs. Unlike other modalities, graphs contain rich structural patterns, yet their structural transferability remains poorly understood. Prior studies consider common substructures in the discrete realm, and we are motivated by a fundamental question: Are common substructures transferable? The underlying theory is largely underexplored. In this work, we shift toward learning transferable structures through the lens of functional behavior. Theoretically, we connect transferable substructures to intrinsic geometry of the representation space. However, characterizing such intrinsic geometry has rarely been touched. Grounded in Riemannian geometry, we develop a graph intrinsic geometry learning framework called Neural Vector Bundle, which enables parsing intrinsic geometry with local coordinates. Building on this, we design GAUGE, a pretrainable neural architecture that constructs the vector bundle, flattening geometrically compatible local coordinates, and a new Dirichlet loss, which also measures the transfer effort. We empirically validate its superior expressiveness in challenging tasks including zero-shot link prediction and graph isomorphism.

Li Sun, Zhenhao Huang, Yiding Wang +3

Beijing University of Posts and Telecommunications, Beijing, China · North China Electric Power University, Beijing, China · University of Cambridge, Cambridge, UK +1

GNN Expressivity Geometric Representation Learning Transfer Learning Graph Foundation Models Riemannian Geometry

Jun 1, 2026cs.RO

Direct Informed Sampling on Riemannian Manifolds via Loewner Order Lower Bounds

Informed sampling techniques accelerate sampling-based motion planners by focusing the search on promising regions of the state space, yet most existing methods rely on Euclidean heuristics that become inadmissible under configuration-dependent Riemannian metrics. While scalar eigenvalue bounds restore admissibility by uniformly scaling the Euclidean distance, they discard the directional structure of the metric, producing overly conservative informed sets. We propose a matrix-valued admissible heuristic that exploits the Loewner order on symmetric positive definite matrices to compute the tightest constant lower bound on the metric tensor while preserving its full directional structure. The Cholesky factorization of this bound defines a linear map to an isotropic Euclidean space in which the Riemannian informed set reduces to a standard prolate hyperspheroid, enabling direct, rejection-free sampling using existing algorithms. Experiments on manipulation tasks with a 6-DoF UR5, 7-DoF Franka, and 14-DoF PR2 under three distinct Riemannian metrics show that our heuristic produces consistently tighter informed sets than both the Euclidean and scalar eigenvalue bounds, accelerating convergence across multiple state-of-the-art asymptotically optimal planners.

Phone Thiha Kyaw, Jonathan Kelly

Space and Terrestrial Autonomous Robotic Systems (STARS) Laboratory, University of Toronto Institute for Aerospace Studies (UTIAS), Toronto, Ontario M3H 5T6, Canada

Motion Planning Riemannian Geometry Sampling-Based Motion Planning

Jun 1, 2026cs.LG

Riemannian Gradient Descent for Low-Rank Architectures

We explore Riemannian optimization techniques for rank-factored matrix parameters, targeting contemporary deep learning applications. We examine ten points in the algorithm design space: two geometries for rank-

r

matrices, three geometries for rank-

r

partial isometries, and block-matrix variants of these five, where factors are shared across block-rows and block-columns. We apply our methods to the multihead attention parameters in small language models. After tuning learning rates, our methods do not conclusively outperform an AdamW baseline. Our implementations are available online.

Nicholas Knight

NVIDIA

Riemannian Optimization Low-Rank Approximation Matrix Optimization Riemannian Geometry Multi-Head Attention

May 31, 2026cs.LG

Riemannian Optimization for Hadamard Products of Low-Rank Matrices

The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the presence of additional symmetries under coupled row/column scalings between the two factors. In order to leverage the geometry of the space, we formulate the learning of such matrices as optimization on a Riemannian quotient manifold. We propose a novel block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product. The metric is shown to be invariant under these symmetries. We develop a Riemannian gradient descent algorithm that uses a tuning-free Gauss--Newton step size and scales linearly in the number of observed entries per iteration. The versatile framework of Riemannian quotient optimization enables both first-order and second-order Riemannian methods, the latter through a closed-form connection and the Riemannian Hessian. Experiments on real and synthetic datasets illustrate the efficacy of our proposed Riemannian approach.

Pratik Jawanpuria, Ankish Chandresh, Bamdev Mishra

Centre for Machine Intelligence and Data Science, Indian Institute of Technology Bombay, India · Microsoft India

Riemannian Optimization Low-Rank Approximation Matrix Optimization Riemannian Geometry Matrix Factorization

May 24, 2026cs.LG

Riemannian-Manifold Steering: Geometry-Aware Generative Autoencoders for Label-Free Steering

Steering a language model - intervening on its internal activations to change downstream behaviour - has recently expanded beyond linear interpolation to nonlinear methods such as angular and kernelized steering, which define intervention transformations without learning an explicit geometry over paths in activation space. Freshly introduced geometry-aware manifold methods do learn such a geometry, but require labelled class centroids together with prescribed cyclic or sequential structure. These assumptions restrict where manifold steering can be applied, since existing constructions require labelled centroids and compatible boundary conditions. We recast manifold steering more broadly as \textbf{Riemannian geodesic computation} on activation space, recovering linear and labelled-spline steering as geodesics under particular choices of metric. A principled metric within this framework is the output-space Hellinger distance pulled back to activations; we approximate this with a learned encoder trained on output distances over a small concept-token schema - no per-prompt labels, no topology prior, and no per-task curve fitting. Empirically, the method reliably drives the model onto the target class across all tasks in a standard four-task language-model arithmetic benchmark, while following more behaviourally natural trajectories than baselines on smaller output spaces. We thereby provide a unified Riemannian framework for manifold steering together with a schema-supervised, label-free instantiation that operates without labelled centroids or prescribed boundary conditions.

Narmeen Oozeer, Shivam Raval, Philip Quirke +4

Martian · Harvard University · Thoughtworks +1

Language Model Steering Representation Steering Riemannian Geometry Representation Geometry in Language Models

May 22, 2026cs.LG

Riemannian Archetypal Analysis: Interpretable non-linear data analysis on deformed star distributions

Classical archetypal analysis is appealing for its interpretability, but its linear geometry can limit performance on data with strongly non-linear structure; at the same time, existing neural extensions improve flexibility while often weakening the geometric meaning of archetypes and interpolations. In this work, we develop a Riemannian version of archetypal analysis based on data-driven pullback geometry for real-valued data, with the goal of combining the interpretability of classical archetypal analysis with the expressive power of modern non-linear models. We introduce a class of deformed star distributions together with associated pullback Riemannian geometry to provide a statistical interpretation of the resulting manifold mappings, define the Riemannian archetypal mapping (RAM) as a projection onto the manifold of geodesically convex combinations of archetypes, and propose a practical optimization scheme based on convex relaxation followed by non-convex refinement. We further propose a learning scheme that yields reasonable, albeit generally suboptimal, deformed star distributions from data. Experiments on synthetic examples and MNIST show that the resulting framework produces meaningful geodesics, useful denoising projections, and geometry-aware classifications, while also clarifying where current optimization limitations remain.

Willem Diepeveen, Deanna Needell

Department of Mathematics University of California, Los Angeles Los Angeles, CA 90095, USA

Manifold Learning Dimensionality Reduction Riemannian Geometry

May 21, 2026cs.LG

Disentanglement Beyond Generative Models with Riemannian ICA

There is a gap between the theoretical foundations of disentanglement and the practice of modern representation learning. Existing theoretical frameworks, particularly Independent Component Analysis (ICA) and its nonlinear variants, assume a generative model with statistically independent latent variables underlying the data so that disentanglement amounts to identifying the latents that could have generated the data. This generative framework is interpretable and theoretically justified, but its strong assumptions make it difficult to apply to modern representation learning. Modern pretrained encoders often learn features that exhibit disentangled properties without making generative assumptions, yet there is no general theory for interpreting these features as independent factors of variation. We take a step toward such a theory by introducing Riemannian ICA (RICA), which replaces ICA's global generative model with local geometric structure. RICA is founded on the observation that in ICA, the factors of variation underlying a data point can be understood through radial curves emanating from the point that map to axis-aligned lines in the latent space. We formalize this perspective using Riemannian geometry and introduce our theory in a way that is consistent with the existing generative approach. Our main contribution is the disentanglement tensor, which encodes a second-order notion of disentanglement that we call pointwise disentanglement. This tensor depends on the Hessian of the data log likelihood as well as the Ricci curvature induced by the model. In a controlled source recovery setting with known ground-truth sources, RICA recovers sources across several manifolds, while the success of ICA baselines depends on the coordinates used to represent the observations. Our work provides a theoretical basis for studying local disentanglement without assuming a global generative model.

Edmond Cunningham

University of Massachusetts Amherst

Disentangled Representation Learning Independent Component Analysis Geometric Representation Learning Riemannian Geometry

May 21, 2026cs.LG

Riemannian geometry meets fMRI: the advantages of modeling correlation manifolds and eigenvector subspaces

Correlation matrices are fundamental summaries of functional brain networks, yet standard analyses often treat entries independently, ignoring the curved geometry of correlation space. Existing geometric methods frequently lack closed-form operations or depend on arbitrary region ordering, limiting scalability. We introduce a scalable geometric framework with two components: (i) the Off-log metric, a smooth transformation mapping correlation matrices to symmetric zero-diagonal matrices. This enables closed-form expressions for distances, Frechet means, and linear models, allowing standard statistical modeling without complex manifold optimization. (ii) Grassmannian subspace discrimination, which compares subjects via principal-angle distances between eigenvector subspaces, resolving inherent sign and basis ambiguities. Both components integrate into standard machine-learning workflows for inference, regression, and classification. Validated across two clinical cohorts (Parkinson's and psychosis) and three ageing fMRI datasets, the Off-log metric increased sensitivity in permutation tests and matched or exceeded Riemannian and Euclidean baselines in classification. Brain-age prediction performance was comparable, with Riemannian metrics excelling in two of three cohorts. The Grassmannian method consistently outperformed Euclidean baselines, highlighting disease-relevant networks. Overall, geometry-aware representations improve sensitivity and predictive performance while remaining straightforward to deploy at scale.

Mario Severino, Manuela Moretto, Robert A. McCutcheon +1

Department of Information Engineering, University of Padova, Padova, Italy · Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King’s College London, London, United Kingdom · Department of Psychiatry, University of Oxford, Warneford Hospital, Warneford Ln, Headington, Oxford OX3 7JX, United Kingdom +2

Functional Connectivity Neuroimaging Riemannian Geometry

May 18, 2026cs.LG

Riemannian Networks over Full-Rank Correlation Matrices

Representations on the Symmetric Positive Definite (SPD) manifold have garnered significant attention across different applications. In contrast, the manifold of full-rank correlation matrices, a normalized alternative to SPD matrices, remains largely underexplored. This paper introduces Riemannian networks over the correlation manifold, leveraging five recently developed correlation geometries. We systematically extend basic layers, including Multinomial Logistic Regression (MLR), Fully Connected (FC), and convolutional layers, to these geometries. Besides, we present methods for accurate backpropagation for two correlation geometries. Experiments comparing our approach against existing SPD and Grassmannian networks demonstrate its effectiveness.

Ziheng Chen, Xiaojun Wu, Bernhard Schölkopf +1

Department of Information Engineering and Computer Science, University of Trento, Trento, Italy · Max Planck Institute for Intelligent Systems, Tübingen, Germany · School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China

Geometric Representation Learning Manifold Learning Riemannian Geometry

May 6, 2026cs.LG

Beyond Rigid Geometries: The Spline-Pullback Metric for Universal Diffeomorphic SPD Representation Learning

The integration of Symmetric Positive Definite (SPD) matrices into deep learning has historically relied on fixed algebraic Riemannian metrics. Analogous to hand-crafted features in classical machine learning, these static formulations impose rigid geometries limiting network expressivity and adaptability. Recent attempts to parameterize these geometries often violate the axioms of primary matrix functions through unconstrained powers or rank-dependent scaling, inviting spatial folding, loss of global surjectivity, and gradient collapse at spectral singularities. In this paper, we introduce the Spline-Pullback Metric (SPM), instantiated as Spectral-SPM and Cholesky-SPM, marking a paradigm shift from static metric selection to universal geometric approximation. By parameterizing the global diffeomorphism via a rank-invariant, monotonically constrained B-spline, SPM acts as a dense universal approximator for strictly increasing

C^1

diffeomorphisms and theoretically subsumes existing pullback metrics while enabling localized non-linear spectral modelling. Topologically, SPM provides a globally bijective pullback geometry precluding rank-swapping discontinuities and gradient instabilities. Empirically, SPM achieves a state-of-the-art performance across 3 datasets utilizing Linear Probes, SPDNets, and deep Riemannian ResNets.

Tushar Das, Subrata Dutta, Sarmistha Neogy +1

National Institute of Technology Jamshedpur · Jadavpur University · Machine Vision and Intelligence Lab National Institute of Technology Jamshedpur

Geometric Representation Learning Metric Learning Riemannian Geometry

May 5, 2026stat.ML

Entropic Riemannian Neural Optimal Transport

Many machine learning problems involve data supported on curved spaces such as spheres, rotation groups, hyperbolic spaces, and general Riemannian manifolds, where Euclidean geometry can distort distances, averages, and the resulting optimal transport (OT) problem. Existing manifold OT methods have pursued amortized out-of-sample maps, while entropic regularization has made discrete OT more scalable, but these advantages have remained largely disjoint. We propose Entropic Riemannian Neural Optimal Transport (Entropic RNOT), a unified framework that combines intrinsic entropic OT with amortized out-of-sample evaluation on Riemannian manifolds. Our method learns a single target-side Schrödinger potential through a neural pullback parameterization, recovers the induced Gibbs coupling, and uses the resulting conditional laws to construct intrinsic transport surrogates. These include barycentric projections on Cartan-Hadamard manifolds and heat-smoothed conditional surrogates on stochastically complete manifolds, the latter turning possibly atomic target laws into absolutely continuous ones. For fixed regularization

\varepsilon>0

, we prove that the proposed hypothesis class recovers the entropic optimal coupling in strong probabilistic metrics. As consequences, barycentric surrogates converge in

L^2

, while heat-smoothed surrogates are stable at fixed heat time and asymptotically unbiased as the heat time vanishes. The guarantees hold for compactly supported data on possibly noncompact manifolds. Empirically, our method matches or improves over Euclidean, tangent-space, and log-Euclidean baselines on benchmarks over

\mathbb{S}^2

,

\mathrm{SO}(3)

,

\mathrm{SPD}(3)

,

\mathrm{SE}(3)

, and

\mathbb{H}^2

, scales favorably relative to discrete manifold Sinkhorn, and in a protein-ligand docking application, refines poses on

\mathrm{SE}(3)

without retraining or per-instance optimization.

Alessandro Micheli, Silvia Sapora, Anthea Monod +1

Imperial College London London, UK · University of Oxford Oxford, UK · Statens Serum Institut Copenhagen, Denmark +1

Riemannian Geometry Optimal Transport

May 5, 2026cs.LG

Flow Matching on Symmetric Spaces

We introduce a general framework for training flow matching models on Riemannian symmetric spaces, a large class of manifolds that includes the sphere, hyperbolic space and Grassmannians. We exploit their algebraic structure to reformulate flow matching on symmetric spaces as flow matching on a subspace of the Lie algebra of their isometry group, thus linearizing the problem and greatly simplifying the handling of geodesics. As an application, we showcase our framework on the real Grassmannians

\operatorname{SO}(n) / \operatorname{SO}(k) \times \operatorname{SO}(n-k)

.

Francesco Ruscelli, Ferdinando Zanchetta, Rita Fioresi

Institute for Mathematics University of Heidelberg 69120 Heidelberg, Germany · Department of Pharmacy and Biotechnologies University of Bologna Bologna, Italy

Flow Matching Riemannian Flow Matching Riemannian Geometry

May 4, 2026math.DG

Foundations of Riemannian Geometry for Riemannian Optimization: A Monograph with Detailed Derivations

Riemannian geometry provides the fundamental framework for optimization on nonlinear spaces such as matrix manifolds, which arise in machine learning, signal processing, and robotics. While the underlying theory is classical, existing literature often presents results at a high level of abstraction, omitting the detailed coordinate-level derivations required for implementation and algorithm development. This work provides a self-contained and rigorous treatment of the foundations of Riemannian geometry, with a focus on explicit derivations tailored to Riemannian optimization. We systematically develop the key geometric structures -- including tangent and cotangent spaces, tensor calculus, metric tensors, Levi-Civita connections, curvature, and geodesics -- emphasizing step-by-step derivations in coordinates and matrix form. Building on these foundations, we derive the Riemannian gradient, Hessian, exponential map, and retraction in a form suitable for numerical computation. We further specialize these constructions to important matrix manifolds, including the Stiefel, Grassmann, and SPD (Symmetric Positive Definite) manifolds, providing explicit formulas widely used in optimization and geometric machine learning. This monograph develops a unified and implementation-oriented treatment of Riemannian geometry for optimization on manifolds. Its main contribution is the systematic organization and detailed derivation of classical geometric constructions in forms directly usable for algorithm design and numerical implementation. By connecting coordinate-level differential geometry with matrix-manifold formulas, the monograph bridges the gap between abstract theory and practical computation, and provides a reference for researchers and practitioners working in Riemannian optimization and related fields.

Benyamin Ghojogh

Waterloo, Ontario, Canada

Riemannian Optimization Riemannian Geometry

May 3, 2026cs.CV

MOC-3D: Manifold-Order Consistency for Text-to-3D Generation

With the burgeoning development of fields such as the Metaverse, Virtual Reality (VR), and Digital Twins, text-to-3D generation has emerged as a research hotspot in both academia and industry. Currently, optimization methods based on Score Distillation Sampling (SDS) utilizing 2D diffusion priors have become the mainstream technological paradigm in this field. However, due to the view bias of 2D priors and the mode-seeking ambiguity combined with gradient noise induced by high Classifier-Free Guidance (CFG), these methods still suffer from macro-topological inconsistency (e.g., the Janus problem) and micro-geometric discontinuity. To address these challenges, we propose MOC-3D, a text-to-3D generation method based on geometric manifold and semantic view-order consistency. Built upon the ScaleDreamer framework, our method incorporates a Semantic View-Order Constraint Module and a Manifold-based Feature Continuity Module. The former aims to rectify macro-topological inconsistency, while the latter focuses on eliminating micro-geometric discontinuity. Specifically, the Semantic View-Order Constraint Module leverages the prior knowledge of CLIP to impose a Monotonicity Rank Constraint on semantic score representations across different views, thereby providing effective guidance for the global topological structure of 3D objects. Meanwhile, the Manifold-based Feature Continuity Module employs the Riemannian Metric on the Symmetric Positive Definite (SPD) manifold. By measuring the distance of feature statistical distributions in the Riemannian space, it promotes the smooth evolution and continuity of micro-textures across multi-views in a statistical sense. Under the macro-micro synergistic optimization of these two modules, our model can simultaneously improve macro-structural consistency and micro-detail continuity.

Chenyang Fan, Junshi Cheng, Wen Yang +5

Chongqing Normal University Chongqing, China

Multi-View Consistency Text-to-3D Generation Riemannian Geometry

Apr 20, 2026cs.CV

Geometric Decoupling: Diagnosing the Structural Instability of Latent

Latent Diffusion Models (LDMs) achieve high-fidelity synthesis but suffer from latent space brittleness, causing discontinuous semantic jumps during editing. We introduce a Riemannian framework to diagnose this instability by analyzing the generative Jacobian, decomposing geometry into \textit{Local Scaling} (capacity) and \textit{Local Complexity} (curvature). Our study uncovers a \textbf{

Geometric Decoupling"}: while curvature in normal generation functionally encodes image detail, OOD generation exhibits a functional decoupling where extreme curvature is wasted on unstable semantic boundaries rather than perceptible details. This geometric misallocation identifies

Geometric Hotspots" as the structural root of instability, providing a robust intrinsic metric for diagnosing generative reliability.

Yuanbang Liang, Zhengwen Chen, Yu-Kun Lai

School of Computer Science and Informatics, Cardiff University, Cardiff, United Kingdom

Neural Representation Geometry Latent Diffusion Models Generative Modeling Riemannian Geometry

Feb 10, 2026cs.LG

Learning on the Manifold: Unlocking Standard Diffusion Transformers with Representation Encoders

Leveraging representation encoders for generative modeling offers a path for efficient, high-fidelity synthesis. However, standard diffusion transformers fail to converge on these representations directly. While recent work attributes this to a capacity bottleneck proposing computationally expensive width scaling of diffusion transformers we demonstrate that the failure is fundamentally geometric. We identify Geometric Interference as the root cause: standard Euclidean flow matching forces probability paths through the low-density interior of the hyperspherical feature space of representation encoders, rather than following the manifold surface. To resolve this, we propose Riemannian Flow Matching with Jacobi Regularization (RJF). By constraining the generative process to the manifold geodesics and correcting for curvature-induced error propagation, RJF enables standard Diffusion Transformer architectures to converge without width scaling. Our method RJF enables the standard DiT-B architecture (131M parameters) to converge effectively, achieving an FID of 3.37 where prior methods fail to converge. Code: https://github.com/amandpkr/RJF

Amandeep Kumar, Vishal M. Patel

Johns Hopkins University

Flow Matching Diffusion Transformer Riemannian Flow Matching Generative Modeling Diffusion-Based Image Generation Flow-Based Generative Modeling+1