High-dimensional data are often assumed to lie on lower-dimensional manifolds. We study how to construct diffusion processes on this data manifold using only point cloud samples and without access to charts, projections, or other geometric primitives. Here, we introduce Implicit Manifold-valued Diffusions (IMDs), a data-driven mathematical formalism for defining stochastic differential equations in the original high-dimensional space that describe drifting Brownian particles evolving intrinsically on the underlying manifold. Our construction hinges on approximating the corresponding infinitesimal generator of the diffusion process using a proximity graph over the data and using the carré-du-champ of the generator, which encodes the local tangent spaces of the manifold and lifts the intrinsic process into ambient coordinates. We show that as the number of samples grows, our discrete diffusion process converges in law on the space of probability paths to its smooth manifold counterpart. We further present an Euler-Maruyama scheme for the numerical integration of IMDs. We validate our framework using numerical experiments on synthetic manifolds and the MNIST data manifold, showing that IMDs remain confined over the manifold and enable its guided exploration. Our work provides the mathematical foundation and practical implementations of diffusion processes on data manifolds, opening new avenues for manifold-aware sampling, exploration, and generative modeling.
Figures & tables
Figure 1: No prior knowledge of T2 beyond the samples (gray) is required to compute the displayed Brownian motion (red).
Figure 2: We use the score of a pre-trained score model sσ(x)≈∇xlogpσ(x) as a retraction toward the data manifold.
Dimension d
Step size h
IMD ( Et[δt] / maxtδt )
IMD+DRGD ( Et[δt] / maxtδt )
2
10−3
0.005±0.002/0.014±0.004
0.002±0.001/0.004±0.001
7
10−3
0.130±0.048/0.310±0.091
0.003±0.001/0.013±0.002
7
10−4
0.104±0.050/0.273±0.092
0.003±0.001/0.007±0.001
7
2×10−5
0.096±0.043/0.262±0.082
0.004±0.000/0.006±0.001
Table 1: Geometric fidelity of IMD Brownian Motion on spheres of varying dimensions. The optional DRGD retraction step of Algorithm 5.1 helps control δt as step size increases, supporting our heuristic from Section 5.1 . Cells report Et[δ]t/maxtδt , each as mean ± one standard deviation over 30 independent runs, with time horizon T=2 . See Appendix for details H.2 .
Figure 3: Visual comparison of IMDs against heuristic approaches. (a) IMDs remain on S2 . (b) CDC-projected noise without Laplacian drift correction accumulates errors. (c) Isotropic noise with DRGD retraction follows the manifold, but not accurately.
Figure 4: Langevin sampling on S7 . Histogram of the endpoint statistic θ=⟨μ,YT⟩∈[−1,1] (teal) under Langevin dynamics computed with IMDs, compared with the theoretical density induced by the von Mises-Fisher distribution from Eq. ( 21 ) (red) on S7⊂R8 . Parameters used: κ=10 , h=10−4 .
Figure 5: Potential-guided implicit manifold exploration. (a) IMD Langevin sampling ( 19 ) starting from the digit ‘one’ and driven by a potential with a minimum at the digit ‘seven’. The nearest data point at a particular time step is shown for visual guidance of the transition. The trajectory evolves continuously on the manifold, generating coherent transformation and consistent transitions. (b) Standard Langevin dynamics with the same potential, retracted via DRGD. (c) Zoomed in trajectory showing incoherent dynamics blending source and target early in the dynamics and rapid, discontinuous transition to the target.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: IMD implementations in 3-dimensional space, with size is 10−3 . We notice that while the Swiss Roll is not boundaryless, the nearest-neighbour approach to estimating L(Xt) via PN∗ prevents off-manifold drift.
Method
Average radial error
Max. radial error
IMD
0.009±0.004
0.021±0.007
CDC only
0.474±0.014
0.866±0.026
R3 Noise + DRGD Retraction
1.705±0.983
3.732±1.629
Appendix
Table 2: Radial errors over trajectories of 5000 steps with step size h=10−3 . We report the time-averaged radial error and the maximum radial error along each trajectory, both averaged over 50 independent runs ( ± one standard deviation). IMDs maintain low, non-cumulative errors, whereas CDC only ( G.1 ) and R3 noise + DRGD ( G.1 ) accumulate errors.
Method
Average NN Distance ↓
Max Jump ↓
IMD
0.019±0.000
0.116
CDC only
0.736±0.035
0.420
R3 Noise + DRGD
0.073±0.000
0.375
Appendix
Table 3: Swiss roll evaluation for a latent sheet. Lower is better for both NN distance and max jump. Reported values are means and standard errors over 50 starting points with a fixed stepsize h=10−3 for 10,000 steps each.
Figure 7: Comparison of diffusion trajectories ( h=10−3 , 10 k steps) on the Swiss roll (top row) and corresponding latent sheet projections (bottom row) . Similar to Fig. 2 , we notice error accumulation in the normal direction, which manifests non-locality as the discontinuous jump in the latent sheet. We also note the lack of continuous exploration of the isotropic BM compared to IMDs.
Figure 8: Implicit manifold interpolation in the MNIST latent space, guided by the potential ( 22 ). IMDs (teal) follow the learned tangential geometry, while DRGD (red) rapidly moves toward the target through a score-based correction. Following Conjecture 7 , we interpret this behaviour to be consistent with locally flat intra-class geometry and lower-dimensional/curvature inter-class transitions.
Figure 9: Orthogonality error of IMD Brownian motion trajectories on the orthogonal matrix manifold O(m) , for dimension m∈{2,3,10} with a fixed step size of 10−4 . Mean (solid line) and ±1 standard deviation (shaded area) over 20 independent runs. Increasing error with increased dimensionality supports Conjecture 7 .
Low-dimensional structure in real-world data plays an important role in the success of generative models, which motivates diffusion models defined on intrinsic data manifolds. Such models are driven by stochastic differential equations (SDEs) on manifolds, which raises the need for convergence theory of numerical schemes for manifold-valued SDEs. In Euclidean space, the Euler--Maruyama (EM) scheme achieves strong convergence with order 1/2, but an analogous result for manifold discretizations is less understood in general settings. In this work, we study a geometric version of the EM scheme for SDEs on Riemannian manifolds and prove strong convergence with order 1/2 under geometric and regularity conditions. As an application, we obtain a Wasserstein bound for sampling on manifolds via the geometric EM discretization of Riemannian Langevin dynamics.
Zhiyuan Zhan, Masashi Sugiyama
The University of Tokyo, Japan · RIKEN Center for AIP, Japan
We propose Riemannian Denoising Diffusion Probabilistic Models (RDDPMs) for learning distributions on submanifolds of Euclidean space that are level sets of functions, including most of the manifolds relevant to applications. Existing methods for generative modeling on manifolds rely on substantial geometric information such as geodesic curves or eigenfunctions of the Laplace-Beltrami operator and, as a result, they are limited to manifolds where such information is available. In contrast, our method, built on a projection scheme, can be applied to more general manifolds, as it only requires being able to evaluate the value and the first order derivatives of the function that defines the submanifold. We provide a theoretical analysis of our method in the continuous-time limit, which elucidates the connection between our RDDPMs and score-based generative models on manifolds. The capability of our method is demonstrated on datasets from previous studies and on new datasets sampled from two high-dimensional manifolds, i.e. SO(10) and the configuration space of molecular system alanine dipeptide with fixed dihedral angle.
Zichen Liu, Wei Zhang, Christof Schütte +1
Center for Data Science, Peking University, Beijing 100871, P.R. China · Zuse Institute Berlin, Takustrasse 7, Berlin 14195, Germany · Institute of Mathematics, Freie Universität Berlin and Zuse Institute Berlin, Takustrasse 7, Berlin 14195, Germany +1
Riemannian diffusion models generalize score-based generative modeling to manifold-supported data via stochastic diffusion equations on the manifold. However, training requires sampling from and differentiating the manifold heat kernel, which is rarely available in closed form beyond a few highly symmetric manifolds. We propose a general approach that approximates the heat kernel by directly solving the manifold heat equation with a physics-informed neural network (PINN). Given an explicit manifold specification, we choose a coordinate system, derive the corresponding heat (Fokker--Planck) equation and a short-time asymptotic approximation, and then train a PINN to learn the log heat kernel. The resulting surrogate enables both forward noising (heat-kernel sampling) and conditional-score evaluation for denoising score matching. We demonstrate the method on diverse manifolds including S2, SO(3), SPD(n), and permutation-quotiented point clouds.
Gyeonghoon Ko, Juho Lee
1Korea Advanced Institute of Science and Technology, Korea.