Rao-Blackwellized Score Matching on Manifolds

Divit Rawal

stat.MLMay 25, 2026

Rao-Blackwellized Score Matching on Manifolds

Authors: Divit Rawal

Abstract

We study denoising score matching (DSM) when the latent distribution is supported on a smooth embedded manifold $M \subset \mathbb{R}^D$ . Under ambient Gaussian corruption, the tangent denoising target contains a singular normal-fiber noise channel whose variance diverges as $d/σ^2$ as $σ\to 0^+$ . We show that conditioning on the nearest-point projection $π(X)$ canonically removes this singularity: the resulting conditional expectation is the unique $L^2$ -optimal Rao-Blackwellized predictor of the tangent DSM target among all estimators depending only on the projected observation $π(X)$ . We then compute the small-noise expansion of this canonical target and show that it equals the intrinsic Riemannian score up to an explicit order- $σ^2$ correction that decomposes into an intrinsic Tweedie term and an extrinsic curvature term involving the Weingarten and Ricci operators. In the flat case, the construction reduces exactly to ordinary lower-dimensional Gaussian DSM, while on $S^d$ the extrinsic correction simplifies to the scalar factor $(1-d/2)\nabla_M \log q$ ; this extrinsic $σ^2$ correction cancels identically on $S^2$ , though the intrinsic Tweedie term remains.

Explore similar work

May 21, 2026stat.ML

Diffusion-based Denoising Beats Vanilla Score Matching in Parameter Estimation: A Theoretical Explanation

Score matching is an alternative to maximum likelihood estimation when the normalizing constant is unknown or too costly to evaluate. However, vanilla score matching has shown to be inefficient relative to maximum likelihood estimation for multimodal distributions with well-separated modes, which are commonly encountered in practical applications. We compare a novel diffusion-based denoising score matching estimator (DDSME) to the vanilla score matching estimator (SME) in this scenario. In particular, we prove statistical guarantees for both estimators, showing that the error bound for the vanilla SME worsens when the separation between the modes increases, which can be avoided in case of the DDSME with suitable hyperparameter tuning. This provides a novel theoretical explanation for the superior behavior of diffusion-based score matching over the vanilla version.

Benedikt Lütke Schwienhorst, Nadja Klein, Johannes Lederer

Score-Based Diffusion Model Estimators Score Function Theoretical Foundations

Ask PDF

May 7, 2025cs.LG

Riemannian Denoising Diffusion Probabilistic Models

We propose Riemannian Denoising Diffusion Probabilistic Models (RDDPMs) for learning distributions on submanifolds of Euclidean space that are level sets of functions, including most of the manifolds relevant to applications. Existing methods for generative modeling on manifolds rely on substantial geometric information such as geodesic curves or eigenfunctions of the Laplace-Beltrami operator and, as a result, they are limited to manifolds where such information is available. In contrast, our method, built on a projection scheme, can be applied to more general manifolds, as it only requires being able to evaluate the value and the first order derivatives of the function that defines the submanifold. We provide a theoretical analysis of our method in the continuous-time limit, which elucidates the connection between our RDDPMs and score-based generative models on manifolds. The capability of our method is demonstrated on datasets from previous studies and on new datasets sampled from two high-dimensional manifolds, i.e.

\mathrm{SO}(10)

and the configuration space of molecular system alanine dipeptide with fixed dihedral angle.

Zichen Liu, Wei Zhang, Christof Schütte +1

Score-Based Diffusion Model Generative Models Graph Laplacians

Ask PDF

May 15, 2026cs.LG

Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifolds

Score-based generative models are trained in high-dimensional ambient spaces, yet many data distributions are supported on low-dimensional nonlinear structures. We prove that, for compact

d

-dimensional smooth manifolds

\mathcal{M} \subset [0,1]^D

with

d > 2

and

β

-Hölder densities strictly positive on

\mathcal{M}

, a variance-preserving SGM estimator attains the intrinsic Wasserstein--1 sample exponent

\tilde{\mathcal{O}}(D^{\mathcal{O}_β(d)}n^{-(β+1)/(d+2β)})

, up to logarithmic factors and explicit geometry and density factors. The full nonasymptotic bound explicitly isolates the finite-order geometry envelope, Hölder radius, density lower bound, ambient dependence, and finite-order correction terms. The analysis separates score approximation into a large-noise tangent-cell regime and a small-noise projection-centered, de-Gaussianized Laplace regime. The key technical ingredient is a ReLU implementation of nearest-projection coordinates via finite intrinsic anchors and Gauss--Newton iterations, rather than approximating the manifold projection as a black-box high-dimensional smooth map. Consequently, for families with polynomially controlled geometry and density lower bounds, the constructed score-network parameters have polynomial ambient dependence.

Guoji Fu, Taiji Suzuki, Wee Sun Lee +1

Wasserstein Distance Statistical Manifold Manifolds Low-Dimensional Structure Generative Models Polynomials

Ask PDF