stat.MLNov 15, 2026

The Manifold-Index Model: provably and efficiently avoiding the statistical and computational curse of dimensionality

Authors: Yantao Wu, Mauro Maggioni

Organizations: Department of Mathematics, Johns Hopkins University, Baltimore, MD 21218, USA · Department of Mathematics and Department of Applied Mathematics & Statistics, Johns Hopkins University, Baltimore, MD 21218, USA

Abstract

We study vector-valued regression with predictors filling a tube around an unknown mm-dimensional manifold M\mathcal{M} in RD\mathbb{R}^D. In the Manifold-Index Model, the mean response is F=f∘ΠMF=f\circΠ_{\mathcal{M}}, where ΠMΠ_{\mathcal{M}} is the nearest-point projection onto M\mathcal{M} and f:M→Rpf:\mathcal{M}\to\mathbb{R}^p is a possibly non-injective link function. Multiscale ANisotropic lOcal Regression (MANOR) estimates tangent planes by multiscale PCA of predictors with similar responses, then refines them by pooling outer products of local slopes. The neighborhoods narrow tangentially while retaining complete normal fibers. We assume a Lipschitz link differential and bounded response noise. For p≥mp\ge m, under our geometric and sampling conditions, nn independent observations give mean tangent-space error of order n−2/(m+4)n^{-2/(m+4)} and integrated squared prediction error at the mm-dimensional nonparametric rate n−4/(m+4)n^{-4/(m+4)}, up to logarithmic factors. In a scaling regime with fixed intrinsic dimension, model and scale parameters, and ratio of manifold volume to the mm-th power of its reach, the required sample size grows polynomially in D+pD+p, and computation is polynomial in nn, DD and pp. Numerical experiments on curves with non-monotone links and surfaces with vector-valued responses compare observed convergence rates with the theoretical bounds; a Langevin dynamics example tests prediction of hitting probabilities under an approximate manifold-index model.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Sep 18, 2026

Riemannian Simultaneous Inference for Tangent Vector Field Regression

We consider nonparametric tangent vector field regression on a Riemannian manifold without boundary. Because responses at different points lie in different tangent spaces, the proposed kernel estimator first parallel transports nearby responses to the target tangent space and then forms a volume-corrected local average. We first derive its uniform second-order bias, finite-bandwidth covariance, and stochastic rate. For simultaneous inference, the tangent norm is written as a supremum over the unit tangent bundle. Exact covariance whitening gives a unit-variance Gaussian field whose correlation length is of order hh along the base manifold and of order one along the fibre. Its local covariance geometry leads to a Gumbel limit with an explicit intrinsic constant. Combining this limit with Gaussian approximation and cross-fitted covariance estimation yields a feasible simultaneous confidence tube for the regression field. We further discuss improved finite-sample inference with bandwidth selection and high-order bias corrections. Simulations on various manifolds support the proposed inference procedure. A randomized reconstruction of global wind data illustrates how the tube's cross-sections describe spatially varying uncertainty.
May 14, 2026

Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index models

We study a prototypical situation when a learned predictor can discover useful low-dimensional structure in data, while using fewer samples than are needed for accurate prediction. Specifically, we consider the problem of recovering a multi-index polynomial f∗(x)=h(Ux)f^*(x)=h(Ux), with U∈Rr×dU\in\mathbb{R}^{r\times d} and r≪dr\ll d, from finitely many data/label pairs. Importantly, the target function depends on input xx only through the projection onto an unknown rr-dimensional central subspace. The algorithm we analyze is appealingly simple: fit kernel ridge regression (KRR) to the data and compute the Average Gradient Outer Product (AGOP) from the fitted predictor. Our main results show that under reasonable assumptions the top rr-dimensional eigenspace of AGOP provably recovers the central subspace, even in regimes when the prediction error remains large. Specifically, if the target function f∗f^* has degree p∗p^*, it is known that n≍dp∗n\asymp d^{p^*} samples are necessary for KRR to achieve accurate prediction. In contrast, we show that if a low degree pp component of f∗f^* already carries all relevant directions for prediction, subspace recovery occurs in the much lower sample regime n≍dp+δn\asymp d^{p+δ} for any δ∈(0,1)δ\in(0,1). Our results thus demonstrate a separation between prediction and representation, and provide an explanation for why iterative kernel methods such as Recursive Feature Machines (RFM) can be sample-efficient in practice.
May 15, 2026

Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifolds

Score-based generative models are trained in high-dimensional ambient spaces, yet many data distributions are supported on low-dimensional nonlinear structures. We prove that, for compact dd-dimensional smooth manifolds M⊂[0,1]D\mathcal{M} \subset [0,1]^D with d>2d > 2 and ββ-Hölder densities strictly positive on M\mathcal{M}, a variance-preserving SGM estimator attains the intrinsic Wasserstein--1 sample exponent O~(DOβ(d)n−(β+1)/(d+2β))\tilde{\mathcal{O}}(D^{\mathcal{O}_β(d)}n^{-(β+1)/(d+2β)}), up to logarithmic factors and explicit geometry and density factors. The full nonasymptotic bound explicitly isolates the finite-order geometry envelope, Hölder radius, density lower bound, ambient dependence, and finite-order correction terms. The analysis separates score approximation into a large-noise tangent-cell regime and a small-noise projection-centered, de-Gaussianized Laplace regime. The key technical ingredient is a ReLU implementation of nearest-projection coordinates via finite intrinsic anchors and Gauss--Newton iterations, rather than approximating the manifold projection as a black-box high-dimensional smooth map. Consequently, for families with polynomially controlled geometry and density lower bounds, the constructed score-network parameters have polynomial ambient dependence.