Riemannian Newton Methods with Statistical Inference

We develop an online multiplier bootstrap for manifold-valued parameters estimated by averaging the iterates of Riemannian stochastic approximation. The method runs weighted iterations alongside the original algorithm, with each path using the same observations while updating its own state and intrinsic average. The weighted averages provide uncertainty assessments without a plug-in asymptotic covariance estimator. Under local stability and regularity conditions, we establish bootstrap consistency and show that a fixed number of paths suffices for asymptotically valid Student intervals and intrinsic confidence regions. The theory accommodates samplewise nonsmooth scores and identifies a critical oracle-bias regime in which the estimator's limiting distribution shifts, but the centered bootstrap distribution does not. Applications include spherical means, positive-definite matrix means and medians, and principal-subspace estimation. Simulations show coverage close to nominal in regular settings, and an analysis of North Atlantic storm locations illustrates descriptive online inference for a spherical center.

We study online inference for averaged Riemannian stochastic gradient descent (RSGD) when the unknown parameter lies on a manifold. Although the averaged RSGD iterate is asymptotically normal, practical inference requires estimating the unknown covariance operator in its limiting distribution. Existing online batch-means methods cannot be applied directly because a general Riemannian manifold lacks an addition operation, and the tangent space containing the Riemannian gradient varies across iterations. Through vector transport, we construct overlapping and non-overlapping covariance estimators that adaptively combine information from successive batches while accounting for the changing tangent spaces. The proposed estimators are online, require neither Riemannian Hessian computations nor a prespecified terminal sample size, and attain the same convergence rates as their Euclidean counterparts (Zhu et al., 2023). Together with asymptotic normality, these estimators enable online statistical inference for RSGD on manifolds. We illustrate the proposed framework through a detailed study of streaming principal component analysis on Grassmann manifolds.

We develop inference for manifold-valued population minimizers when each observation belongs to a different participant and only locally private messages reach the analyst. The method releases randomized tangent gradients and combines them through Riemannian stochastic approximation and Polyak-Ruppert averaging. Directly inserting a private data surrogate into a nonlinear loss can shift its population target, whereas conditional centring of the released gradient preserves the first-order equation. We introduce symmetric-pair regression (SPR) to estimate the asymptotic variance from the same private messages used for point estimation, without holding out participants or requesting a second release. We prove the central limit theorem and consistency of the fully transcript-based sandwich covariance and intrinsic Wald region under local differential privacy. Simulations across various statistical problems and manifolds support the predicted decrease in estimation error and near-nominal coverage under moderate privacy. An application to NHANES anthropometric data illustrates private estimation of a leading body-size direction and its uncertainty.