Self-Normalized Inference from Riemannian Stochastic Gradient Paths
Organizations: Division of Mathematical Sciences, Nanyang Technological University. · School of Statistics and Data Science, Shanghai University of Finance and Economics. · York University.
Abstract
Riemannian stochastic gradient paths estimate constrained parameters, but confidence regions usually require estimating the Hessian and gradient-noise covariance. We develop self-normalized inference from one path by expressing retained iterates in a common random tangent space. Their average and centered partial sums share an unknown covariance factor that cancels in a Brownian bridge pivot. Under suitable regularity conditions, a joint expansion of the averaged estimator and integrated path controls nonlinear coordinate errors using conditional second moments. The procedure uses fixed-dimensional memory and extends to nondegenerate smooth functionals and alternative local coordinates. Numerical studies show competitive coverage relative to plug-in Wald inference while avoiding Hessian and residual-density estimation.
Figures & tables
| Setting | Self-normalized | Plug-in | Signed-H | Oracle | |
|---|---|---|---|---|---|
| Multi-frequency, cap 100 | 4000 | 0.928 | 0.846 | 0.870 | 0.818 |
| Multi-frequency, cap 100 | 12000 | 0.948 | 0.900 | 0.900 | 0.858 |
| Multi-frequency, cap 200 | 4000 | 0.916 | 0.798 | 0.884 | 0.834 |
| Multi-frequency, cap 200 | 12000 | 0.950 | 0.880 | 0.884 | 0.856 |
| Phase retrieval, mixture | 4000 | 0.934 | 0.944 | – | 0.960 |
| Phase retrieval, mixture | 12000 | 0.944 | 0.938 | – | 0.956 |
| Setting | Self-normalized | Short-batch | Population | Plug-in | MSE | ||
|---|---|---|---|---|---|---|---|
| Sphere | 5000 | 2 | 0.922 | 0.646 | 0.940 | 0.936 | |
| Sphere | 10000 | 2 | 0.926 | 0.658 | – | – | |
| SPD | 5000 | 3 | 0.922 | 0.626 | 0.932 | 0.930 | |
| SPD | 10000 | 3 | 0.920 | 0.602 | – | – |
| Variable | Squared loading | 95% interval |
|---|---|---|
| Density | 0.217 | |
| Alcohol | 0.199 | |
| Total sulfur dioxide | 0.177 | |
| Residual sugar | 0.165 | |
| Free sulfur dioxide | 0.102 | |
| Chlorides | 0.056 |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| 1 | 2 | 3 | 5 | 6 | 10 | 15 | 19 | |
|---|---|---|---|---|---|---|---|---|
| 46.14 | 100.79 | 172.91 | 366.61 | 477.04 | 1017.28 | 1972.35 | 2917.21 |
| Setting | Self-normalized | Plug-in | Oracle | |
|---|---|---|---|---|
| Single frequency | 4000 | 0.938 | 0.940 | 0.942 |
| Single frequency | 12000 | 0.926 | 0.956 | 0.956 |
| Intrinsic circle, | 4000 | 0.894 | 0.946 | 0.942 |
| Intrinsic circle, | 12000 | 0.924 | 0.952 | 0.954 |
| Intrinsic circle, | 4000 | 0.926 | 0.934 | 0.956 |
| Intrinsic circle, | 12000 | 0.920 | 0.938 | 0.970 |
Confidence Region Estimation
Confidence region estimation builds sets around unknown parameters or predictions that quantify uncertainty while controlling coverage.
See all 44 papers