cs.LGNov 9, 2026

Optimal Rates for Adaptive Private kk-PCA

Authors: Johanna Düngler, Amartya Sanyal

Organizations: Department of Computer Science, University of Copenhagen

Abstract

Given nn i.i.d. random matrices Ai∈Rd×dA_i \in \mathbb{R}^{d \times d} with common expectation ΣΣ, the goal of Differentially Private Stochastic PCA is to identify a kk-dimensional subspace capturing the leading variance directions of ΣΣ, while preserving differential privacy~(DP) for each individual sample AiA_i. \citet{dungler2025iterative} introduced \kdppca, the first algorithm to simultaneously (1) achieve sample complexity n=O~(d)n=\widetilde O(d) for sub-Gaussian data, (2) adapt its privacy noise to the intrinsic randomness of the data, and (3) extend seamlessly to any target dimension k≤dk\le d. However, its sample complexity has a suboptimal dependence on kk. We propose the first algorithm that achieves optimal sample complexity in both dd and kk, while retaining properties (2) and (3) of \kdppca. Additionally, prior utility guarantees require a minimum sample size with exponential dependence on the eigengap. Our method removes this dependence. It also improves the dependence on spectral parameters, matching known lower bounds for the spiked covariance model. Unlike deflation-based approaches like \kdppca, our algorithm updates the full d×kd\times k subspace jointly rather than one eigenvector at a time. This non-deflation structure simplifies the algorithm, reduces the number of hyper-parameters, and improves computational efficiency, particularly when using block linear algebra libraries.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. On the Curse of Dimensionality in Private Sparse Covariance Estimation and PCA

    Jun 20, 2026Syamantak Kumar, Shourya Pandey, Purnamrita Sarkar +1Principal Component AnalysisMinimax Estimation

  2. Adaptive Power Iteration Method for Differentially Private PCA

    Feb 12, 2026Ta Duy Nguyen, Alina Ene, Huy Le NguyenPrincipal Component AnalysisDifferential Privacy