cs.LGNov 9, 2026

Optimal Rates for Adaptive Private kk-PCA

Authors: Johanna Düngler, Amartya Sanyal

Organizations: Department of Computer Science, University of Copenhagen

Abstract

Given nn i.i.d. random matrices Ai∈Rd×dA_i \in \mathbb{R}^{d \times d} with common expectation ΣΣ, the goal of Differentially Private Stochastic PCA is to identify a kk-dimensional subspace capturing the leading variance directions of ΣΣ, while preserving differential privacy~(DP) for each individual sample AiA_i. \citet{dungler2025iterative} introduced \kdppca, the first algorithm to simultaneously (1) achieve sample complexity n=O~(d)n=\widetilde O(d) for sub-Gaussian data, (2) adapt its privacy noise to the intrinsic randomness of the data, and (3) extend seamlessly to any target dimension k≤dk\le d. However, its sample complexity has a suboptimal dependence on kk. We propose the first algorithm that achieves optimal sample complexity in both dd and kk, while retaining properties (2) and (3) of \kdppca. Additionally, prior utility guarantees require a minimum sample size with exponential dependence on the eigengap. Our method removes this dependence. It also improves the dependence on spectral parameters, matching known lower bounds for the spiked covariance model. Unlike deflation-based approaches like \kdppca, our algorithm updates the full d×kd\times k subspace jointly rather than one eigenvector at a time. This non-deflation structure simplifies the algorithm, reduces the number of hyper-parameters, and improves computational efficiency, particularly when using block linear algebra libraries.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Jun 20, 2026

On the Curse of Dimensionality in Private Sparse Covariance Estimation and PCA

We study high-dimensional differentially private (DP) covariance estimation in the operator norm, and principal component analysis (PCA), under kk-row-column sparsity (kk-RCS) of the covariance matrix. In the non-private setting, it is known that poly(k,log⁡d)\mathsf{poly}(k, \log d) samples suffice to solve both of these problems. However, the only comparable result known under DP (Wang et al. 2021) requires Ω(d)Ω(d) samples under standard parameterizations of the problem. We investigate when this curse of dimensionality is inherent for sparse covariance estimation tasks under DP. On the upper bound front, we show that a poly(k,log⁡d)\mathsf{poly}(k, \log d) sample complexity for PCA is possible under DP, if we also posit sparsity of the leading eigenvector. We complement this result with poly(d)\mathsf{poly}(d) lower bounds under DP for both sparse covariance estimation and PCA, establishing an exponential gap between the private and non-private variants of these problems when k=polylog(d)k = \mathsf{polylog}(d). To our knowledge, no such separation has previously been demonstrated for any sparse estimation problems in private high-dimensional statistics. Our techniques are flexible enough that they imply stronger lower bounds even for the well-studied problem of standard DP PCA, without sparsity assumptions.
Feb 12, 2026

Adaptive Power Iteration Method for Differentially Private PCA

We study (ε,δ)\left(ε,δ\right)-differentially private algorithms for the problem of approximately computing the top singular vector of a matrix A∈Rn×dA\in\mathbb{R}^{n\times d} where each row of AA is a data point in Rd\mathbb{R}^{d}. Following Dwork-Talwar-Thakurta-Zhang (STOC 2014), we consider the privacy model where neighboring inputs differ by one single row. We give a novel algorithm that achieves beyond-worst-case guarantees for input matrices with low coherence, which is a structural property of matrices in many applications, including but not limited to i.i.d. data. Our algorithm contributes to the extensive literature on private power iteration methods, where we introduce a new filtering technique which adapts to this coherence parameter. Our work departs from and complements the work by Hardt-Roth (STOC 2013) which achieves beyond-worst-case guarantees for the more restrictive privacy model where neighboring inputs differ in one single entry by at most 1.
Sep 22, 2026

Gap-Free Streaming PCA Beyond Rank-One Updates: Near-Optimal Rates and Applications to Differential Privacy

Streaming principal component analysis (PCA) seeks to recover a leading spectral subspace in a single pass over a data stream. We give a new analysis of the ubiquitous Oja's algorithm [Oja82] for the most general, gap-free variant of this problem, where no eigengap assumptions are made on the underlying mean matrix, complemented by a nearly-matching lower bound. Prior works achieving near-optimal rates for streaming PCA either required gap assumptions [JJK+16, HNWW21], or were limited to rank-one updates [AZL17, Lia23]. Our proof only uses a second moment bound on the individual stochastic updates, bypassing the almost sure bounds needed by prior near-optimal analyses, and the analogous offline matrix Bernstein bound. We also extend our result to a Rayleigh quotient notion of approximate PCA, addressing an open question of [JJK+16]. As our main application, we give gap-free differentially private PCA guarantees for sub-Gaussian data, settling Conjecture 1.1 of [Bro26] up to logarithmic factors.