cs.DSFeb 12, 2026

Adaptive Power Iteration Method for Differentially Private PCA

Authors: Ta Duy Nguyen, Alina Ene, Huy Le Nguyen

Organizations: Department of Computer Science, Boston University · Khoury College of Computer and Information Science, Northeastern University

Abstract

We study (ε,δ)\left(ε,δ\right)-differentially private algorithms for the problem of approximately computing the top singular vector of a matrix A∈Rn×dA\in\mathbb{R}^{n\times d} where each row of AA is a data point in Rd\mathbb{R}^{d}. Following Dwork-Talwar-Thakurta-Zhang (STOC 2014), we consider the privacy model where neighboring inputs differ by one single row. We give a novel algorithm that achieves beyond-worst-case guarantees for input matrices with low coherence, which is a structural property of matrices in many applications, including but not limited to i.i.d. data. Our algorithm contributes to the extensive literature on private power iteration methods, where we introduce a new filtering technique which adapts to this coherence parameter. Our work departs from and complements the work by Hardt-Roth (STOC 2013) which achieves beyond-worst-case guarantees for the more restrictive privacy model where neighboring inputs differ in one single entry by at most 1.

Figures & tables

Explore similar work

Sep 25, 2026

Gap-free Differentially Private PCA for Gaussian Data

We give a gap-free (ε,δ)(ε,δ)-differentially private algorithm for the principal component analysis (PCA) problem with Gaussian data. The algorithm is based on a private variant of the power iteration method, and it is computationally efficient.
Nov 9, 2026

Optimal Rates for Adaptive Private kk-PCA

Given nn i.i.d. random matrices Ai∈Rd×dA_i \in \mathbb{R}^{d \times d} with common expectation ΣΣ, the goal of Differentially Private Stochastic PCA is to identify a kk-dimensional subspace capturing the leading variance directions of ΣΣ, while preserving differential privacy~(DP) for each individual sample AiA_i. \citet{dungler2025iterative} introduced \kdppca, the first algorithm to simultaneously (1) achieve sample complexity n=O~(d)n=\widetilde O(d) for sub-Gaussian data, (2) adapt its privacy noise to the intrinsic randomness of the data, and (3) extend seamlessly to any target dimension k≤dk\le d. However, its sample complexity has a suboptimal dependence on kk. We propose the first algorithm that achieves optimal sample complexity in both dd and kk, while retaining properties (2) and (3) of \kdppca. Additionally, prior utility guarantees require a minimum sample size with exponential dependence on the eigengap. Our method removes this dependence. It also improves the dependence on spectral parameters, matching known lower bounds for the spiked covariance model. Unlike deflation-based approaches like \kdppca, our algorithm updates the full d×kd\times k subspace jointly rather than one eigenvector at a time. This non-deflation structure simplifies the algorithm, reduces the number of hyper-parameters, and improves computational efficiency, particularly when using block linear algebra libraries.
Jun 20, 2026

On the Curse of Dimensionality in Private Sparse Covariance Estimation and PCA

We study high-dimensional differentially private (DP) covariance estimation in the operator norm, and principal component analysis (PCA), under kk-row-column sparsity (kk-RCS) of the covariance matrix. In the non-private setting, it is known that poly(k,log⁡d)\mathsf{poly}(k, \log d) samples suffice to solve both of these problems. However, the only comparable result known under DP (Wang et al. 2021) requires Ω(d)Ω(d) samples under standard parameterizations of the problem. We investigate when this curse of dimensionality is inherent for sparse covariance estimation tasks under DP. On the upper bound front, we show that a poly(k,log⁡d)\mathsf{poly}(k, \log d) sample complexity for PCA is possible under DP, if we also posit sparsity of the leading eigenvector. We complement this result with poly(d)\mathsf{poly}(d) lower bounds under DP for both sparse covariance estimation and PCA, establishing an exponential gap between the private and non-private variants of these problems when k=polylog(d)k = \mathsf{polylog}(d). To our knowledge, no such separation has previously been demonstrated for any sparse estimation problems in private high-dimensional statistics. Our techniques are flexible enough that they imply stronger lower bounds even for the well-studied problem of standard DP PCA, without sparsity assumptions.