PRIM-cipal components analysis

Tianhao Liu; Daniel Andrés Díaz-Pachón; J. Sunil Rao

stat.MLApr 16, 2026

PRIM-cipal components analysis

Authors: Tianhao Liu, Daniel Andrés Díaz-Pachón, J. Sunil Rao

Organizations: Division of Biostatistics, University of Miami, Miami, FL, 33136 USA · University of Minnesota, Minneapolis, MN, 55414 USA

Abstract

Supervised No Free Lunch Theorems (NFLTs) are well studied, yet unsupervised NFLTs remain underexplored. For elliptical distributions, we prove that there exist two equally optimal, scientifically meaningful bump-hunting strategies that are exact opposites, with no universal winner. Specifically, peeling $k$ orthogonal dimensions from $\mathbb{R}^d$ ( $d \ge k$ ), retaining an inter-quantile region of probability $1-α$ per peeled dimension, maximizes total variance and Frobenius norm when the $k$ smallest principal components (called pettiest components) are selected, and minimizes them when the selected dimensions are the $k$ leading principal components. These optima inspire PRIM-based bump-hunting algorithms either by minimizing variance or by minimizing volume, thereby motivating an NFLT. We test our results on the Fashion-MNIST database, showing that peeling the largest principal components captures multiplicity, while peeling the smallest principal components isolates popular styles.

Explore similar work

SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA
Sep 22, 2026Irina-Beatrice Haas, Maike Meier, Yuji Nakatsukasa +1Principal Component Analysis
Bandit PCA with Minimax Optimal Regret
Jul 12, 2026Moïse Blanchard, Dmitrii Ostrovskii, Aadirupa SahaPrincipal Component Analysis Regret
Mean-Shift PCA by Knockoff Mean
May 25, 2026Mengda Li, Zeng Li, Jianfeng YaoPrincipal Component Analysis Largest Eigenvalue

Sep 22, 2026 · stat.MLJ/K move · Enter open · S save

SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA

Irina-Beatrice Haas, Maike Meier, Yuji Nakatsukasa, Taejun Park

Principal component analysis (PCA) is a fundamental tool to reduce the dimensionality of the data in many applications. PCA finds a few signal directions that contain most of the variability of the data by computing the eigenvectors of the sample covariance matrix. In this work, we focus on the spiked covariance model, in which the data vectors are defined by a few orthogonal signals plus an isotropic Gaussian noise, and our goal is to estimate one or more of the leading signals. Our main theoretical finding is that the subspace spanned by several leading eigenvectors of the sample covariance matrix contains significant information about the desired signals long before the individual eigenvectors converge to the population principal components. To prove this, we derive a posteriori bounds for the angle between the subspace spanned by the desired population signals and the subspace obtained from the sample using perturbation theory for singular vectors. This leads to a new algorithm, SuperPCA (SUbsPace subsamplER PCA), which capitalizes on an approximate eigenspace of the sample covariance matrix to find the leading signals far more efficiently and accurately than classical PCA in the high-dimensional, multi-signal setting. SuperPCA exploits only a small number of subsampled coordinates of the data, which can lead to tremendous savings in data acquisition cost, especially when the signals are approximately sparse. For the same number of measurements, SuperPCA can offer a factor

10

improvement in accuracy compared to the classical PCA method.

Principal Component Analysis

Open paper Ask about it PDF

Abstract

Explore similar work

SuperPCA: subspace analysis and an efficient algorithm for high-dimensional PCA

Bandit PCA with Minimax Optimal Regret

Mean-Shift PCA by Knockoff Mean