An Interpretable and Stable Framework for Sparse Principal Component Analysis

Ying Hu; Hu Yang

stat.MLMar 14, 2026

An Interpretable and Stable Framework for Sparse Principal Component Analysis

Authors: Ying Hu, Hu Yang

Organizations: College of Mathematics and Statistics, Chongqing University, Chongqing, 401331, China

Abstract

Sparse principal component analysis (SPCA) addresses the poor interpretability and variable redundancy often encountered by principal component analysis (PCA) in high-dimensional data. However, SPCA typically imposes uniform penalties on variables and does not account for differences in variable importance, which may lead to unstable performance in highly noisy or structurally complex settings. We propose SP-SPCA, a method that introduces a single equilibrium parameter into the regularization framework to adaptively adjust variable penalties. This modification of the L2 penalty provides flexible control over the trade-off between sparsity and explained variance while maintaining computational efficiency. Simulation studies show that the proposed method consistently outperforms standard sparse principal component methods in identifying sparse loading patterns, filtering noise variables, and preserving cumulative variance, especially in high-dimensional and noisy settings. Empirical applications to crime and financial market data further demonstrate its practical utility. In real data analyses, the method selects fewer but more relevant variables, thereby reducing model complexity while maintaining explanatory power. Overall, the proposed approach offers a robust and efficient alternative for sparse modeling in complex high-dimensional data, with clear advantages in stability, feature selection, and interpretability

Figures & tables

X^{*}=\begin{pmatrix}X\\ Z\end{pmatrix},\quad y^{*}=\begin{pmatrix}y\\ 0\end{pmatrix}

Algorithm 1 Coordinate Descent Algorithm for SPPCSO

	PCA		SPCA		SP-SPCA
	PC1	PC2	PC1	PC2	PC1	PC2
$X_{1}$	-0.315	0.400	0	0.590	0	0.491
$X_{2}$	-0.315	0.349	0	0.206	0	0.501
$X_{3}$	-0.315	0.404	0	0.566	0	0.502
$X_{4}$	-0.315	0.396	0	0.538	0	0.498
$X_{5}$	0.317	0.277	0.388	0	0.499	0

Table 1: Loading matrix and explained variance for PCA, SPCA, and SP-SPCA on a low-dimensional dataset.

Parameter	Symbol
Sample size	$n$
Total number of variables	$p$
Number of latent factors	$d$
Variables per factor	$s$
Number of mixed variables	$m$

Table 2: Key parameter settings used in the simulation studies

	PC1	PC2	PC3	PC4	PC5
$X_{1}\textendash X_{50}$	Nonzero	0	0	0	0
$X_{51}\textendash X_{100}$	0	Nonzero	0	0	0
$X_{101}\textendash X_{150}$	0	0	0	Nonzero	0
$X_{151}\textendash X_{200}$	0	0	0	0	Nonzero
$X_{201}\textendash X_{250}$	0	0	Nonzero	0	0
$X_{251}\textendash X_{300}$	0	0	0	0	0

Table 3: Case 1: Ideal Loadings Matrix Results.

$p$	150	300	450	600	750	900
SPCA	57%	56.5%	29.2%	20%	16.3%	14.4%
SP-SPCA	68.8%	65.7%	60%	56.9%	55.9%	54.6%

Table 4: Case 2: Comparison of cumulative variance explained (%) under different dimensions

p

$m$	10	50	100	150
SPCA	61%	56.5%	38.1%	27.2%
SP-SPCA	71.2%	65.7%	47.3%	35.3%

Table 5: Case 3: Comparison of cumulative variance explained (%) under different numbers of mixed variables

m

with

n=100

p=300

$d$	2	5	10
SPCA	22.4%	56.5%	44%
SP-SPCA	60%	65.7%	61.1%

Table 6: Case 4: Comparison of cumulative variance explained (%) under different numbers of latent factors

d

with

n=100

p=300

Panel of Figure 3 — Figure 3: Comparison of numerical simulation results for high-dimensional data.

Name	Samples	Features	Type	Description	Source
Crime	1994	99	Regular	Social data	UCI
S&P 500	150	503	High-dimensional	Financial data	Kaggle

Table 7: Description of datasets used for empirical analysis

Panel of Figure 4 — Figure 4: Cumulative explanatory rate curves and gravel plots for the Crime data.

Variance	10%	20%	30%	40%	50%	60%	70%	80%
SPCA	8	27	45	123	220	432	666	1485
SP-SPCA	14	24	40	59	104	347	644	1485

Table 8: Comparison of the total number of nonzero loadings under different cumulative variance explained levels for SPCA and SP-SPCA on the Crime dataset

Panel of Figure 6 — Figure 6: Cumulative explanatory rate curves and gravel plots for the S&P500 data.

Variance	10%	20%	30%	40%	50%	60%	70%	80%
SPCA	16	75	209	2187	6319	9852	12856	25150
SP-SPCA	16	52	152	2002	5721	9055	12103	25150

Table 9: Comparison of the total number of nonzero loadings under different cumulative variance explained levels for SPCA and SP-SPCA

Panel of Figure 7 — Figure 7: Comparison of results for the S&P500 data .

Principal Component Analysis

Principal component analysis (PCA) finds orthogonal directions of greatest variance for data compression, visualization, denoising, and feature discovery.

See all 67 papers

SPPCSO: Adaptive Penalized Estimation Method for High-Dimensional Correlated Data
Mar 6, 2026Ying Hu, Hu YangFeature Selection
College of Mathematics and Statistics, Chongqing University, Chongqing, 401331, China.
A Robust Optimization Approach to Sparse Principal Component Analysis
Jun 2, 2026David Vävinggren, Francis Bach, André M. H. Teixeira +2Principal Component Analysis Robust Optimization
Uppsala University, Sweden · PSL Research University / INRIA, France · Science for Life Laboratory, Sweden
Combinatorial Sparse PCA Beyond the Spiked Identity Model
Mar 3, 2026Syamantak Kumar, Purnamrita Sarkar, Kevin Tian +1Principal Component Analysis
UT Austin · UW Madison

Mar 6, 2026J/K move · Enter open · S save

SPPCSO: Adaptive Penalized Estimation Method for High-Dimensional Correlated Data

Ying Hu, Hu Yang

College of Mathematics and Statistics, Chongqing University, Chongqing, 401331, China.

Feature Selection

Open paper AskPDF arXiv

With the rise of high-dimensional correlated data, multicollinearity poses a significant challenge to model stability, often leading to unstable estimation and reduced predictive accuracy. This work proposes the Single-Parametric Principal Component Selection Operator (SPPCSO), an innovative penalized estimation method that integrates single-parametric principal component regression and

L_{1}

regularization to adaptively adjust the shrinkage factor by incorporating principal component information. This approach achieves a balance between variable selection and coefficient estimation, ensuring model stability and robust estimation even in high-dimensional, high-noise environments. The primary contribution lies in addressing the instability of traditional variable selection methods when applied to high-noise, high-dimensional correlated data. Theoretically, our method exhibits selection consistency and achieves a smaller estimation error bound compared to traditional penalized estimation approaches. Extensive numerical experiments demonstrate that SPPCSO not only delivers stable and reliable estimation in high-noise settings but also accurately distinguishes signal variables from noise variables in group-effect structured data with highly correlated noise variables, effectively eliminating redundant variables and achieving more stable variable selection. Furthermore, SPPCSO successfully identifies disease-associated genes in gene expression data analysis, showcasing strong practical value. The results indicate that SPPCSO serves as an ideal tool for high-dimensional variable selection, offering an efficient and interpretable solution for modeling correlated data.

Abstract

Principal Component Analysis

Explore similar work

SPPCSO: Adaptive Penalized Estimation Method for High-Dimensional Correlated Data

A Robust Optimization Approach to Sparse Principal Component Analysis

Combinatorial Sparse PCA Beyond the Spiked Identity Model