Non-asymptotic Estimation of Sparse Partial Correlation via Joint Partial Regression

Samuel Erickson; Tobias Rydén

stat.MLFeb 12, 2025

Non-asymptotic Estimation of Sparse Partial Correlation via Joint Partial Regression

Authors: Samuel Erickson, Tobias Rydén

Organizations: Division of Decision and Control Systems KTH Royal Institute of Technology Stockholm, Sweden · Lynx Asset Management AB Stockholm, Sweden

Abstract

Partial correlation analysis is widely used in modern multivariate analysis. However, prior methods lack non-asymptotic guarantees and have slow asymptotic estimation rates. We present a method for simultaneously estimating sparse high-dimensional partial correlation matrices and inverse covariance with non-asymptotic rates. The method is a two-stage estimation which exploits the connection between the inverse covariance matrix and linear regression estimation method while enforcing positive semi-definiteness simultaneously. An efficient proximal splitting algorithm for numerically computing the estimate is also derived, and we provide a fast implementation of the algorithm. The effectiveness of the proposed method is demonstrated on both synthetic and real-world data.

Figures & tables

\hat{\theta}_{j}=\operatornamewithlimits{argmin}_{\theta\in{\mathbf{R}}^{p-1}}\left\{\frac{1}{2n}\|X_{j}-X_{\setminus j}\theta\|_{2}^{2}+\lambda\|\theta\|_{1}\right\},

Algorithm 3.1 Joint partial regression

Panel of Figure 1 — Figure 1: Average Frobenius-error $\|\widehat{\Omega}-\Omega^{\star}\|_{\mathrm{F}}$ versus number of features with $\pm 2\times\text{SE}$ bands.

Panel of Figure 2 — Figure 2: Average Frobenius-error $\|\widehat{Q}-Q^{\star}\|_{\mathrm{F}}$ versus number of features with $\pm 2\times\text{SE}$ bands.

Panel of Figure 3 — Figure 3: Average $F_{1}$ -scores versus number of features with $\pm 2\times\text{SE}$ bands.

Sector	Color	Sector	Color
Consumer Discretionary		Technology
Utilities		Finance
Real Estate		Industrials
Miscellaneous		Energy
Health Care		Consumer Staples
Basic Materials		Telecommunications

Table 1: Sector to color mapping for the Nasdaq data set.

Panel of Figure 4 — Figure 4: Estimated stock market network at an approximate edge density of 1%.

Covariance Estimation

Covariance estimation develops methods for learning dependence and uncertainty structure from noisy, incomplete, or high-dimensional data.

See all 60 papers

The Regularization Parameter: Sparse Precision Matrix Estimation
Jul 7, 2026Aryan Eftekhari, Daniel Sergio Vega, Ernst-Jan Camiel Wit +1Graph Structure Learning Parameter Estimation
Institute of Computing, Faculty of Informatics, Universit`a della Svizzera italiana (USI), Lugano, Switzerland
Machine Learning-Assisted High-Dimensional Matrix Estimation
Mar 30, 2026Wan Tian, Hui Yang, Zhouhui Lian +2Learning to Optimize Alternating Direction Method of Multipliers
Advanced Institute of Information Technology, Peking University · Wangxuan Institute of Computer Technology, Peking University, China, 100871 · Guanghua School of Management, Peking University, Beijing, China, 100871 +3
On the Curse of Dimensionality in Private Sparse Covariance Estimation and PCA
Jun 20, 2026Syamantak Kumar, Shourya Pandey, Purnamrita Sarkar +1Principal Component Analysis Minimax Estimation
University of Texas at Austin

Jul 7, 2026J/K move · Enter open · S save

The Regularization Parameter: Sparse Precision Matrix Estimation

Aryan Eftekhari, Daniel Sergio Vega, Ernst-Jan Camiel Wit, Olaf Schenk

Institute of Computing, Faculty of Informatics, Universit`a della Svizzera italiana (USI), Lugano, Switzerland

Graph Structure Learning Parameter Estimation Sparse Recovery Hyperparameter Optimization

Open paper AskPDF arXiv

Sparse precision matrix estimation provides an interpretable and computationally efficient framework for modeling conditional dependencies in high-dimensional, low-sample-size data. A recurring challenge is appropriately selecting the regularization parameter that controls estimator sparsity and strikes a balance between underfitting and overfitting. We propose a closed-form, matrix-valued regularization parameter derived from the sampling distribution of the first-order optimality conditions of the

\ell_1

-regularized Gaussian maximum-likelihood estimator. By prescribing the probability that each nonzero entry of the estimator satisfies its optimality condition under resampling, we eliminate the need for cross-validation. The resulting regularization parameter is shown to attain asymptotic scaling properties that, under standard conditions, provide consistency and sparsistency of the estimator. On synthetic Gaussian and non-Gaussian datasets, as well as real-world gene microarray and neuroimaging applications, the proposed approach achieves estimation accuracy comparable to cross-validation, delivers superior support recovery, and reduces runtime by several orders of magnitude.

Abstract

Covariance Estimation

Explore similar work

The Regularization Parameter: Sparse Precision Matrix Estimation

Machine Learning-Assisted High-Dimensional Matrix Estimation

On the Curse of Dimensionality in Private Sparse Covariance Estimation and PCA