Machine Learning-Assisted High-Dimensional Matrix Estimation

Wan Tian; Hui Yang; Zhouhui Lian; Lingyue Zhang; Yijie Peng

cs.LGMar 30, 2026

Machine Learning-Assisted High-Dimensional Matrix Estimation

Authors: Wan Tian, Hui Yang, Zhouhui Lian, Lingyue Zhang, Yijie Peng

Organizations: Advanced Institute of Information Technology, Peking University · Wangxuan Institute of Computer Technology, Peking University, China, 100871 · Guanghua School of Management, Peking University, Beijing, China, 100871 · School of Statistics, Dongbei University of Finance and Economics, Dalian, China, 116025 ·

Abstract

Efficient estimation of high-dimensional matrices-including covariance and precision matrices-is a cornerstone of modern multivariate statistics. Most existing studies have focused primarily on the theoretical properties of the estimators (e.g., consistency and sparsity), while largely overlooking the computational challenges inherent in high-dimensional settings. Motivated by recent advances in learning-based optimization method-which integrate data-driven structures with classical optimization algorithms-we explore high-dimensional matrix estimation assisted by machine learning. Specifically, for the optimization problem of high-dimensional matrix estimation, we first present a solution procedure based on the Linearized Alternating Direction Method of Multipliers (LADMM). We then introduce learnable parameters and model the proximal operators in the iterative scheme with neural networks, thereby improving estimation accuracy and accelerating convergence. Theoretically, we first prove the convergence of LADMM, and then establish the convergence, convergence rate, and monotonicity of its reparameterized counterpart; importantly, we show that the reparameterized LADMM enjoys a faster convergence rate. Notably, the proposed reparameterization theory and methodology are applicable to the estimation of both high-dimensional covariance and precision matrices. We validate the effectiveness of our method by comparing it with several classical optimization algorithms across different structures and dimensions of high-dimensional matrices.

Figures & tables

Figure 1: Overview of the proposed LBO algorithm framework. The left panel illustrates the forward process with total

K

iterations of the algorithm, while the right panel shows the target loss function which is used to update learnable parameters. The operators

\eta_{k},\xi_{k}

are parameterized by

(w_{1})_{k},(w_{2})_{k}

, respectively.

Figure 2: Toeplitz covariance matrices under different correlation decay coefficients $\varrho$ .

Figure 3: Factor model under different numbers of low-rank components $m$ .

Figure 4: Covariance matrices under different sparsity levels $q$ .

Figure 5: Covariance matrices under different block size.

$\varrho$	Dimension	Time (s)						Frobenius norm						Nuclear norm
$\varrho$	Dimension	LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA
$\varrho=0.1$	$p=$ 1000	$6.348^{1}$	$5.131^{1}$	$1.757^{1}$	$4.813^{1}$	$7.757^{0}$	$1.790^{1}$	$2.089^{0}$	$3.194^{1}$	$5.515^{0}$	$5.594^{0}$	$5.617^{0}$	$5.552^{0}$	$6.575^{1}$	$10.00^{2}$	$1.484^{2}$	$1.507^{2}$	$1.507^{2}$	$1.493^{2}$
	$p=$ 2000	$1.162^{1}$	$3.997^{2}$	$7.069^{1}$	$1.961^{2}$	$3.013^{1}$	$7.686^{1}$	$9.074^{-2}$	$4.517^{1}$	$9.198^{0}$	$9.237^{0}$	$9.163^{0}$	$9.202^{0}$	$2.391^{0}$	$2.000^{3}$	$3.476^{2}$	$3.486^{2}$	$3.460^{2}$	$3.474^{2}$
	$p=$ 3000	$1.529^{1}$	$1.517^{3}$	$1.750^{2}$	$4.935^{2}$	$7.599^{1}$	$2.008^{2}$	$1.879^{-1}$	$5.532^{1}$	$1.275^{1}$	$1.271^{1}$	$1.281^{1}$	$1.271^{1}$	$8.285^{0}$	$3.000^{3}$	$5.868^{2}$	$5.852^{2}$	$5.904^{2}$	$5.858^{2}$
	$p=$ 4000	$2.312^{1}$	$4.029^{3}$	$3.610^{2}$	$9.581^{2}$	$1.498^{2}$	$3.892^{2}$	$1.346^{0}$	$6.388^{1}$	$1.629^{1}$	$1.630^{1}$	$1.625^{1}$	$1.620^{1}$	$8.468^{1}$	$4.000^{3}$	$8.637^{2}$	$8.633^{2}$	$8.625^{2}$	$8.592^{2}$
$\varrho=0.3$	$p=$ 1000	$5.812^{1}$	$4.954^{1}$	$1.650^{1}$	$4.619^{1}$	$7.281^{0}$	$1.763^{1}$	$2.183^{0}$	$3.461^{1}$	$7.292^{0}$	$7.346^{0}$	$7.409^{0}$	$7.332^{0}$	$6.876^{1}$	$10.00^{2}$	$1.852^{2}$	$1.863^{2}$	$1.876^{2}$	$1.867^{2}$

Table 1: Experimental results of different methods on the Toeplitz covariance structure

Factor number	Dimension	Time (s)						Frobenius norm						Nuclear norm
Factor number	Dimension	LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA
$m=3$	$p=$ 1000	$6.605^{1}$	$9.947^{1}$	$7.239^{1}$	$1.780^{2}$	$8.210^{1}$	$1.221^{3}$	$3.218^{1}$	$1.465^{2}$	$1.465^{2}$	$1.469^{2}$	$1.463^{2}$	$1.465^{2}$	$8.149^{2}$	$6.546^{2}$	$6.546^{2}$	$7.388^{2}$	$6.406^{2}$	$6.546^{2}$
	$p=$ 2000	$4.229^{1}$	$6.151^{2}$	$6.249^{2}$	$7.954^{2}$	$2.664^{2}$	$5.530^{3}$	$4.357^{1}$	$4.465^{2}$	$4.465^{2}$	$4.471^{2}$	$4.463^{2}$	$4.465^{2}$	$1.483^{2}$	$1.586^{3}$	$1.586^{3}$	$1.812^{3}$	$1.543^{3}$	$1.586^{3}$
	$p=$ 3000	$8.342^{1}$	$1.542^{3}$	$1.697^{3}$	$1.705^{3}$	$3.798^{2}$	$1.171^{4}$	$6.528^{1}$	$4.385^{2}$	$4.385^{2}$	$4.400^{2}$	$4.380^{2}$	$4.385^{2}$	$2.377^{2}$	$2.211^{3}$	$2.211^{3}$	$2.591^{3}$	$2.135^{3}$	$2.211^{3}$
	$p=$ 4000	$2.877^{1}$	$1.040^{4}$	$1.437^{4}$	$2.114^{4}$	$5.231^{3}$	$1.690^{5}$	$8.746^{1}$	$5.919^{2}$	$5.919^{2}$	$5.940^{2}$	$5.912^{2}$	$5.919^{2}$	$3.018^{2}$	$3.148^{3}$	$3.148^{3}$	$3.708^{3}$	$3.032^{3}$	$3.148^{3}$
$m=5$	$p=$ 1000	$1.189^{1}$	$2.089^{3}$	$5.206^{2}$	$3.077^{3}$	$1.079^{3}$	$1.183^{4}$	$3.193^{1}$	$1.974^{2}$	$1.974^{2}$	$1.977^{2}$	$1.973^{2}$	$1.974^{2}$	$6.097^{2}$	$1.068^{3}$	$1.068^{3}$	$1.235^{3}$	$1.057^{3}$	$1.068^{3}$

Table 2: Experimental results of different methods on the Factor covariance structure

Sparsity level	Dimension	Time (s)						Frobenius norm						Nuclear norm
Sparsity level	Dimension	LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA
$q=0.1$	$p=$ 1000	$1.791^{1}$	$1.696^{2}$	$1.110^{2}$	$2.662^{2}$	$4.011^{1}$	$2.374^{2}$	$1.704^{2}$	$1.252^{3}$	$1.252^{3}$	$1.251^{3}$	$1.252^{3}$	$1.252^{3}$	$5.369^{3}$	$3.002^{4}$	$3.002^{4}$	$3.000^{4}$	$3.002^{4}$	$3.002^{4}$
	$p=$ 2000	$4.180^{1}$	$6.248^{2}$	$3.582^{2}$	$9.583^{2}$	$1.582^{2}$	$1.082^{3}$	$1.001^{3}$	$3.582^{3}$	$3.582^{3}$	$3.577^{3}$	$3.582^{3}$	$3.582^{3}$	$4.470^{4}$	$1.138^{5}$	$1.138^{5}$	$1.133^{5}$	$1.138^{5}$	$1.138^{5}$
	$p=$ 3000	$1.709^{1}$	$1.452^{3}$	$8.957^{2}$	$2.432^{3}$	$3.849^{2}$	$3.093^{3}$	$1.259^{3}$	$6.649^{3}$	$6.649^{3}$	$6.639^{3}$	$6.649^{3}$	$6.649^{3}$	$6.878^{4}$	$2.374^{5}$	$2.374^{5}$	$2.360^{5}$	$2.374^{5}$	$2.374^{5}$
	$p=$ 4000	$2.527^{1}$	$9.748^{3}$	$5.041^{3}$	$1.859^{4}$	$3.254^{3}$	$2.711^{4}$	$7.315^{3}$	$1.024^{4}$	$1.024^{4}$	$1.023^{4}$	$1.024^{4}$	$1.024^{4}$	$4.623^{5}$	$3.869^{5}$	$3.869^{5}$	$3.843^{5}$	$3.869^{5}$	$3.869^{5}$
$q=0.3$	$p=$ 1000	$1.649^{1}$	$7.722^{2}$	$3.542^{2}$	$1.063^{3}$	$3.105^{2}$	$1.134^{3}$	$6.421^{0}$	$1.089^{3}$	$1.089^{3}$	$1.088^{3}$	$1.089^{3}$	$1.089^{3}$	$1.702^{2}$	$2.616^{4}$	$2.616^{4}$	$2.614^{4}$	$2.616^{4}$	$2.616^{4}$

Table 3: Experimental results of different methods on the sparse covariance structure

Block size	Dimension	Time (s)						Frobenius norm						Nuclear norm
Block size	Dimension	LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA
$10$	$p=$ 1000	$6.510^{1}$	$6.161^{1}$	$3.976^{1}$	$2.071^{2}$	$3.660^{1}$	$8.718^{1}$	$9.091^{0}$	$1.398^{1}$	$1.398^{1}$	$1.398^{1}$	$1.398^{1}$	$1.398^{1}$	$2.754^{2}$	$2.898^{2}$	$2.898^{2}$	$2.898^{2}$	$2.898^{2}$	$2.898^{2}$
	$p=$ 2000	$3.115^{1}$	$5.935^{2}$	$3.615^{4}$	$1.016^{4}$	$1.122^{2}$	$2.695^{2}$	$1.747^{1}$	$2.667^{1}$	$2.667^{1}$	$2.667^{1}$	$2.667^{1}$	$2.667^{1}$	$7.655^{2}$	$8.791^{2}$	$8.791^{2}$	$8.791^{2}$	$8.791^{2}$	$8.791^{2}$
	$p=$ 3000	$7.809^{1}$	$1.501^{3}$	$1.240^{4}$	$1.927^{3}$	$2.875^{2}$	$1.030^{4}$	$2.510^{1}$	$4.230^{1}$	$4.230^{1}$	$4.201^{1}$	$4.273^{1}$	$4.230^{1}$	$1.356^{3}$	$1.771^{3}$	$1.771^{3}$	$1.750^{3}$	$1.791^{3}$	$1.771^{3}$
	$p=$ 4000	$1.110^{2}$	$2.822^{3}$	$2.972^{3}$	$4.434^{3}$	$7.198^{2}$	$1.337^{4}$	$3.961^{1}$	$6.118^{1}$	$6.118^{1}$	$6.069^{1}$	$6.291^{1}$	$6.118^{1}$	$2.480^{3}$	$2.962^{3}$	$2.962^{3}$	$2.917^{3}$	$3.051^{3}$	$2.962^{3}$
$20$	$p=$ 1000	$1.463^{2}$	$1.086^{2}$	$6.271^{1}$	$2.053^{2}$	$2.297^{1}$	$7.434^{1}$	$1.026^{1}$	$1.659^{1}$	$1.659^{1}$	$1.659^{1}$	$1.659^{1}$	$1.659^{1}$	$2.875^{2}$	$2.281^{2}$	$2.281^{2}$	$2.281^{2}$	$2.281^{2}$	$2.281^{2}$

Table 4: Experimental results of different methods on the block covariance structure

Panel of Figure 6 — Figure 6: Visualization of LBO for Toeplitz covariance matrix estimation. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $\varrho=0.1$ and $p=3000$ , while the second row corresponds to $\varrho=0.3$ and $p=2000$ .

Panel of Figure 7 — Figure 7: Visualization of LBO for factor covariance matrix estimation. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $m=3$ and $p=2000$ , while the second row corresponds to $m=3$ and $p=3000$ .

Panel of Figure 8 — Figure 8: Visualization of LBO for sparse covariance matrix estimation. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $q=0.5$ and $p=3000$ , while the second row corresponds to $q=0.5$ and $p=4000$ .

Panel of Figure 9 — Figure 9: Visualization of LBO for block covariance matrix estimation. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to block size $20$ and $p=4000$ , while the second row corresponds to block size $50$ and $p=4000$ .

Figure 10: Visualization of the precision matrices for the three structures at dimension 100.

Matrix structures	Dimension	Time (s)						Frobenius norm						Nuclear norm
Matrix structures	Dimension	LBO/ADMM/LADMM/ProxGrad/QUIC/SPG						LBO/ADMM/LADMM/ProxGrad/QUIC/SPG						LBO/ADMM/LADMM/ProxGrad/QUIC/SPG
Banded 1	$p=$ 1024	$1.704^{1}$	$1.879^{1}$	$3.668^{1}$	$1.209^{1}$	$1.241^{1}$	$8.694^{0}$	$7.887^{0}$	$9.874^{0}$	$9.872^{0}$	$9.875^{0}$	$9.209^{0}$	$9.875^{0}$	$2.207^{2}$	$2.535^{2}$	$2.535^{2}$	$2.535^{2}$	$2.449^{2}$	$2.535^{2}$
	$p=$ 2025	$4.268^{1}$	$2.045^{3}$	$4.592^{3}$	$3.423^{2}$	$4.649^{2}$	$4.244^{2}$	$1.129^{1}$	$1.491^{1}$	$1.490^{1}$	$1.491^{1}$	$1.581^{1}$	$1.491^{1}$	$4.444^{2}$	$5.533^{2}$	$5.532^{2}$	$5.533^{2}$	$5.663^{2}$	$5.533^{2}$
	$p=$ 3025	$8.675^{1}$	$1.958^{3}$	$3.164^{3}$	$9.287^{2}$	$9.456^{2}$	$8.344^{2}$	$1.412^{1}$	$1.936^{1}$	$1.935^{1}$	$1.936^{1}$	$2.006^{1}$	$1.936^{1}$	$3.452^{2}$	$8.902^{2}$	$8.907^{2}$	$8.902^{2}$	$8.876^{2}$	$8.902^{2}$
	$p=$ 3969	$1.491^{2}$	$1.968^{3}$	$4.395^{3}$	$1.974^{3}$	$1.261^{3}$	$8.140^{2}$	$1.632^{1}$	$2.324^{1}$	$2.330^{1}$	$2.324^{1}$	$2.371^{1}$	$2.324^{1}$	$4.340^{2}$	$1.232^{3}$	$1.238^{3}$	$1.232^{3}$	$1.209^{3}$	$1.232^{3}$
Banded 2	$p=$ 1024	$1.783^{1}$	$5.804^{2}$	$2.281^{3}$	$7.400^{2}$	$7.344^{2}$	$6.306^{2}$	$1.266^{1}$	$1.588^{1}$	$1.588^{1}$	$1.588^{1}$	$1.636^{1}$	$1.588^{1}$	$3.408^{2}$	$3.404^{2}$	$3.404^{2}$	$3.404^{2}$	$3.412^{2}$	$3.404^{2}$

Table 5: Experimental results of different methods on the three precision matrix structures

Panel of Figure 11 — Figure 11: Visualization of LBO for precision matrix estimation under the Banded 1 structure. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $p=3025$ , while the second row corresponds to $p=3969$ .

Panel of Figure 12 — Figure 12: Visualization of LBO for precision matrix estimation under the Banded 2 structure. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $p=3025$ , while the second row corresponds to $p=3969$ .

Panel of Figure 13 — Figure 13: Visualization of LBO for precision matrix estimation under the Grid structure. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $p=3025$ , while the second row corresponds to $p=3969$ .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Metric	Dimension	ADMM	LADMM	TOSA	PFBS	FISTA	CVXPY	MMA
Time (s)	$p=1000$	1.696 $\times 10^{2}$	1.110 $\times 10^{2}$	2.662 $\times 10^{2}$	4.011 $\times 10^{1}$	2.374 $\times 10^{2}$	2.427 $\times 10^{2}$	1.157 $\times 10^{4}$
	$p=2000$	6.248 $\times 10^{2}$	3.582 $\times 10^{2}$	9.583 $\times 10^{2}$	1.582 $\times 10^{2}$	1.082 $\times 10^{3}$	1.602 $\times 10^{3}$	3.817 $\times 10^{4}$
	$p=3000$	1.452 $\times 10^{3}$	8.957 $\times 10^{2}$	2.432 $\times 10^{3}$	3.849 $\times 10^{2}$	3.093 $\times 10^{3}$	5.038 $\times 10^{3}$	6.702 $\times 10^{5}$
Frobenius norm	$p=1000$	1.252 $\times 10^{3}$	1.252 $\times 10^{3}$	1.251 $\times 10^{3}$	1.252 $\times 10^{3}$	1.252 $\times 10^{3}$	1.252 $\times 10^{3}$	1.251 $\times 10^{3}$
	$p=2000$	3.582 $\times 10^{3}$	3.582 $\times 10^{3}$	3.577 $\times 10^{3}$	3.582 $\times 10^{3}$	3.582 $\times 10^{3}$	3.582 $\times 10^{3}$	3.600 $\times 10^{3}$
	$p=3000$	6.649 $\times 10^{3}$	6.649 $\times 10^{3}$	6.639 $\times 10^{3}$	6.649 $\times 10^{3}$	6.649 $\times 10^{3}$	6.649 $\times 10^{3}$	6.738 $\times 10^{3}$

Appendix

Table 6: Experimental results of different methods on the sparse covariance structure

Learning to Optimize

Learning to Optimize uses data to adapt algorithms, solver decisions, and solution maps for faster or better optimization.

See all 138 papers

Jul 7, 2026

The Regularization Parameter: Sparse Precision Matrix Estimation

Sparse precision matrix estimation provides an interpretable and computationally efficient framework for modeling conditional dependencies in high-dimensional, low-sample-size data. A recurring challenge is appropriately selecting the regularization parameter that controls estimator sparsity and strikes a balance between underfitting and overfitting. We propose a closed-form, matrix-valued regularization parameter derived from the sampling distribution of the first-order optimality conditions of the

\ell_1

-regularized Gaussian maximum-likelihood estimator. By prescribing the probability that each nonzero entry of the estimator satisfies its optimality condition under resampling, we eliminate the need for cross-validation. The resulting regularization parameter is shown to attain asymptotic scaling properties that, under standard conditions, provide consistency and sparsistency of the estimator. On synthetic Gaussian and non-Gaussian datasets, as well as real-world gene microarray and neuroimaging applications, the proposed approach achieves estimation accuracy comparable to cross-validation, delivers superior support recovery, and reduces runtime by several orders of magnitude.

Aryan Eftekhari, Daniel Sergio Vega, Ernst-Jan Camiel Wit +1

Institute of Computing, Faculty of Informatics, Universit`a della Svizzera italiana (USI), Lugano, Switzerland

Graph Structure Learning Parameter Estimation Sparse Recovery Hyperparameter Optimization

Ask PDF

May 6, 2026

Convexity in Disguise: A Theoretical Framework for Nonconvex Low-Rank Matrix Estimation

Nonconvex methods have emerged as a dominant approach for low-rank matrix estimation, a problem that arises widely in machine learning and AI for learning and representing high-dimensional data. Existing analyses for these methods often require additional regularization to mitigate nonconvexity, even though such regularization is often unnecessary in practice. Moreover, most analyses rely on problem-specific arguments that are difficult to generalize to more complex settings. In this paper, we develop a theoretical framework for studying nonconvex procedures across a broad class of low-rank matrix estimation problems. Rather than focusing on a specific model, we reveal a fundamental mechanism that explains why nonconvex procedures can behave well in low-rank estimation. Our key device is a {\it benign regularizer} that does not alter the original update rule, but yields an equivalent locally strongly convex formulation of the algorithm. This perspective uncovers a disguised convexity inherent in the nonconvex procedure and provides a new route to theoretical guarantees for nonconvex low-rank matrix estimation.

Chengyu Cui, Gongjun Xu

Department of Statistics, University of Michigan

Nonconvex Optimization

Ask PDF

Feb 12, 2025

Non-asymptotic Estimation of Sparse Partial Correlation via Joint Partial Regression

Partial correlation analysis is widely used in modern multivariate analysis. However, prior methods lack non-asymptotic guarantees and have slow asymptotic estimation rates. We present a method for simultaneously estimating sparse high-dimensional partial correlation matrices and inverse covariance with non-asymptotic rates. The method is a two-stage estimation which exploits the connection between the inverse covariance matrix and linear regression estimation method while enforcing positive semi-definiteness simultaneously. An efficient proximal splitting algorithm for numerically computing the estimate is also derived, and we provide a fast implementation of the algorithm. The effectiveness of the proposed method is demonstrated on both synthetic and real-world data.

Samuel Erickson, Tobias Rydén

Division of Decision and Control Systems KTH Royal Institute of Technology Stockholm, Sweden · Lynx Asset Management AB Stockholm, Sweden

Covariance Estimation

Ask PDF

Abstract

Learning to Optimize

Explore similar work

The Regularization Parameter: Sparse Precision Matrix Estimation

Convexity in Disguise: A Theoretical Framework for Nonconvex Low-Rank Matrix Estimation

Non-asymptotic Estimation of Sparse Partial Correlation via Joint Partial Regression