Machine Learning-Assisted High-Dimensional Matrix Estimation

Wan Tian; Hui Yang; Zhouhui Lian; Lingyue Zhang; Yijie Peng

cs.LGMar 30, 2026

Machine Learning-Assisted High-Dimensional Matrix Estimation

Authors: Wan Tian, Hui Yang, Zhouhui Lian, Lingyue Zhang, Yijie Peng

Organizations: Advanced Institute of Information Technology, Peking University · Wangxuan Institute of Computer Technology, Peking University, China, 100871 · Guanghua School of Management, Peking University, Beijing, China, 100871 · School of Statistics, Dongbei University of Finance and Economics, Dalian, China, 116025 ·

Abstract

Efficient estimation of high-dimensional matrices-including covariance and precision matrices-is a cornerstone of modern multivariate statistics. Most existing studies have focused primarily on the theoretical properties of the estimators (e.g., consistency and sparsity), while largely overlooking the computational challenges inherent in high-dimensional settings. Motivated by recent advances in learning-based optimization method-which integrate data-driven structures with classical optimization algorithms-we explore high-dimensional matrix estimation assisted by machine learning. Specifically, for the optimization problem of high-dimensional matrix estimation, we first present a solution procedure based on the Linearized Alternating Direction Method of Multipliers (LADMM). We then introduce learnable parameters and model the proximal operators in the iterative scheme with neural networks, thereby improving estimation accuracy and accelerating convergence. Theoretically, we first prove the convergence of LADMM, and then establish the convergence, convergence rate, and monotonicity of its reparameterized counterpart; importantly, we show that the reparameterized LADMM enjoys a faster convergence rate. Notably, the proposed reparameterization theory and methodology are applicable to the estimation of both high-dimensional covariance and precision matrices. We validate the effectiveness of our method by comparing it with several classical optimization algorithms across different structures and dimensions of high-dimensional matrices.

Figures & tables

Figure 1: Overview of the proposed LBO algorithm framework. The left panel illustrates the forward process with total

K

iterations of the algorithm, while the right panel shows the target loss function which is used to update learnable parameters. The operators

\eta_{k},\xi_{k}

are parameterized by

(w_{1})_{k},(w_{2})_{k}

, respectively.

Figure 2: Toeplitz covariance matrices under different correlation decay coefficients $\varrho$ .

Figure 3: Factor model under different numbers of low-rank components $m$ .

Figure 4: Covariance matrices under different sparsity levels $q$ .

Figure 5: Covariance matrices under different block size.

$\varrho$	Dimension	Time (s)						Frobenius norm						Nuclear norm
$\varrho$	Dimension	LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA
$\varrho=0.1$	$p=$ 1000	$6.348^{1}$	$5.131^{1}$	$1.757^{1}$	$4.813^{1}$	$7.757^{0}$	$1.790^{1}$	$2.089^{0}$	$3.194^{1}$	$5.515^{0}$	$5.594^{0}$	$5.617^{0}$	$5.552^{0}$	$6.575^{1}$	$10.00^{2}$	$1.484^{2}$	$1.507^{2}$	$1.507^{2}$	$1.493^{2}$
	$p=$ 2000	$1.162^{1}$	$3.997^{2}$	$7.069^{1}$	$1.961^{2}$	$3.013^{1}$	$7.686^{1}$	$9.074^{-2}$	$4.517^{1}$	$9.198^{0}$	$9.237^{0}$	$9.163^{0}$	$9.202^{0}$	$2.391^{0}$	$2.000^{3}$	$3.476^{2}$	$3.486^{2}$	$3.460^{2}$	$3.474^{2}$
	$p=$ 3000	$1.529^{1}$	$1.517^{3}$	$1.750^{2}$	$4.935^{2}$	$7.599^{1}$	$2.008^{2}$	$1.879^{-1}$	$5.532^{1}$	$1.275^{1}$	$1.271^{1}$	$1.281^{1}$	$1.271^{1}$	$8.285^{0}$	$3.000^{3}$	$5.868^{2}$	$5.852^{2}$	$5.904^{2}$	$5.858^{2}$
	$p=$ 4000	$2.312^{1}$	$4.029^{3}$	$3.610^{2}$	$9.581^{2}$	$1.498^{2}$	$3.892^{2}$	$1.346^{0}$	$6.388^{1}$	$1.629^{1}$	$1.630^{1}$	$1.625^{1}$	$1.620^{1}$	$8.468^{1}$	$4.000^{3}$	$8.637^{2}$	$8.633^{2}$	$8.625^{2}$	$8.592^{2}$
$\varrho=0.3$	$p=$ 1000	$5.812^{1}$	$4.954^{1}$	$1.650^{1}$	$4.619^{1}$	$7.281^{0}$	$1.763^{1}$	$2.183^{0}$	$3.461^{1}$	$7.292^{0}$	$7.346^{0}$	$7.409^{0}$	$7.332^{0}$	$6.876^{1}$	$10.00^{2}$	$1.852^{2}$	$1.863^{2}$	$1.876^{2}$	$1.867^{2}$

Table 1: Experimental results of different methods on the Toeplitz covariance structure

Factor number	Dimension	Time (s)						Frobenius norm						Nuclear norm
Factor number	Dimension	LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA
$m=3$	$p=$ 1000	$6.605^{1}$	$9.947^{1}$	$7.239^{1}$	$1.780^{2}$	$8.210^{1}$	$1.221^{3}$	$3.218^{1}$	$1.465^{2}$	$1.465^{2}$	$1.469^{2}$	$1.463^{2}$	$1.465^{2}$	$8.149^{2}$	$6.546^{2}$	$6.546^{2}$	$7.388^{2}$	$6.406^{2}$	$6.546^{2}$
	$p=$ 2000	$4.229^{1}$	$6.151^{2}$	$6.249^{2}$	$7.954^{2}$	$2.664^{2}$	$5.530^{3}$	$4.357^{1}$	$4.465^{2}$	$4.465^{2}$	$4.471^{2}$	$4.463^{2}$	$4.465^{2}$	$1.483^{2}$	$1.586^{3}$	$1.586^{3}$	$1.812^{3}$	$1.543^{3}$	$1.586^{3}$
	$p=$ 3000	$8.342^{1}$	$1.542^{3}$	$1.697^{3}$	$1.705^{3}$	$3.798^{2}$	$1.171^{4}$	$6.528^{1}$	$4.385^{2}$	$4.385^{2}$	$4.400^{2}$	$4.380^{2}$	$4.385^{2}$	$2.377^{2}$	$2.211^{3}$	$2.211^{3}$	$2.591^{3}$	$2.135^{3}$	$2.211^{3}$
	$p=$ 4000	$2.877^{1}$	$1.040^{4}$	$1.437^{4}$	$2.114^{4}$	$5.231^{3}$	$1.690^{5}$	$8.746^{1}$	$5.919^{2}$	$5.919^{2}$	$5.940^{2}$	$5.912^{2}$	$5.919^{2}$	$3.018^{2}$	$3.148^{3}$	$3.148^{3}$	$3.708^{3}$	$3.032^{3}$	$3.148^{3}$
$m=5$	$p=$ 1000	$1.189^{1}$	$2.089^{3}$	$5.206^{2}$	$3.077^{3}$	$1.079^{3}$	$1.183^{4}$	$3.193^{1}$	$1.974^{2}$	$1.974^{2}$	$1.977^{2}$	$1.973^{2}$	$1.974^{2}$	$6.097^{2}$	$1.068^{3}$	$1.068^{3}$	$1.235^{3}$	$1.057^{3}$	$1.068^{3}$

Table 2: Experimental results of different methods on the Factor covariance structure

Sparsity level	Dimension	Time (s)						Frobenius norm						Nuclear norm
Sparsity level	Dimension	LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA
$q=0.1$	$p=$ 1000	$1.791^{1}$	$1.696^{2}$	$1.110^{2}$	$2.662^{2}$	$4.011^{1}$	$2.374^{2}$	$1.704^{2}$	$1.252^{3}$	$1.252^{3}$	$1.251^{3}$	$1.252^{3}$	$1.252^{3}$	$5.369^{3}$	$3.002^{4}$	$3.002^{4}$	$3.000^{4}$	$3.002^{4}$	$3.002^{4}$
	$p=$ 2000	$4.180^{1}$	$6.248^{2}$	$3.582^{2}$	$9.583^{2}$	$1.582^{2}$	$1.082^{3}$	$1.001^{3}$	$3.582^{3}$	$3.582^{3}$	$3.577^{3}$	$3.582^{3}$	$3.582^{3}$	$4.470^{4}$	$1.138^{5}$	$1.138^{5}$	$1.133^{5}$	$1.138^{5}$	$1.138^{5}$
	$p=$ 3000	$1.709^{1}$	$1.452^{3}$	$8.957^{2}$	$2.432^{3}$	$3.849^{2}$	$3.093^{3}$	$1.259^{3}$	$6.649^{3}$	$6.649^{3}$	$6.639^{3}$	$6.649^{3}$	$6.649^{3}$	$6.878^{4}$	$2.374^{5}$	$2.374^{5}$	$2.360^{5}$	$2.374^{5}$	$2.374^{5}$
	$p=$ 4000	$2.527^{1}$	$9.748^{3}$	$5.041^{3}$	$1.859^{4}$	$3.254^{3}$	$2.711^{4}$	$7.315^{3}$	$1.024^{4}$	$1.024^{4}$	$1.023^{4}$	$1.024^{4}$	$1.024^{4}$	$4.623^{5}$	$3.869^{5}$	$3.869^{5}$	$3.843^{5}$	$3.869^{5}$	$3.869^{5}$
$q=0.3$	$p=$ 1000	$1.649^{1}$	$7.722^{2}$	$3.542^{2}$	$1.063^{3}$	$3.105^{2}$	$1.134^{3}$	$6.421^{0}$	$1.089^{3}$	$1.089^{3}$	$1.088^{3}$	$1.089^{3}$	$1.089^{3}$	$1.702^{2}$	$2.616^{4}$	$2.616^{4}$	$2.614^{4}$	$2.616^{4}$	$2.616^{4}$

Table 3: Experimental results of different methods on the sparse covariance structure

Block size	Dimension	Time (s)						Frobenius norm						Nuclear norm
Block size	Dimension	LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA						LBO/ADMM/LADMM/TOSA/PFBS/FISTA
$10$	$p=$ 1000	$6.510^{1}$	$6.161^{1}$	$3.976^{1}$	$2.071^{2}$	$3.660^{1}$	$8.718^{1}$	$9.091^{0}$	$1.398^{1}$	$1.398^{1}$	$1.398^{1}$	$1.398^{1}$	$1.398^{1}$	$2.754^{2}$	$2.898^{2}$	$2.898^{2}$	$2.898^{2}$	$2.898^{2}$	$2.898^{2}$
	$p=$ 2000	$3.115^{1}$	$5.935^{2}$	$3.615^{4}$	$1.016^{4}$	$1.122^{2}$	$2.695^{2}$	$1.747^{1}$	$2.667^{1}$	$2.667^{1}$	$2.667^{1}$	$2.667^{1}$	$2.667^{1}$	$7.655^{2}$	$8.791^{2}$	$8.791^{2}$	$8.791^{2}$	$8.791^{2}$	$8.791^{2}$
	$p=$ 3000	$7.809^{1}$	$1.501^{3}$	$1.240^{4}$	$1.927^{3}$	$2.875^{2}$	$1.030^{4}$	$2.510^{1}$	$4.230^{1}$	$4.230^{1}$	$4.201^{1}$	$4.273^{1}$	$4.230^{1}$	$1.356^{3}$	$1.771^{3}$	$1.771^{3}$	$1.750^{3}$	$1.791^{3}$	$1.771^{3}$
	$p=$ 4000	$1.110^{2}$	$2.822^{3}$	$2.972^{3}$	$4.434^{3}$	$7.198^{2}$	$1.337^{4}$	$3.961^{1}$	$6.118^{1}$	$6.118^{1}$	$6.069^{1}$	$6.291^{1}$	$6.118^{1}$	$2.480^{3}$	$2.962^{3}$	$2.962^{3}$	$2.917^{3}$	$3.051^{3}$	$2.962^{3}$
$20$	$p=$ 1000	$1.463^{2}$	$1.086^{2}$	$6.271^{1}$	$2.053^{2}$	$2.297^{1}$	$7.434^{1}$	$1.026^{1}$	$1.659^{1}$	$1.659^{1}$	$1.659^{1}$	$1.659^{1}$	$1.659^{1}$	$2.875^{2}$	$2.281^{2}$	$2.281^{2}$	$2.281^{2}$	$2.281^{2}$	$2.281^{2}$

Table 4: Experimental results of different methods on the block covariance structure

Panel of Figure 6 — Figure 6: Visualization of LBO for Toeplitz covariance matrix estimation. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $\varrho=0.1$ and $p=3000$ , while the second row corresponds to $\varrho=0.3$ and $p=2000$ .

Panel of Figure 7 — Figure 7: Visualization of LBO for factor covariance matrix estimation. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $m=3$ and $p=2000$ , while the second row corresponds to $m=3$ and $p=3000$ .

Panel of Figure 8 — Figure 8: Visualization of LBO for sparse covariance matrix estimation. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $q=0.5$ and $p=3000$ , while the second row corresponds to $q=0.5$ and $p=4000$ .

Panel of Figure 9 — Figure 9: Visualization of LBO for block covariance matrix estimation. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to block size $20$ and $p=4000$ , while the second row corresponds to block size $50$ and $p=4000$ .

Figure 10: Visualization of the precision matrices for the three structures at dimension 100.

Matrix structures	Dimension	Time (s)						Frobenius norm						Nuclear norm
Matrix structures	Dimension	LBO/ADMM/LADMM/ProxGrad/QUIC/SPG						LBO/ADMM/LADMM/ProxGrad/QUIC/SPG						LBO/ADMM/LADMM/ProxGrad/QUIC/SPG
Banded 1	$p=$ 1024	$1.704^{1}$	$1.879^{1}$	$3.668^{1}$	$1.209^{1}$	$1.241^{1}$	$8.694^{0}$	$7.887^{0}$	$9.874^{0}$	$9.872^{0}$	$9.875^{0}$	$9.209^{0}$	$9.875^{0}$	$2.207^{2}$	$2.535^{2}$	$2.535^{2}$	$2.535^{2}$	$2.449^{2}$	$2.535^{2}$
	$p=$ 2025	$4.268^{1}$	$2.045^{3}$	$4.592^{3}$	$3.423^{2}$	$4.649^{2}$	$4.244^{2}$	$1.129^{1}$	$1.491^{1}$	$1.490^{1}$	$1.491^{1}$	$1.581^{1}$	$1.491^{1}$	$4.444^{2}$	$5.533^{2}$	$5.532^{2}$	$5.533^{2}$	$5.663^{2}$	$5.533^{2}$
	$p=$ 3025	$8.675^{1}$	$1.958^{3}$	$3.164^{3}$	$9.287^{2}$	$9.456^{2}$	$8.344^{2}$	$1.412^{1}$	$1.936^{1}$	$1.935^{1}$	$1.936^{1}$	$2.006^{1}$	$1.936^{1}$	$3.452^{2}$	$8.902^{2}$	$8.907^{2}$	$8.902^{2}$	$8.876^{2}$	$8.902^{2}$
	$p=$ 3969	$1.491^{2}$	$1.968^{3}$	$4.395^{3}$	$1.974^{3}$	$1.261^{3}$	$8.140^{2}$	$1.632^{1}$	$2.324^{1}$	$2.330^{1}$	$2.324^{1}$	$2.371^{1}$	$2.324^{1}$	$4.340^{2}$	$1.232^{3}$	$1.238^{3}$	$1.232^{3}$	$1.209^{3}$	$1.232^{3}$
Banded 2	$p=$ 1024	$1.783^{1}$	$5.804^{2}$	$2.281^{3}$	$7.400^{2}$	$7.344^{2}$	$6.306^{2}$	$1.266^{1}$	$1.588^{1}$	$1.588^{1}$	$1.588^{1}$	$1.636^{1}$	$1.588^{1}$	$3.408^{2}$	$3.404^{2}$	$3.404^{2}$	$3.404^{2}$	$3.412^{2}$	$3.404^{2}$

Table 5: Experimental results of different methods on the three precision matrix structures

Panel of Figure 11 — Figure 11: Visualization of LBO for precision matrix estimation under the Banded 1 structure. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $p=3025$ , while the second row corresponds to $p=3969$ .

Panel of Figure 12 — Figure 12: Visualization of LBO for precision matrix estimation under the Banded 2 structure. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $p=3025$ , while the second row corresponds to $p=3969$ .

Panel of Figure 13 — Figure 13: Visualization of LBO for precision matrix estimation under the Grid structure. The three columns display (from left to right) the visualization of the error matrix, the trajectory of the primal–dual gap versus epochs, and the leading 1000 eigenvalues. The first row corresponds to $p=3025$ , while the second row corresponds to $p=3969$ .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Metric	Dimension	ADMM	LADMM	TOSA	PFBS	FISTA	CVXPY	MMA
Time (s)	$p=1000$	1.696 $\times 10^{2}$	1.110 $\times 10^{2}$	2.662 $\times 10^{2}$	4.011 $\times 10^{1}$	2.374 $\times 10^{2}$	2.427 $\times 10^{2}$	1.157 $\times 10^{4}$
	$p=2000$	6.248 $\times 10^{2}$	3.582 $\times 10^{2}$	9.583 $\times 10^{2}$	1.582 $\times 10^{2}$	1.082 $\times 10^{3}$	1.602 $\times 10^{3}$	3.817 $\times 10^{4}$
	$p=3000$	1.452 $\times 10^{3}$	8.957 $\times 10^{2}$	2.432 $\times 10^{3}$	3.849 $\times 10^{2}$	3.093 $\times 10^{3}$	5.038 $\times 10^{3}$	6.702 $\times 10^{5}$
Frobenius norm	$p=1000$	1.252 $\times 10^{3}$	1.252 $\times 10^{3}$	1.251 $\times 10^{3}$	1.252 $\times 10^{3}$	1.252 $\times 10^{3}$	1.252 $\times 10^{3}$	1.251 $\times 10^{3}$
	$p=2000$	3.582 $\times 10^{3}$	3.582 $\times 10^{3}$	3.577 $\times 10^{3}$	3.582 $\times 10^{3}$	3.582 $\times 10^{3}$	3.582 $\times 10^{3}$	3.600 $\times 10^{3}$
	$p=3000$	6.649 $\times 10^{3}$	6.649 $\times 10^{3}$	6.639 $\times 10^{3}$	6.649 $\times 10^{3}$	6.649 $\times 10^{3}$	6.649 $\times 10^{3}$	6.738 $\times 10^{3}$

Appendix

Table 6: Experimental results of different methods on the sparse covariance structure

Learning to Optimize

Learning to Optimize uses data to adapt algorithms, solver decisions, and solution maps for faster or better optimization.

See all 138 papers

The Regularization Parameter: Sparse Precision Matrix Estimation
Jul 7, 2026Aryan Eftekhari, Daniel Sergio Vega, Ernst-Jan Camiel Wit +1Graph Structure Learning Parameter Estimation
Institute of Computing, Faculty of Informatics, Universit`a della Svizzera italiana (USI), Lugano, Switzerland
Convexity in Disguise: A Theoretical Framework for Nonconvex Low-Rank Matrix Estimation
May 6, 2026Chengyu Cui, Gongjun XuNonconvex Optimization
Department of Statistics, University of Michigan
Non-asymptotic Estimation of Sparse Partial Correlation via Joint Partial Regression
Feb 12, 2025Samuel Erickson, Tobias RydénCovariance Estimation
Division of Decision and Control Systems KTH Royal Institute of Technology Stockholm, Sweden · Lynx Asset Management AB Stockholm, Sweden

Jul 7, 2026J/K move · Enter open · S save

The Regularization Parameter: Sparse Precision Matrix Estimation

Aryan Eftekhari, Daniel Sergio Vega, Ernst-Jan Camiel Wit, Olaf Schenk

Institute of Computing, Faculty of Informatics, Universit`a della Svizzera italiana (USI), Lugano, Switzerland

Graph Structure Learning Parameter Estimation Sparse Recovery Hyperparameter Optimization

Open paper AskPDF arXiv

Sparse precision matrix estimation provides an interpretable and computationally efficient framework for modeling conditional dependencies in high-dimensional, low-sample-size data. A recurring challenge is appropriately selecting the regularization parameter that controls estimator sparsity and strikes a balance between underfitting and overfitting. We propose a closed-form, matrix-valued regularization parameter derived from the sampling distribution of the first-order optimality conditions of the

\ell_1

-regularized Gaussian maximum-likelihood estimator. By prescribing the probability that each nonzero entry of the estimator satisfies its optimality condition under resampling, we eliminate the need for cross-validation. The resulting regularization parameter is shown to attain asymptotic scaling properties that, under standard conditions, provide consistency and sparsistency of the estimator. On synthetic Gaussian and non-Gaussian datasets, as well as real-world gene microarray and neuroimaging applications, the proposed approach achieves estimation accuracy comparable to cross-validation, delivers superior support recovery, and reduces runtime by several orders of magnitude.

Abstract

Learning to Optimize

Explore similar work

The Regularization Parameter: Sparse Precision Matrix Estimation

Convexity in Disguise: A Theoretical Framework for Nonconvex Low-Rank Matrix Estimation

Non-asymptotic Estimation of Sparse Partial Correlation via Joint Partial Regression