We study transfer learning for low-rank plus sparse matrix denoising when both ambient and latent dimensions grow. A data-rich source task provides subspace information for a higher-dimensional, data-scarce target with a small rank increment and localized sparse edits. We develop an anchored alternating-projection estimator that preserves transferred subspaces while learning target-specific coefficients, additional latent directions, and sparse modifications. We establish deterministic error bounds that separate target noise, representation growth, and source-estimation error, identifying improved rates through small structural increments and reduced noise within inherited subspaces. We further extend it to approximate anchoring quantifies robustness to subspace mismatch. We establish application-specific guarantees for Markov transition matrix estimation from single trajectories with dependent noise and for structured covariance estimation, with the latter analysis provided in the appendix. Experiments demonstrate consistent improvements over non-transfer baselines on synthetic data and an approximately 45% reduction in principal-angle error on a NYC taxi transition-estimation task with expanding state spaces.
Figures & tables
Figure 1: Synthetic covariance estimation under enlarged dimension. Curves are means over 50 trials with ±1 standard-error shading. Transfer AltProj consistently outperforms Non-Transfer and single-task PCA across all n2 , with the largest gains in the low-sample regime, consistent with error scaling in δr,2=1 rather than r2=4 .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Notation
Meaning
Value
p1
Source task ambient dimension
10
r1
Rank of the source low-rank component L(1)
3
n1
Number of source samples used to form Σ(1)
500
p2
Target task ambient dimension after enlargement
50
δr,2
Rank increment from source to target (innovation rank)
1
r2=r1+δr,2
Rank of the target low-rank component L(2)
4
Appendix
Table 1: Key dimensions, ranks, sample sizes, and convergence parameters for the synthetic enlarged-dimension experiment.
n2
rotated dirs k
switch strength ρ⋆
Transfer sin(Θ) at ρ⋆
Non-Transfer sin(Θ) at ρ⋆
50
1
>1.00
—
—
50
2
>1.00
—
—
50
3
>1.00
—
—
50
4
0.80
2.022
1.981
100
1
>1.00
—
—
100
2
0.50
1.261
1.098
Appendix
Table 2: Regime-switch table for the rotation robustness study. For each combination of target sample size n2 and number of rotated source directions k , the table reports the smallest rotation strength ρ⋆ at which the non-transfer estimator first achieves lower subspace error than transfer. Entries “ >1.00 ” indicate no switch was observed even at full 90∘ rotation; corresponding error values are omitted. Source sample size n1=200 , source rank r1=4 .
∥F⋆−FLS∥F
0.0037
∥F⋆−FL∥F
0.0349
∥P⋆−PLS∥1,1/p
0.0075
∥P⋆−PL∥1,1/p
0.0363
Appendix
Table 3: Approximation error of the low-rank-plus-sparse model ( LS ) and the low-rank model ( L ) on the NYC taxi transition matrix.
Figure 2: Spectral estimator (blue) vs. alternating minimization (black) on semi-synthetic NYC taxi data. Left: Frobenius error of the estimated frequency matrix. Right: per-row ℓ1 error of the estimated transition matrix. The alternating procedure improves over the tested range, while the spectral curve plateaus.
Estimator
Mean ∥sin(Θ)∥F ( ± 1 sd)
Transfer AltProj (ours)
0.915±0.07
Non-transfer structured estimator
1.679±0.09
Truncated SVD
1.703±0.09
Appendix
Table 4: Subspace recovery error on the NYC taxi transfer task ( p1=200 , p2=255 ), averaged over 20 trials. Transfer AltProj reduces the principal-angle error by approximately 45% relative to both non-transfer baselines.
Multi-task learning is effective for related applications, but its performance can deteriorate when the target sample size is small. Transfer learning can borrow strength from related studies; yet, many existing methods rely on restrictive bounded-difference assumptions between the source and target models. We propose SMART, a spectral transfer method for multi-task linear regression that instead assumes spectral similarity: the target left and right singular subspaces lie within the corresponding source subspaces and are sparsely aligned with the source singular bases. Such an assumption is natural when studies share latent structures and enables transfer beyond the bounded-difference settings. SMART estimates the target coefficient matrix through structured regularization that incorporates spectral information from a source study. Importantly, it requires only a fitted source model rather than the raw source data, making it useful when data sharing is limited. Although the optimization problem is nonconvex, we develop a practical ADMM-based algorithm. We establish general, non-asymptotic error bounds and a minimax lower bound in the noiseless-source regime. Under additional regularity conditions, these results yield near-minimax Frobenius error rates up to logarithmic factors. Simulations confirm improved estimation accuracy and robustness to negative transfer, and analysis of multi-modal single-cell data demonstrates better predictive performance. The Python implementation of SMART, along with the code to reproduce all experiments in this paper, is publicly available at https://github.com/boxinz17/smart.
Boxin Zhao, Mladen Kolar, Jinchi Lv
University of Chicago · University of Southern California
We study a distributional generalization of the matrix completion problem in which each entry of the target matrix is a probability distribution rather than a scalar. In this setting, only a subset of matrix entries is observed, and even for observed entries, the underlying distributions are not directly accessible; instead, we observe finitely many samples drawn from them. To represent distributional entries, we employ kernel mean embeddings and introduce a notion of Tucker rank for distribution-valued matrices to capture their low-rank structure. The infinite-dimensional nature of kernel embeddings poses significant methodological challenges. To address this, we introduce functional unfolding operators that link the proposed distributional low-rank structure to the classical Tucker rank for finite-dimensional tensors. Based on this framework, we propose a novel estimator for distributional matrix completion. We establish non-asymptotic error bounds that characterize the statistical performance of the estimator. Extensive experiments on synthetic data and a real-world application demonstrate the effectiveness of the proposed method.
Jiayi Wang, Raymond K. W. Wong
University of Texas at Dallas · Texas A&M University
We investigate when a sparse nonnegative matrix can be recovered from a real-valued matrix of much lower rank by zeroing out its negative elements. The potential for such decompositions suggests a mathematical connection between sparsity and rank; we analyze a number of sparse matrices with this latent low-rank structure and use them to illustrate the geometric origins of this connection. Previous algorithms have discovered these decompositions via an alternating minimization over the factors of a low-rank matrix, but to do so, they have also needed to compute and store another matrix, neither sparse nor low-rank, that is the size of their product. We develop a stochastic, alternating least-squares algorithm that operates on smaller blocks of this dense matrix and scales as a result to much larger problems. We also show how to further accelerate this algorithm with sparse optimizations and customized CUDA kernels. As one example, we use the algorithm to analyze the sparse matrix of synaptic weights for the recently published Drosphilia connectome. The nonzero elements of this matrix, with 139,255 rows and columns, record the number of synapses between cells in the nervous system of a female fruit fly. Despite a slowly decaying spectrum of singular values, this matrix exhibits a latent low-rank structure that is predictive of cell categories across multiple levels of specificity.
Lawrence K. Saul, Ningyuan Huang, Dennis Bollweg +2
Center for Computational Mathematics, Flatiron Institute, New York, NY 10010 · Scientific Computing Core, Flatiron Institute, New York, NY 10010 · Courant Institute, NYU, New York, NY 10012