Transfer Denoising of Low-Rank Plus Sparse Matrices under Ambient and Rank Growth

Elynn Chen; Jinhang Chai; Xuyuan Liu; Yujun Yan

cs.LGJan 29, 2026

Transfer Denoising of Low-Rank Plus Sparse Matrices under Ambient and Rank Growth

Authors: Elynn Chen, Jinhang Chai, Xuyuan Liu, Yujun Yan

Organizations: Department of Technology, Operations, & Statistics, New York University · Department of Computer Science, Dartmouth College

Abstract

We study transfer learning for low-rank plus sparse matrix denoising when both ambient and latent dimensions grow. A data-rich source task provides subspace information for a higher-dimensional, data-scarce target with a small rank increment and localized sparse edits. We develop an anchored alternating-projection estimator that preserves transferred subspaces while learning target-specific coefficients, additional latent directions, and sparse modifications. We establish deterministic error bounds that separate target noise, representation growth, and source-estimation error, identifying improved rates through small structural increments and reduced noise within inherited subspaces. We further extend it to approximate anchoring quantifies robustness to subspace mismatch. We establish application-specific guarantees for Markov transition matrix estimation from single trajectories with dependent noise and for structured covariance estimation, with the latter analysis provided in the appendix. Experiments demonstrate consistent improvements over non-transfer baselines on synthetic data and an approximately 45% reduction in principal-angle error on a NYC taxi transition-estimation task with expanding state spaces.

Figures & tables

Figure 1: Synthetic covariance estimation under enlarged dimension. Curves are means over 50 trials with

\pm 1

standard-error shading. Transfer AltProj consistently outperforms Non-Transfer and single-task PCA across all

n_{2}

, with the largest gains in the low-sample regime, consistent with error scaling in

\delta_{r,2}=1

rather than

r_{2}=4

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Notation	Meaning	Value
$p_{1}$	Source task ambient dimension	$10$
$r_{1}$	Rank of the source low-rank component $L^{(1)}$	$3$
$n_{1}$	Number of source samples used to form $\widehat{\Sigma}^{(1)}$	$500$
$p_{2}$	Target task ambient dimension after enlargement	$50$
$\delta_{r,2}$	Rank increment from source to target (innovation rank)	$1$
$r_{2}=r_{1}+\delta_{r,2}$	Rank of the target low-rank component $L^{(2)}$	$4$

Appendix

Table 1: Key dimensions, ranks, sample sizes, and convergence parameters for the synthetic enlarged-dimension experiment.

$n_{2}$	rotated dirs $k$	switch strength $\rho^{\star}$	Transfer $\sin(\Theta)$ at $\rho^{\star}$	Non-Transfer $\sin(\Theta)$ at $\rho^{\star}$
$50$	$1$	$>1.00$	—	—
$50$	$2$	$>1.00$	—	—
$50$	$3$	$>1.00$	—	—
$50$	$4$	$0.80$	$2.022$	$1.981$
$100$	$1$	$>1.00$	—	—
$100$	$2$	$0.50$	$1.261$	$1.098$

Appendix

Table 2: Regime-switch table for the rotation robustness study. For each combination of target sample size

n_{2}

and number of rotated source directions

k

, the table reports the smallest rotation strength

\rho^{\star}

at which the non-transfer estimator first achieves lower subspace error than transfer. Entries “

>1.00

” indicate no switch was observed even at full

90^{\circ}

rotation; corresponding error values are omitted. Source sample size

n_{1}=200

, source rank

r_{1}=4

$\\|F^{\star}-F_{LS}\\|_{F}$	$0.0037$	$\\|F^{\star}-F_{L}\\|_{F}$	$0.0349$
$\\|P^{\star}-P_{LS}\\|_{1,1}/p$	$0.0075$	$\\|P^{\star}-P_{L}\\|_{1,1}/p$	$0.0363$

Appendix

Table 3: Approximation error of the low-rank-plus-sparse model (

LS

) and the low-rank model (

L

) on the NYC taxi transition matrix.

Figure 2: Spectral estimator (blue) vs. alternating minimization (black) on semi-synthetic NYC taxi data. Left: Frobenius error of the estimated frequency matrix. Right: per-row

\ell_{1}

error of the estimated transition matrix. The alternating procedure improves over the tested range, while the spectral curve plateaus.

Estimator	Mean $\\|\sin(\Theta)\\|_{F}$ ( $\pm$ 1 sd)
Transfer AltProj (ours)	$\mathbf{0.915\pm 0.07}$
Non-transfer structured estimator	$1.679\pm 0.09$
Truncated SVD	$1.703\pm 0.09$

Appendix

Table 4: Subspace recovery error on the NYC taxi transfer task (

p_{1}=200

p_{2}=255

), averaged over

20

trials. Transfer AltProj reduces the principal-angle error by approximately

45\%

relative to both non-transfer baselines.

Transfer Learning

Transfer learning studies how models reuse knowledge across tasks, domains, or modalities when target data or computation are limited.

See all 416 papers

SMART: A Spectral Transfer Approach to Multi-Task Learning
Apr 22, 2026Boxin Zhao, Mladen Kolar, Jinchi LvMulti-Task Learning Transfer Learning
University of Chicago · University of Southern California
Low-rank Distributional Matrix Completion
Jun 2, 2026Jiayi Wang, Raymond K. W. WongLow-Rank Approximation Matrix Completion
University of Texas at Dallas · Texas A&M University
Subzero matrix completion for sparse data analysis: large-scale learning of latent low-rank structure
Aug 21, 2026Lawrence K. Saul, Ningyuan Huang, Dennis Bollweg +2Low-Rank Matrix Decomposition Matrix Completion
Center for Computational Mathematics, Flatiron Institute, New York, NY 10010 · Scientific Computing Core, Flatiron Institute, New York, NY 10010 · Courant Institute, NYU, New York, NY 10012

Apr 22, 2026J/K move · Enter open · S save

SMART: A Spectral Transfer Approach to Multi-Task Learning

Boxin Zhao, Mladen Kolar, Jinchi Lv

University of Chicago · University of Southern California

Multi-Task Learning Transfer Learning Statistical Learning Theory

Open paper AskPDF arXiv

Multi-task learning is effective for related applications, but its performance can deteriorate when the target sample size is small. Transfer learning can borrow strength from related studies; yet, many existing methods rely on restrictive bounded-difference assumptions between the source and target models. We propose SMART, a spectral transfer method for multi-task linear regression that instead assumes spectral similarity: the target left and right singular subspaces lie within the corresponding source subspaces and are sparsely aligned with the source singular bases. Such an assumption is natural when studies share latent structures and enables transfer beyond the bounded-difference settings. SMART estimates the target coefficient matrix through structured regularization that incorporates spectral information from a source study. Importantly, it requires only a fitted source model rather than the raw source data, making it useful when data sharing is limited. Although the optimization problem is nonconvex, we develop a practical ADMM-based algorithm. We establish general, non-asymptotic error bounds and a minimax lower bound in the noiseless-source regime. Under additional regularity conditions, these results yield near-minimax Frobenius error rates up to logarithmic factors. Simulations confirm improved estimation accuracy and robustness to negative transfer, and analysis of multi-modal single-cell data demonstrates better predictive performance. The Python implementation of SMART, along with the code to reproduce all experiments in this paper, is publicly available at https://github.com/boxinz17/smart.

Abstract

Transfer Learning

Explore similar work

SMART: A Spectral Transfer Approach to Multi-Task Learning

Low-rank Distributional Matrix Completion

Subzero matrix completion for sparse data analysis: large-scale learning of latent low-rank structure