stat.MLJun 25, 2025

Global Convergence of Iteratively Reweighted Least Squares for Robust Subspace Recovery

Authors: Gilad Lerman, Kang Li, Tyler Maunu, Teng Zhang

Organizations: University of Minnesota · University of Central Florida · Brandeis University

Abstract

Robust subspace estimation is fundamental to many machine learning and data analysis tasks. Iteratively Reweighted Least Squares (IRLS) is an elegant and empirically effective approach to this problem, yet its theoretical properties remain poorly understood. This paper establishes that, under deterministic conditions, a variant of IRLS with dynamic smoothing regularization converges linearly to the underlying subspace from any initialization. We extend these guarantees to affine subspace estimation, a setting that lacks prior recovery theory. Additionally, we illustrate the practical benefits of IRLS through an application to low-dimensional neural network training. Our results provide the first global convergence guarantees for IRLS in robust subspace recovery and, more broadly, for nonconvex IRLS on a Riemannian manifold.

Figures & tables

Explore similar work

CardsList
  1. Tight Majorizations and Convergence Rates of Nuclear Norm Minimization IRLS

    Aug 24, 2026Christian Kümmerle, Tomas Masak, Dominik StögerNuclear Norm MinimizationOptimization Convergence Analysis

  2. The Sharp Phase Transition of Tyler's M-Estimator for Robust Subspace Recovery

    Jun 4, 2026Gilad Lerman, Teng ZhangPhase TransitionsRobust Estimation

  3. Linearized subspace refinement framework to expose hidden accuracy in trained neural networks

    Jan 20, 2026Wenbo Cao, Weiwei ZhangSubspace OptimizationNeural Network Optimization