Overview

Deep linear networks are compositions of linear transformations used to study how depth changes learning, even when the overall input-output function remains linear. Researchers analyze gradient flow and gradient descent, matrix factorization, spectral dynamics, and convex reformulations across settings such as regression, classification, and matrix completion. These models help explain depth-dependent implicit biases, including preferences for low-rank or sparse solutions, as well as feature compression, convergence, and loss-landscape geometry. A central open tension is how reliably these exact results transfer to nonlinear networks, since behavior can depend strongly on initialization, data structure, optimizer, and finite-time dynamics.

Latest papers

CardsList

34 papers6 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

  1. The Impact of Anisotropic Covariance Structure on the Training Dynamics and Generalization Error of Linear Networks

    Jan 11, 2026Taishi Watanabe, Ryo Karakida, Jun-nosuke TeramaeNeural Network GeneralizationNeural Network Training Dynamics

  2. The Geometry of the Set of Equivalent Linear Neural Networks

    Apr 23, 2024Jonathan Richard Shewchuk, Sagnik BhattacharyaDeep Linear NetworksWeight-Space Analysis