Deep Linear Networks
Overview
Deep linear networks are compositions of linear transformations used to study how depth changes learning, even when the overall input-output function remains linear. Researchers analyze gradient flow and gradient descent, matrix factorization, spectral dynamics, and convex reformulations across settings such as regression, classification, and matrix completion. These models help explain depth-dependent implicit biases, including preferences for low-rank or sparse solutions, as well as feature compression, convergence, and loss-landscape geometry. A central open tension is how reliably these exact results transfer to nonlinear networks, since behavior can depend strongly on initialization, data structure, optimizer, and finite-time dynamics.
Latest papers
34 papers6 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.