Newton-Schulz Iteration

Newton–Schulz iteration approximates matrix operations, especially inverse square roots and polar factors, through repeated matrix multiplications. In machine learning it is used to orthogonalize optimizer updates, including Muon, and to build manifold methods; related work also applies it to attention representations and structured models. Current approaches tune finite polynomial iterations using spectral estimates, preconditioning, or tiled and hardware-aware computations, while theory studies convergence and optimization behavior. A central tension is that fewer or approximate iterations save computation but can change accuracy, convergence guarantees, or an optimizer’s implicit bias, and the best tradeoff depends on the matrix spectrum and task.