cs.LGOct 30, 2026

A Spectral Renormalization-Group View of Learning

Authors: Charles H. Martin

Organizations: Calculation Consulting, San Francisco, CA

Abstract

We study how the layers in deep neural networks (NNs) converge, using Wilsonian Renormalization-Group (RG) ideas to organize the spectral picture. During training, these layers often develop heavy-tailed power-law (PL) spectra, and the strongest checkpoints often appear near the PL exponent α≈2α\approx 2. This pattern raises a simple question: why should this value mark a well-trained layer? The answer is both statistical and physical. Our statistical hypothesis is that α=2α=2 marks the boundary between stable if not optimal learning and overfitting via non-self-averaging. The physical answer is that the same value makes the spectral energy gain scale evenly across logarithmic spectral scales, which is the proposed Wilsonian marginal condition. We then apply the ERG trace-log condition, the proposed SETOL theory, which we justify by taking volume-preserving, scale-invariant transformations to remove arbitrary layer scale along the proposed RG flow. The resulting spectral RG flow connects the HTSR and SETOL theories to optimizers such as Muon, and motivates a new optimizer extension, WW-PGD. Together, these results point toward a spectral Renormalization-Group view of learning, in which NNs may approach shared spectral structure in the settings studied here.

Figures & tables

Explore similar work

CardsList
  1. Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer

    May 8, 2026Clarissa Lauditi, Cengiz Pehlevan, Blake BordelonDeep Linear NetworksNeural Network Training Dynamics

  2. Scaling and Renormalization in Neural Networks: An Effective Renormalization-Group Description of Grokking

    Oct 26, 2026Min Chen, Junqi Wang, Zeguan Wu +2Renormalization GroupGrokking