Deep Learning of Compositional Targets with Hierarchical Spectral Methods
Organizations: Information Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL)
Abstract
Why depth yields a computational advantage over shallow methods remains a central open question in learning theory. We study this question in a controlled high-dimensional Gaussian setting, focusing on compositional target functions. We analyze their learnability using an explicit three-layer fitting model trained via layer-wise spectral estimators. The target combines polynomial latent features with a scalar readout, and its compositional structure allows learning to proceed in stages. Recovering an intermediate representation reduces the problem to lower-degree spectral estimation tasks, as in multi-index models. Our analysis establishes Gaussian comparison bounds for projected moments when , and motivates separations in sample complexity between two- and three-layer learning strategies, conditional on a conjectured spectral recovery step.
Figures & tables
| (3.1) |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Statistical Learning Theory
Statistical learning theory studies when data-driven algorithms generalize, how many samples they need, and what statistical or computational limits apply.
See all 633 papers