stat.MLFeb 11, 2026

Deep Learning of Compositional Targets with Hierarchical Spectral Methods

Authors: Hugo Tabanelli, Yatin Dandi, Luca Pesce, Florent Krzakala

Organizations: Information Learning and Physics Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) · Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL)

Abstract

Why depth yields a computational advantage over shallow methods remains a central open question in learning theory. We study this question in a controlled high-dimensional Gaussian setting, focusing on compositional target functions. We analyze their learnability using an explicit three-layer fitting model trained via layer-wise spectral estimators. The target combines polynomial latent features with a scalar readout, and its compositional structure allows learning to proceed in stages. Recovering an intermediate representation reduces the problem to lower-degree spectral estimation tasks, as in multi-index models. Our analysis establishes Gaussian comparison bounds for projected moments when 0<ε<1/30<\varepsilon<1/3, and motivates separations in sample complexity between two- and three-layer learning strategies, conditional on a conjectured spectral recovery step.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Optimal scaling laws in learning hierarchical multi-index models

    Feb 5, 2026Leonardo Defilippis, Florent Krzakala, Bruno Loureiro +1Neural Scaling LawsTwo-Layer Neural Networks

  2. Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model

    May 14, 2026Arie Wortsman-Zurich, Hugo Tabanelli, Yatin Dandi +2Hierarchical Representation LearningScaling Laws

  3. Learning Sparse Compositional Functions with Norm-Constrained Neural Networks

    May 25, 2026Shuo Huang, Lorenzo Fiorito, Lorenzo Rosasco +1Neural Network GeneralizationHierarchical Representation Learning