stat.MLNov 14, 2026

Geometric Stability and Sample Complexity for Compositional Function Priors

Authors: Christopher L. Hammond

Organizations: Susquehanna International Group

Abstract

We study when compositional structure yields dimension reduction in supervised learning. We frame learning as empirical risk minimization with a function-space prior: a regularization functional selects a subset of functions with favorable approximation and metric entropy. The models are compositions f(x)=g(Ax+Δh(x))f(x)=g(Ax+Δh(x)), where A∈Rk×dA\in\mathbb{R}^{k\times d} is a linear base map, ΔhΔh is a nonlinear correction, and gg is an outer function; that is, ff is an arbitrary smooth function of a kk-dimensional representation of the input that is linear up to a correction. We prove covering-number bounds showing that the statistical complexity of this class interpolates between an intrinsic regime, parametric in the kdkd entries of the linear map and nonparametric only in the representation dimension kk, and an ambient dd-dimensional nonparametric rate; the interpolation is controlled by the correction size ε=∥Δh∥\varepsilon=\|Δh\|. We then connect ε\varepsilon to data geometry. If the data lie in a small tube around a kk-dimensional manifold and the regression target depends on the input only through its nearest point on that manifold, then locally the target factorizes exactly as g(Ax+Δh(x))g(Ax+Δh(x)) with AA a tangent chart and a correction whose Sobolev norm scales linearly with curvature at the localization scale. Patching such local models over an atlas gives a finite-sample excess-risk bound for a least-squares estimator over a finite net of local models, whose ambient term vanishes with curvature. For a one-dimensional model problem we also prove a converse: on curved data, every affine compositional predictor faces an error floor unless its correction budget scales with curvature, while the matching curved model class attains the intrinsic rate. The correction size ε\varepsilon is thus the effective-dimension parameter linking curvature to statistical complexity.

Explore similar work

CardsList
  1. Deep Learning of Compositional Targets with Hierarchical Spectral Methods

    Feb 11, 2026Hugo Tabanelli, Yatin Dandi, Luca Pesce +1Statistical Learning TheoryCompositional Representation Learning

  2. A Function-Space Dichotomy for Compositional Learning: Exponential Sub-Optimality of the Neural Tangent Kernel

    Jul 7, 2026Arkaprabha Ganguli, Emil ConstantinescuNeural Network GeneralizationKernel Regression

  3. Learning Sparse Compositional Functions with Norm-Constrained Neural Networks

    May 25, 2026Shuo Huang, Lorenzo Fiorito, Lorenzo Rosasco +1Neural Network GeneralizationHierarchical Representation Learning