Randomly initialised neural networks of width m converge to Gaussian processes as m→∞, with two types of quantitative bounds characterising this convergence. Finite-sample bounds compare the network with its Gaussian limit at a fixed finite sample set, on an arbitrary domain, in the quadratic Wasserstein distance W2, achieving a convergence rate of m−1/2 with explicit constants. However, these constants grow with the number of samples, so the bound becomes uninformative as the sample set is refined. Functional bounds remove that dependence by comparing the two as random functions, at the price of spherical input, metrics weaker than W2, or slower rates. In this paper, we therefore lift a finite-sample bound to a functional one. The lifting refines the sample set until it is dense in the domain, so the bound must first be made independent of the number of samples. We achieve this by measuring the error per sample, which makes the constant uniform. For a polynomial activation and any compact input set, this gives a rate of m−1/2 in the supremum norm, with an explicit constant.
Figures & tables
Result
Depth
Inputs
Metric
Activation
Rate
with n
finite sample
( Basteri and Trevisan, 2024 , Thm. 3.1)
any
finite
W2
Lipschitz
m−1/2
n § 2.3
( Bordino et al., 2024 , Thm. 6)
1
finite
W1
C2 , poly. gr.
m−1/2
n
( Balasubramanian and Ross, 2025 , Thm.1.1)
any
finite
W1
Lipschitz
m−1/6
implicit
( Favaro et al., 2025 , Thm. 3.6(1))
any
finite ∗
dcvx
poly. gr.
m−1/2
( Trevisan, 2023 , Thm. 1.1(a))
any
finite ∗
Wp
Lipschitz
m−1
( Celli and Peccati, 2025 , Thm. 5)
any
finite ∗
W2
C1 or PL
m−1
Table 1: Quantitative Gaussian approximation for wide networks; rates are in the width m , and n is the number of inputs. Finite-sample bounds hold at a fixed sample set with a constant growing in n ; “implicit” means the bound depends on the input set, but this dependence is not quantified. Functional bounds avoid this at the cost of the input set, metric or rate. finite ∗ : finite input, non-degenerate kernel; dcvx : convex distance; poly. gr.: polynomial growth; PL: piecewise linear. Rates for Balasubramanian and Ross (2025) and Balasubramanian et al. (2023) are one-hidden-layer specialisations. For the latter, inputs lie on the unit sphere of Rd and qd=8d−21−ε for arbitrary ε>0 . By Corollary 1 the last row holds on every compact subset of Rd , with the constant of an enclosing hyperrectangle.
Figure 1: Roadmap of Section 3 . Each inequality bounds the quantity on its left by the one on its right, and the reference names the result that does it. A norm equivalence bounds the supremum norm by the L2(I) norm, a discretization bounds that by the grid values, and a coupling bounds those by the width. Composing the three steps proves Theorem 2 .
We consider the infinite-width limit of a fully connected deep neural network with general weights, and we prove quantitative general bounds on the 2-Wasserstein distance between the network and its infinite-width Gaussian limit, under appropriate regularity assumptions on the activation function. Our main tool is a Lindeberg principle for Deep Neural Networks, which we use to successively replace the weights on each layer by Gaussian random variables.
Filippo Giovagnini, Sotirios Kotitsas, Marco Romito
We study the infinite-width Gaussian-process limit of random neural networks through the lens of tensor programs, and we provide a quantitative convergence theory in Wasserstein distance. Our main result gives explicit finite-width error bounds, of order inverse square-root of the widths between finite-network executions and their Gaussian-process limits. The framework is architecture-agnostic and covers feed-forward models together with weight-sharing schemes relevant for recurrent and transformer-type architectures.
We establish central and non-central limit theorems for sequences of functionals of the Gaussian output of an infinitely-wide random neural network on the d-dimensional sphere . We show that the asymptotic behaviour of these functionals as the depth of the network increases depends crucially on the fixed points of the covariance function, resulting in three distinct limiting regimes: convergence to the same functional of a limiting Gaussian field, convergence to a Gaussian distribution, convergence to a distribution in the Qth Wiener chaos. Our proofs exploit tools that are now classical (Hermite expansions, Diagram Formula, Stein-Malliavin techniques), but also ideas which have never been used in similar contexts: in particular, the asymptotic behaviour is determined by the fixed-point structure of the iterative operator associated with the covariance, whose nature and stability governs the different limiting regimes.
Simmaco Di Lillo, Leonardo Maini, Domenico Marinucci
RoMaDS - Department of Mathematics, University of Rome Tor Vergata, Rome, Italy