math.NAOct 5, 2025

Configuration-Dependent Lower Bounds for Approximation by Shallow ReLU^k Networks on the Sphere

Authors: Tong MaoJinchao Xu

Abstract

We establish two related but logically distinct results for shallow ReLUk^k neural networks on the unit sphere \SSd\SS^d. First, for an arbitrary set of inner neural-network parameters, the best L2(\SSd)\mathcal{L}^2(\SS^d) approximation of a fixed target function with smoothness r>d+2k+12r>\tfrac{d+2k+1}{2} admits an asymptotic lower bound given by a constant multiple of n1/2hk+1/2n^{-1/2}\underline{h}^{k+1/2}, where h\underline{h} denotes the antipodal separation distance of the normalized inner-parameter set. This lower bound depends explicitly on the parameter configuration through h\underline{h} and applies without additional assumptions on the parameters. Second, for antipodally quasi-uniform parameters, hn1/d\underline{h}\simeq n^{-1/d}, and the lower bound establishes the exact saturation order nd+2k+12dn^{-\frac{d+2k+1}{2d}} for such parameter families: a target function with regularity greater than d+2k+12\frac{d+2k+1}{2} and satisfying the required parity condition can be approximated at this rate, whereas approximation at any strictly faster rate forces the target function to be zero. Our results therefore place linearized neural-network approximation within the classical saturation framework and show that, although ReLUk^k network spaces can outperform finite elements of the same degree, this advantage is intrinsically limited.

Explore similar work

May 18, 2026stat.ML

Shallow ReLU^s Networks in L^p-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization

This paper studies approximation by shallow ReLUs^s networks, σs(t)=max{0,t}sσ_s(t)=\max\{0,t\}^s, together with their generalization behavior under 1\ell_1 path-norm control. For the LpL^p-type integral spaces F~p,τd,s\widetilde{\mathcal{F}}_{p,τ_d,s}, 1p21\le p\le2, spherical harmonic analysis yields approximation bounds for shallow networks. In particular, when τdτ_d is the uniform measure and 1p<21\le p<2, the approximation rate is O ⁣(mp(2s+2d+1)2d2dp)O\!\left(m^{-\frac{p(2s+2d+1)-2d}{2dp}}\right) for 1pp1\le p\le p^* and O ⁣(mp(4s+3d1)2d+24dp)O\!\left(m^{-\frac{p(4s+3d-1)-2d+2}{4dp}}\right) for p<p<2p^*<p<2, where p=2d+2d+3p^*=\frac{2d+2}{d+3}. Approximation bounds for Sobolev spaces Wα,pW^{α,p}, 1p<21\le p<2, are obtained through embeddings into spectral Barron spaces. For nonparametric regression with sub-Gaussian noise, path-norm-regularized shallow ReLUs^s networks achieve minimax-optimal rates O ⁣(nd+2s+12d+2s+1logn)O\!\left(n^{-\frac{d+2s+1}{2d+2s+1}}\log n\right) over Bs\mathscr{B}_s and O ⁣(n2α2α+dlogn)O\!\left(n^{-\frac{2α}{2α+d}}\log n\right) over Wα,W^{α,\infty}, with matching lower bounds up to logarithmic factors.
Weizhao Li, Fanghui Liu, Lei Shi
Sep 14, 2026stat.ML

ReLU Neural Network Approximation to Smooth Functional Operator: Dimensional Decay and Error Analysis

We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by deep ReLU neural networks. Writing the functional input as X(t)=d1ξdνd(t)X(t)=\sum_{d\geq1}ξ_dν_d(t), we quantify the importance of coordinate dd through wdsdw_ds_d, where sds_d bounds the magnitude of the corresponding basis score and wdw_d controls the directional Fréchet sensitivity of the target functional. Our constructive analysis combines coordinate truncation, anisotropic partitioning, local Taylor approximation, and ReLU network realization, while allowing unrestricted interactions among the retained coordinates. We establish a general nonasymptotic upper bound for the uniform approximation error and a complementary pseudo-dimension-based lower bound for the worst-case approximation error. Under generalized exponential coordinate decay wdsdexp(cdρ)w_ds_d\asymp\exp(-cd^ρ), with ρ>0ρ>0, the upper and lower bounds match at the leading order and thus yield the nearly optimal approximation rate, which is stretched-exponential in the logarithm of the network budget. This is the first work to characterize neural network approximation error for infinite-dimensional functional inputs explicitly through the joint dimensional decay of coordinate magnitudes and directional sensitivities.
Shuhao Jiao
Aug 7, 2026math.NA

Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples

We develop a rigorous theory of discrete residual least-squares approximation for elliptic spectral equations Lβu=f\mathfrak L_βu=f using linearized ReLUk^k neural networks on the sphere, where Lβ\mathfrak L_β is a positive elliptic spectral multiplier of order ββ. Given a parameter set Θn={θj}j=1nSdΘ_n=\{θ_{j}^*\}_{j=1}^n\subset\mathbb S^d, we approximate uu in the linearized network space Lnk(Θn)L_n^k(Θ_n) by the discrete residual on the collocation points {ηi}i=1m\{η_i^*\}_{i=1}^m \begin{equation*} u_{n,m}\in\arg\min_{v_n\in L_n^k(Θ_n)}\frac1m\sum_{i=1}^m\left(f(η_i^)-\mathfrak L_βv_n(η_i^)\right)^2. \end{equation*} With k>d12+βk>\frac{d-1}{2}+β, for antipodally quasi-uniform network parameter sets and any quasi-uniform collocation points with mnm\gtrsim n, we prove that \begin{equation*} |u-u_{n,m}|{\mathcal H^β(\mathbb S^d)}\eqsim|f-\mathfrak L_βu{n,m}|{\mathcal L^2(\mathbb S^d)}\lesssim n^{-\frac{r}{d}} \begin{cases} |f|{\mathcal W^{r,p}(\mathbb S^d)},&\frac{d}{p}<r\leq \frac{d}{2},~p>2,\ |f|{\mathcal H^r(\mathbb S^d)},&r>\frac{d}{2}. \end{cases} \end{equation*} We also establish a high-probability residual estimate, up to a logarithmic factor and an arbitrarily small smoothness loss, for i.i.d.\ uniformly distributed collocation points. The key analytical ingredient is a Bernstein inequality for linearized ReLUk^k network spaces. If h\underline h denotes the antipodal separation distance of the network parameters, then \begin{equation*} |v_n|{\mathcal H^r(\mathbb S^d)}\lesssim\underline h^{-(r-s)}|v_n|_{\mathcal H^s(\mathbb S^d)},\qquad 0\leq s<r<k+\tfrac12. \end{equation*}
Xinliang Liu, Tong Mao, Jinchao Xu