Boundary Geometry Determines Optimal Shallow ReLU Approximation under Weighted Coefficient Constraints

Jia-Qi Yang; Lei Shi

stat.MLOct 27, 2026

Boundary Geometry Determines Optimal Shallow ReLU Approximation under Weighted Coefficient Constraints

Authors: Jia-Qi Yang, Lei Shi

Organizations: School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China.

Abstract

We study how boundary geometry affects $L^q$ approximation by shallow ReLU neural networks on convex bodies $K\subset\mathbb R^d$ . The target classes are unit balls of weighted variation spaces that can enlarge classical spaces by incorporating domain geometry into constraints on output coefficients. We establish upper and lower bounds for arbitrary convex bodies and determine the optimal polynomial approximation exponents for all planar convex bodies and their products with polytopes, including cubes and circular cylinders. For $C^2$ convex bodies, we determine the optimal exponents in a dimension-dependent range of $q$ extending beyond $2$ . When all principal curvatures are positive, the result extends to every finite $q\ge2$ . For these classes, the dependence of the optimal exponent on the domain is captured by a single geometric growth exponent associated with small boundary caps. We also identify sharp weight thresholds for retaining the classical approximation exponent and improve previous weights on Euclidean balls while preserving the classical $L^2$ approximation rate. When the target is given as a finite ReLU network, the upper bounds can be realized by a compressed network using only the original neurons, without increasing the weighted coefficient sum. These results show that boundary geometry can change the optimal exponent even when dimension, error norm, and weighting rule are fixed.

Figures & tables

Panel of Figure 1 — Figure 1: Activation caps and the wet part. In (a), $C_{z}$ has relative volume $\theta_{z}$ , and its height $h_{z}$ equals the maximum of $\phi_{z}$ . In (b), the shaded annulus is the union of all caps of the disk with relative volume at most $\tau$ . Three representative cutting lines are shown.

Domain $K$	Dimension $d$	$\kappa(K)$	$r_{*}(K,\eta,q)$
Convex polytope (including cubes)	$d$	$0$	$\dfrac{d+3}{2d}$
Disk $B^{2}$	$2$	$1/3$	$\min\left\{\dfrac{5}{4},\,1+\dfrac{2}{q}-3\eta\right\}$
Cylinder $B^{2}\times[-1,1]$	$3$	$1/3$	$\min\left\{1,\,1+\dfrac{2}{q}-3\eta\right\}$
Product $B^{2}\times P$	$m+2$	$1/3$	$\min\left\{\dfrac{m+5}{2(m+2)},\,1+\dfrac{2}{q}-3\eta\right\}$
Ball $B^{d}$	$d$	$\dfrac{d-1}{d+1}$	$\min\left\{\dfrac{d+3}{2d},\,1+\dfrac{2}{(d-1)q}-\dfrac{d+1}{d-1}\eta\right\}$
$C^{2}$ body with positive curvature	$d$	$\dfrac{d-1}{d+1}$	$\min\left\{\dfrac{d+3}{2d},\,1+\dfrac{2}{(d-1)q}-\dfrac{d+1}{d-1}\eta\right\}$

Table 1: Examples of geometric exponents and optimal polynomial approximation exponents for

2\leq q<\infty

and

\eta<1/q

Domain $K$	$\kappa(K)$	$r_{*}(K,2/5,2)$
Convex polytope (including the cube)	$0$	$1$
Circular cylinder $B^{2}\times[-1,1]$	$1/3$	$4/5$
Ball $B^{3}$	$1/2$	$7/10$

Table 2: Optimal exponents for

d=3

q=2

, and

\eta=2/5

Panel of Figure 2 — Figure 2: The local cap construction, showing only part of $D$ . The arcs $[A,z]$ and $[z,B]$ each cut off area $T=2\tau$ . A cap of area at most $\tau$ whose boundary arc contains $z$ has endpoints $L\in[A,z]$ and $R\in[z,B]$ and contains the triangle $zLR$ .

Neural Network Approximation Theory

Neural network approximation theory studies how network size, depth, activations, and structure determine the accuracy of functions and operators.

See all 241 papers

Oct 5, 2025

Configuration-Dependent Lower Bounds for Approximation by Shallow ReLU $^k$ Networks on the Sphere

We establish two related but logically distinct results for shallow ReLU

^k

neural networks on the unit sphere

\SS^d

. First, for an arbitrary set of inner neural-network parameters, the best

\mathcal{L}^2(\SS^d)

approximation of a fixed target function with smoothness

r>\tfrac{d+2k+1}{2}

admits an asymptotic lower bound given by a constant multiple of

n^{-1/2}\underline{h}^{k+1/2}

, where

\underline{h}

denotes the antipodal separation distance of the normalized inner-parameter set. This lower bound depends explicitly on the parameter configuration through

\underline{h}

and applies without additional assumptions on the parameters. Second, for antipodally quasi-uniform parameters,

\underline{h}\simeq n^{-1/d}

, and the lower bound establishes the exact saturation order

n^{-\frac{d+2k+1}{2d}}

for such parameter families: a target function with regularity greater than

\frac{d+2k+1}{2}

and satisfying the required parity condition can be approximated at this rate, whereas approximation at any strictly faster rate forces the target function to be zero. Our results therefore place linearized neural-network approximation within the classical saturation framework and show that, although ReLU

^k

network spaces can outperform finite elements of the same degree, this advantage is intrinsically limited.

Tong Mao, Jinchao Xu

Shenzhen Loop Area Institute, Shenzhen 518000, P. R. China · King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia.

Shallow Neural Networks Neural Network Approximation Theory Neural Network Expressivity Function Approximation

Ask PDF

May 18, 2026

Shallow ReLU $^s$ Networks in $L^p$ -Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization

This paper studies approximation by shallow ReLU

^s

networks,

σ_s(t)=\max\{0,t\}^s

, together with their generalization behavior under

\ell_1

path-norm control. For the

L^p

-type integral spaces

\widetilde{\mathcal{F}}_{p,τ_d,s}

1\le p\le2

, spherical harmonic analysis yields approximation bounds for shallow networks. In particular, when

τ_d

is the uniform measure and

1\le p<2

, the approximation rate is

O\!\left(m^{-\frac{p(2s+2d+1)-2d}{2dp}}\right)

for

1\le p\le p^*

and

O\!\left(m^{-\frac{p(4s+3d-1)-2d+2}{4dp}}\right)

for

p^*<p<2

, where

p^*=\frac{2d+2}{d+3}

. Approximation bounds for Sobolev spaces

W^{α,p}

1\le p<2

, are obtained through embeddings into spectral Barron spaces. For nonparametric regression with sub-Gaussian noise, path-norm-regularized shallow ReLU

^s

networks achieve minimax-optimal rates

O\!\left(n^{-\frac{d+2s+1}{2d+2s+1}}\log n\right)

over

\mathscr{B}_s

and

O\!\left(n^{-\frac{2α}{2α+d}}\log n\right)

over

W^{α,\infty}

, with matching lower bounds up to logarithmic factors.

Weizhao Li, Fanghui Liu, Lei Shi

School of Mathematical Sciences Fudan University Shanghai, China · School of Mathematical Sciences Institute of Natural Sciences and MOE-LSC2026 Shanghai Jiao Tong University Shanghai, China · School of Mathematical Sciences Shanghai Key Laboratory for Contemporary Applied Mathematics25 Fudan University Shanghai, China

Shallow Neural Networks Nonparametric Regression Neural Network Approximation Theory Generalization Bounds

Ask PDF

Sep 17, 2026

Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks

Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights. We extend these results from uniform approximation to approximation in Sobolev norm. Specifically, we analyze how well Sobolev functions in

W^{n,p}

can be approximated by neural networks with width

W

, depth

L

and path norm bounded by

K

, when the approximation error is measured in the

W^{1,p}

-norm. For shallow networks with depth

L=1

, we derive the approximation error bound

\mathcal{O}(\max\{W^{-(n-1)/d}, K^{-(n-1)/(s-n)}\})

, when the smoothness index satisfies

n<s=(d+3)/2

and the input is

d

-dimensional. For deep networks, we remove the restriction on the smoothness by showing that the approximation bound

\mathcal{O}(K^{-(n-1)/(d+d/p+1)})

holds if the width

W

and depth

L

are sufficiently large.

Xianjun Li, Yunfei Yang

School of Statistics and Data Science, Capital University of Economics and Business, Beijing, China. · School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Zhuhai, China.

Shallow Neural Networks ReLU Neural Networks Neural Network Approximation Theory

Ask PDF

Abstract

Neural Network Approximation Theory

Explore similar work

Configuration-Dependent Lower Bounds for Approximation by Shallow ReLUk^kk Networks on the Sphere

Shallow ReLUs^ss Networks in LpL^pLp-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization

Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks

Configuration-Dependent Lower Bounds for Approximation by Shallow ReLU $^k$ Networks on the Sphere

Shallow ReLU $^s$ Networks in $L^p$ -Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization