Department of Applied Mathematics Hong Kong Polytechnic University · Department of Mathematics National University of Singapore · Department of Mathematics and Statistics Old Dominion University
Depth is widely viewed as a central contributor to the success of deep neural networks, whereas standard neural network approximation theory typically provides guarantees only for the final output and leaves the role of intermediate layers largely unclear. We address this gap by developing a quantitative framework in which depth admits a precise scale-dependent interpretation. Specifically, we design a single shared mixed-activation architecture of fixed width
2dN+d+2 and any prescribed finite depth such that each intermediate readout
Φℓ is itself an approximant to the target function
f. For
f∈Lp([0,1]d) with
p∈[1,∞), the approximation error of
Φℓ is controlled by
(2d+1) times the
Lp modulus of continuity at the geometric scale
N−ℓ for all
ℓ. The estimate reduces to the geometric rate
(2d+1)N−ℓ if
f is
1-Lipschitz. Our network design is inspired by multigrade deep learning, where depth serves as a progressive refinement mechanism. For every prescribed terminal depth, the construction yields a finite nested family of prefix readouts whose earlier correction terms remain embedded in later readouts. Thus the approximation may be truncated within the prescribed depth range once the desired certified accuracy is reached.