Neural Network Approximation Theory

The field asks which functions and operators neural networks can represent, how many parameters or layers they need, and how approximation error depends on target smoothness and structure. Researchers prove constructive approximation rates and lower bounds for architectures including ReLU and smooth-activation networks, Kolmogorov-Arnold networks, Transformers, and neural operators, often using tools from harmonic analysis, approximation spaces, and geometry. Applications include nonparametric regression and PDE solution maps, where Sobolev, Barron, Besov, compositional, and symmetry-aware assumptions can yield sharper rates than general worst-case guarantees. A central tension is that favorable rates often rely on strong structure or idealized parameter precision, while optimization, sampling, and finite-precision limits can determine whether those guarantees are useful in practice.