Neural Network Approximation Theory

Follow this topic Ask about this topic

The field asks which functions and operators neural networks can represent, how many parameters or layers they need, and how approximation error depends on target smoothness and structure. Researchers prove constructive approximation rates and lower bounds for architectures including ReLU and smooth-activation networks, Kolmogorov-Arnold networks, Transformers, and neural operators, often using tools from harmonic analysis, approximation spaces, and geometry. Applications include nonparametric regression and PDE solution maps, where Sobolev, Barron, Besov, compositional, and symmetry-aware assumptions can yield sharper rates than general worst-case guarantees. A central tension is that favorable rates often rely on strong structure or idealized parameter precision, while optimization, sampling, and finite-precision limits can determine whether those guarantees are useful in practice.

Latest papers

242 papers29 papers in the last four weeks, up 21% on the four weeks before. 0.2% of all new papers.

BEACONS: Bounded-Error, Algebraically-Composable Neural Solvers for Partial Differential Equations
Feb 16, 2026Jonathan Gorard, Ammar Hakim, James JunoNeural PDE Solvers Partial Differential Equations
Princeton University Princeton, NJ 08544, USA · Princeton Plasma Physics Laboratory Princeton, NJ 08540, USA
Divine Benevolence is an $x^2$ : GLUs scale asymptotically faster than MLPs
Feb 16, 2026Alejandro Francisco QueirugaNeural Scaling Laws Neural Network Approximation Theory
Rational Neural Networks have Expressivity Advantages
Feb 12, 2026Maosen Tang, Alex TownsendNeural Network Activation Functions Neural Network Approximation Theory
Center for Applied Mathematics, Cornell University, United States · Department of Mathematics, Cornell University, United States
Clifford Kolmogorov-Arnold Networks
Feb 5, 2026Matthias Wolff, Francesco Alesiani, Christof Duhme +1Kolmogorov-Arnold Networks Neural Network Approximation Theory
Dept. of Computer Science University of Münster Münster, Germany · Data Science Div. NEC Labs Europe Heidelberg, Germany
Geometry-Preserving Neural Architectures on Manifolds with Boundary
Feb 3, 2026Karthik Elamvazhuthi, Shiba Biswal, Kian Rosenblum +4Geometric Representation Learning Neural Network Approximation Theory
Los Alamos National Laboratory · Boston College
Agile Reinforcement Learning through Separable Neural Architecture and Applications
Jan 30, 2026Rajib Mostakim, Reza T. Batley, Sourav SahaRL Control Neural Network Approximation Theory
Department of Mechanical Engineering, Bangladesh University of Engineering and Technology, Dhaka, Bangladesh · Kevin T. Crofton Department of Aerospace and Ocean Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA, USA
Multigrade Neural Network Approximation
Jan 23, 2026Shijun Zhang, Zuowei Shen, Yuesheng XuNeural Network Approximation Theory ReLU Neural Networks
Department of Applied Mathematics, Hong Kong Polytechnic University · Department of Mathematics, National University of Singapore · Department of Mathematics and Statistics, Old Dominion University
Linearized subspace refinement framework to expose hidden accuracy in trained neural networks
Jan 20, 2026Wenbo Cao, Weiwei ZhangSubspace Optimization Neural Network Optimization
Institute of AI for Industries, Chinese Academy of Sciences, Nanjing 211135, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China · School of Aeronautics, Northwestern Polytechnical University, Xi’an 710072, China +2
SUPN: Shallow Universal Polynomial Networks for Efficient Function Approximation
Nov 26, 2025Zachary Morrow, Michael Penwarden, Brian Chen +3Polynomial Neural Networks Neural Network Approximation Theory
The Alexander-Hirschowitz theorem for neurovarieties
Nov 24, 2025A. Massarenti, M. MellaPolynomial Neural Networks Parameter Identifiability
On the Dimension-Free Approximation of Deep Neural Networks for Symmetric Korobov Functions
Nov 16, 2025Yulong Lu, Tong Mao, Jinchao Xu +1Neural Network Generalization Neural Network Approximation Theory
School of Mathematics, University of Minnesota, Minneapolis, 55455, MN, USA · Computer, Electrical and Mathematical Science and Engineering Division, King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia · School of Mathematics, Georgia Institute of Technology, Atlanta, 30332, GA, USA
Autonomous-Flow-Based Generation
Nov 13, 2025Hossein Rouhvarzi, Anastasis KratsiosUniversal Approximation Continuous Normalizing Flows
McMaster University and the Vector Institute Department of Mathematics 1280 Main Street West, Hamilton, Ontario, L8S 4K1, Canada
DFNN: A Deep Fréchet Neural Network Framework for Learning Metric-Space-Valued Responses
Oct 20, 2025Kyum Kim, Yaqing Chen, Paromita DubeyNeural Network Generalization Neural Network Approximation Theory
Rutgers University · University of Southern California
Distributionally robust approximation property of neural networks
Oct 10, 2025Mihriban Ceylan, David J. PrömelNeural Network Approximation Theory Universal Approximation
Configuration-Dependent Lower Bounds for Approximation by Shallow ReLU $^k$ Networks on the Sphere
Oct 5, 2025Tong Mao, Jinchao XuShallow Neural Networks Neural Network Approximation Theory
Shenzhen Loop Area Institute, Shenzhen 518000, P. R. China · King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia.
Neural Operators Which Certifiably Simultaneously Solve Infinitely Many Convex Programs using Logarithmic Rank and Linear Complexity
Aug 20, 2025Anastasis Kratsios, Ariel Neufeld, Philipp SchmockerNeural Operators Operator Learning
Department of Mathematics, McMaster University and the Vector Institute, Canada · Division of Mathematical Sciences, Nanyang Technological University, Singapore
Learning Latent Graph Geometry via Fixed-Point Schrödinger-Type Activation: A Theoretical Study
Jul 27, 2025Dmitry Pasechnyuk-Vilensky, Martin TakáčGraph Structure Learning Graph Neural Networks
MBZUAI, UAE
Understanding Two-Layer Neural Networks with Smooth Activation Functions
Jul 11, 2025Changcun HuangNeural Network Activation Functions Neural Network Approximation Theory
Shuitu Institute of Applied Mathematics, Chongqing 400700, P.R.C
Discretization-independent operator learning for partial differential equations
Jul 9, 2025Jacob Hauck, Yanzhi ZhangPDE Operator Learning Neural Network Approximation Theory
Department of Mathematics and Statistics, Missouri University of Science and Technology, Rolla, MO 65409
Uniform Approximation of Functions with Asymmetric Growth and Decay by Deep Weighted Polynomials
Jun 26, 2025Kingsley Yeon, Steven B. DamelinNeural Network Approximation Theory Function Approximation
Department of Statistics and CCAM, University of Chicago, Chicago, 60637, IL, USA · Department of Mathematics, ZBMATH-OPEN, Hermann-von-Helmholtz-Platz 1, Eggenstein-Leopoldshafen, 76344, Germany
Parallel Layer Normalization for Universal Approximation
May 19, 2025Yunhao Ni, Yuxin Guo, Yuhe Liu +4Layer Normalization Neural Network Approximation Theory
State Key Laboratory of Complex and Critical Software Environment (SKLCCSE), Beihang University, Beijing, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, China
Causal pieces: analysing and improving spiking neural networks piece by piece
Apr 18, 2025Dominik Dold, Philipp Christian PetersenNeural Network Initialization Spiking Neural Networks
On shallow feedforward neural networks with inputs from a topological space
Apr 3, 2025Vugar IsmailovFeedforward Neural Networks Neural Network Approximation Theory
Institute of Mathematics and Mechanics, Baku, Azerbaijan · Center for Mathematics and its Applications, Khazar University, Baku, Azerbaijan
New universal operator approximation theorem for encoder-decoder architectures
Mar 31, 2025Janek Gödeke, Pascal FernselNeural Network Approximation Theory Deep Operator Networks
Center for Industrial Mathematics University of Bremen
The Mathematical Theory of Double Descent as Shrinkage, not Overfitting: Matching Bounds for Path-Regularized Multilayer Neural Networks
Mar 3, 2025Hao YuDouble Descent Neural Network Generalization
Department of Automation, Tsinghua University, Beijing, 100086, China
Fourier Multi-Component and Multi-Layer Neural Networks: Unlocking High-Frequency Potential
Feb 26, 2025Shijun Zhang, Hongkai Zhao, Yimin Zhong +1Fourier Feature Embeddings Neural Network Optimization
Department of Applied Mathematics Hong Kong Polytechnic University · Department of Mathematics Duke University · Department of Mathematics and Statistics Auburn University +1
Networks with Finite VC Dimension: Pro and Contra
Feb 4, 2025Vera Kurkova, Marcello SanguinetiNeural Network Generalization Neural Network Approximation Theory
Institute of Computer Science of the Czech Academy of Sciences Pod Vodárenskou věží, 2 - 18207 Prague, Czech Republic · DIBRIS University of Genova Via Opera Pia, 13 - 16145 Genova, Italy
Dense ReLU Neural Networks for Temporal-spatial Model
Nov 15, 2024Carlos Misael Madrid Padilla, Zhi Zhang, Xiaokai Luo +2Nonparametric Regression ReLU Neural Networks
Department of Statistics and Data Science, Washington University in St. Louis. · Department of Statistics and Data Science, University of California, Los Angeles · Department of Statistics, University of Notre Dame
Polynomial Scaling is Possible For Neural Operator Approximations of Structured Families of BSDEs
Oct 18, 2024Takashi Furuya, Anastasis KratsiosStochastic Differential Equations PDE Solving
Doshisha University, Japan · Department of Mathematics McMaster University and Vector Institute, Canada
Statistical Properties of Deep Neural Networks with Dependent Data
Oct 14, 2024Chad BrownNonparametric Regression Neural Network Approximation Theory
Department of Economics, University of Colorado Boulder, 80309, USA.

Feb 16, 2026J/K move · Enter open · S save

BEACONS: Bounded-Error, Algebraically-Composable Neural Solvers for Partial Differential Equations

Jonathan Gorard, Ammar Hakim, James Juno

Princeton University Princeton, NJ 08544, USA · Princeton Plasma Physics Laboratory Princeton, NJ 08540, USA

Neural PDE Solvers Partial Differential Equations Neural Network Generalization Neural Network Verification Neural Network Approximation Theory

Open paper AskPDF arXiv

The traditional limitations of neural networks in reliably generalizing beyond the convex hulls of their training data present a significant problem for computational physics, in which one often wishes to solve PDEs in regimes far beyond anything which can be experimentally or analytically validated. In this paper, we show how it is possible to circumvent these limitations by constructing formally-verified neural network solvers for PDEs, with rigorous convergence, stability, and conservation properties, whose correctness can therefore be guaranteed even in extrapolatory regimes. By using the method of characteristics to predict the analytical properties of PDE solutions a priori (even in regions arbitrarily far from the training domain), we show how it is possible to construct rigorous extrapolatory bounds on the worst-case L^inf errors of shallow neural network approximations. Then, by decomposing PDE solutions into compositions of simpler functions, we show how it is possible to compose these shallow neural networks together to form deep architectures, based on ideas from compositional deep learning, in which the large L^inf errors in the approximations have been suppressed. The resulting framework, called BEACONS (Bounded-Error, Algebraically-COmposable Neural Solvers), comprises both an automatic code-generator for the neural solvers themselves, as well as a bespoke automated theorem-proving system for producing machine-checkable certificates of correctness. We apply the framework to a variety of linear and non-linear PDEs, including the linear advection and inviscid Burgers' equations, as well as the full compressible Euler equations, in both 1D and 2D, and illustrate how BEACONS architectures are able to extrapolate solutions far beyond the training data in a reliable and bounded way. Various advantages of the approach over the classical PINN approach are discussed.