Neural Network Approximation Theory

Ask about this topic

Overview

The field asks which functions and operators neural networks can represent, how many parameters or layers they need, and how approximation error depends on target smoothness and structure. Researchers prove constructive approximation rates and lower bounds for architectures including ReLU and smooth-activation networks, Kolmogorov-Arnold networks, Transformers, and neural operators, often using tools from harmonic analysis, approximation spaces, and geometry. Applications include nonparametric regression and PDE solution maps, where Sobolev, Barron, Besov, compositional, and symmetry-aware assumptions can yield sharper rates than general worst-case guarantees. A central tension is that favorable rates often rely on strong structure or idealized parameter precision, while optimization, sampling, and finite-precision limits can determine whether those guarantees are useful in practice.

Latest papers

CardsList

241 papers29 papers in the last four weeks, up 21% on the four weeks before. 0.2% of all new papers.

  1. Representation Complexity Along Learning Trajectories: Trading Width for Precision and Time

    Oct 28, 2026Ryan Singh, Miguel De Llanza, Christopher L. BuckleyOverparameterized Neural NetworkNeural Network Training Dynamics

    Department of Engineering and Informatics, University of Sussex

  2. Para-Gaussian Approximations of Random Deep Neural Networks

    Oct 27, 2026Mihai Nica, Janosch OrtmannNeural Network Approximation TheoryReLU Neural Networks

  3. Boundary Geometry Determines Optimal Shallow ReLU Approximation under Weighted Coefficient Constraints

    Oct 27, 2026Jia-Qi Yang, Lei ShiNeural Network Approximation TheoryShallow Neural Networks

    School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China.

  4. Holomorphic Cauchy Networks for Efficient Stationary Tail Computation in High-Dimensional Reflected Brownian Motions

    Oct 27, 2026Kehan WangComplex-Valued Neural NetworksStochastic Differential Equations

    School of Data Science, The Chinese University of Hong Kong, Shenzhen, China

  5. Proofs of strong forms of the Bovdi--Geche universality conjecture and a complex analogue

    Oct 26, 2026Victor Bovdi, Ambrus PalUniversal ApproximationNeural Network Approximation Theory

  6. The Geometry of Limits in ReLU Function Spaces

    Oct 24, 2026Kathryn A. LindseyReLU Neural NetworksNeural Network Approximation Theory

  7. Universal neural network approximation of symmetric polyconvex energies and envelopes

    Oct 23, 2026Florian GammerNeural Network Approximation TheoryInput Convex Neural Networks

    Faculty of Mathematics, University of Regensburg, Universitätsstraße 31 93053 Regensburg, Germany

  8. Phase Transition from Random Features to Fully Trainable Networks

    Oct 22, 2026Xin He, Huan Lao, Xiliang Lu +1Neural Network Approximation TheoryPhase Transitions in Neural Networks

  9. Deep ReLU Networks Can Fail the Best Approximation Property for Continuous Target Functions

    Oct 22, 2026Sebastian KassingNeural Network Approximation TheoryReLU Neural Networks

  10. Tractable Inference with Kolmogorov-Arnold Networks

    Oct 21, 2026Juan Diego Toscano, Sascha Ranftl, Zhiwei Gao +2Kolmogorov-Arnold NetworksNeural Surrogate Modeling

    Division of Applied Mathematics, Brown University, Providence, 02912, RI, USA · Division of Mathematical Sciences, Nanyang Technological University, 637371, Singapore

  11. Depth Separation for Neural Networks under Gaussian Measure

    Oct 20, 2026Cynthia Bortolotto, Francisco Ganacim, Roberto I. OliveiraNeural Network Approximation TheoryReLU Neural Networks

  12. Constructive Universal Approximation by Infinite-Dimensional Neural Networks via Gaussian-Weighted Ridgelet Transform

    Oct 16, 2026Sho Sonoda, Luca Galimberti, Yuanchao Xu +2Neural Network Approximation TheoryNeural Operators

    RIKEN Center for Advanced Intelligence Project (AIP), Nihonbashi 1-chome Mitsui Building, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan · AI tech studio, CyberAgent, Inc., Shibuya Scramble Square, 2-24-12 Shibuya, Shibuya-ku, Tokyo 150-6121, Japan · Department of Mathematics, King’s College London, Strand Building, London, WC2R 2LS, UK +2

  13. Wavelength Diversity Expands the Expressive Power of Optical Computing

    Oct 16, 2026Shiqi Chen, Che-Yung Shen, Parnian Ghapandar Kashani +2Photonic ComputingUniversal Approximation

    Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA · Bioengineering Department, University of California, Los Angeles, CA, 90095, USA · California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA

  14. Optimal approximation of smooth functions by ReLU neural networks

    Oct 15, 2026Thomas NaglerNeural Network Approximation TheoryReLU Neural Networks

    Department of Statistics, LMU Munich Munich Center for Machine Learning (MCML)

  15. Upper bounds on the number of causal pieces in spiking neural networks

    Oct 15, 2026Dominik Dold, Dennis Elbrächter, Philipp PetersenSpiking Neural NetworksNeural Network Expressivity

  16. Activation-Adjacent Layer Normalization: Effects on Approximation Power

    Oct 15, 2026Yunhao Ni, Yuxin Guo, Jie Luo +2Layer NormalizationNeural Network Approximation Theory

    State Key Laboratory of Complex and Critical Software Environment, Beihang University, Beijing, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, China

  17. Kolmogorov-Arnold Networks: Curse of Dimensionality and Structural Bias

    Oct 15, 2026Yunhao Ni, Yuxin Guo, Jie Luo +2Curse of DimensionalityKolmogorov-Arnold Networks

    State Key Laboratory of Complex and Critical Software Environment, Beihang University, Beijing, China · Hangzhou International Innovation Institute, Beihang University, Hangzhou, China

  18. Adaptive Randomized Neural Networks for PDEs: Structure-Aware Feature Redistribution and Residual-Driven Block Enrichment

    Oct 15, 2026You Yang, Fei WangNeural PDE SolversRandom Feature Methods

    School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China. · School of Mathematics and Statistics & State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China.

  19. The minimal width of neural networks and immersions of real projective space

    Oct 14, 2026BaoZhi Chu, Ben WeinkoveNeural Network Approximation TheoryNeural Network Expressivity

  20. How Long Is the Coast of a Neuron? Resolution-Dependent Interaction Order and Computational Capacity

    Oct 14, 2026Nima DehghaniNeural EncodingNeural Network Approximation Theory

    McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI), Cambridge, Massachusetts, USA

  21. Wong--Zakai Convergence and FNO Learning for Stochastic Volterra Equations

    Oct 13, 2026Wonjae Lee, Jiuk Jang, Hyungbin ParkNeural OperatorsFourier Neural Operator

    InnoCore AI Meta-Scientist Research Group, KAIST, Daejeon 34141, South Korea. · Department of Mathematical Sciences, Seoul National University, Seoul 08826, South Korea. · Department of Mathematical Sciences and Research Institute of Mathematics, Seoul National University, 1, Gwanak-ro, Gwanak-gu, Seoul 08826, South Korea.

  22. Operator Surrogates

    Oct 13, 2026Christoph Schwab, Josephine Westermann, Jakob ZechSurrogate ModelingOperator Learning

    SAM, ETH Zürich, Rämistrasse 101, 8092 Zürich, Switzerland · IWR, Uni Heidelberg, INF 205, 69120 Heidelberg, Germany

  23. Lifting Finite-Sample Gaussian Approximation of Wide Neural Networks to Functional Convergence

    Oct 13, 2026Luis Wiederhold, Nil Ayday, Debarghya GhoshdastidarConvergence AnalysisNeural Network Approximation Theory

    School of Computation, Information and Technology Technical University of Munich

  24. Approximation in Contrastive Representation Learning

    Oct 11, 2026Yuanfan Li, Zihan Zhang, Yiming Ying +1Contrastive LearningMultimodal Contrastive Learning

    School of mathematics and statistics, University of Sydney

  25. Rationally Linearized MLPs

    Oct 9, 2026Arijit Sehanobish, Byeongchan Kim, Avinava Dubey +2Multilayer PerceptronsEfficient Neural Network Inference

    Kensho Technologies · Seoul National University · Google Research +2

  26. The Expressive Power of Random-Feature MPNNs Across Graph Sizes

    Oct 8, 2026Robert R. Nerem, Mitchell Black, Antonis Vasileiou +2GNN ExpressivityMessage Passing Neural Networks

    University of California, San Diego · Rheinisch Westfälische Technische Hochschule Aachen

  27. NP-Hardness of Minimizing Neurons in Two-Hidden-Layer ReLU Neural Networks

    Oct 8, 2026Sangrock LeeReLU Neural NetworksNeural Network Approximation Theory

    FAMU–FSU College of Engineering, Florida State University, Tallahassee, FL, 32310

  28. FOSLS-deRhaNN: native de Rham neural classes for H(div) and H(curl) with applications to first-order system least-squares neural network methods for partial differential equations

    Oct 6, 2026Shun ZhangNeural PDE SolversNeural Network Approximation Theory

    Department of Mathematics, City University of Hong Kong, Kowloon Tong, Hong Kong, China

  29. Singular parameters and missing limits in neural PDE solvers

    Oct 5, 2026Daniel FernándezNeural PDE SolversNeural Network Optimization

    Chair for Dynamics, Control, Machine Learning, and Numerics (Alexander von Humboldt Professorship), Department of Mathematics, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91058 Erlangen, Germany.

  30. Beyond Overparameterization: Provable Learning of Input-Convex Multi-Layer Polynomial Networks with Active Queries

    Oct 4, 2026Jinqi Tang, Qian Chen, Shihong Ding +1Polynomial Neural NetworksInput Convex Neural Networks

    Peking University