Neural Network Approximation Theory

Ask about this topic

Overview

The field asks which functions and operators neural networks can represent, how many parameters or layers they need, and how approximation error depends on target smoothness and structure. Researchers prove constructive approximation rates and lower bounds for architectures including ReLU and smooth-activation networks, Kolmogorov-Arnold networks, Transformers, and neural operators, often using tools from harmonic analysis, approximation spaces, and geometry. Applications include nonparametric regression and PDE solution maps, where Sobolev, Barron, Besov, compositional, and symmetry-aware assumptions can yield sharper rates than general worst-case guarantees. A central tension is that favorable rates often rely on strong structure or idealized parameter precision, while optimization, sampling, and finite-precision limits can determine whether those guarantees are useful in practice.

Latest papers

CardsList

242 papers29 papers in the last four weeks, up 21% on the four weeks before. 0.2% of all new papers.

  1. Implicit Neural Representations: A Signal Processing Perspective

    Apr 16, 2026Dhananjaya Jayasundara, Vishal M. PatelImplicit Neural RepresentationsSpectral Bias

    Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218 USA

  2. Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension

    Apr 8, 2026Jianfei Li, Shuo Huang, Han Feng +2Curse of DimensionalityNeural Network Approximation Theory

    Department of Mathematics, Ludwig-Maximilians-Universität München (LMU Munich), Akademiestr. 7, 80799 München · Department of Mathematics, City University of Hong Kong, Hong Kong, China · School of Mathematics and Statistics, The University of Sydney, Sydney, New South Wales 2006, Australia +3

  3. Generalization Bounds and Statistical Guarantees for Multi-Task and Multiple Operator Learning with MNO Networks

    Apr 2, 2026Adrien Weihs, Hayden SchaefferOperator LearningGeneralization Bounds

    Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA.

  4. MVNN: A Measure-Valued Neural Network for Learning McKean-Vlasov Dynamics from Particle Data

    Apr 1, 2026Liyao Lyu, Xinyue Yu, Hayden SchaefferEmergent Behavior in Multi-Agent SystemsMean-Field Theory

    Department of Mathematics University of California, Los Angeles Los Angeles, CA 90024, USA

  5. pp-adic Character Neural Network

    Mar 31, 2026Tomoki MiharaNeural Network Approximation TheoryNeural Network Activation Functions

  6. Universal Approximation Constraints of Narrow ResNets: The Tunnel Effect

    Mar 30, 2026Christian Kuehn, Sara-Viola Kuntz, Tobias WöhrerUniversal ApproximationNeural Network Approximation Theory

    Technical University of Munich, School of Computation, Information and Technology, Department of Mathematics, Boltzmannstraße 3, 85748 Garching, Germany · Munich Data Science Institute (MDSI), Garching, Germany · Munich Center for Machine Learning (MCML), München, Germany +1

  7. FI-KAN: Fractal Interpolation Kolmogorov-Arnold Networks

    Mar 30, 2026Gnankan Landry Regis N'guessanKolmogorov-Arnold NetworksNeural Network Approximation Theory

    Axiom Research Group · Dept. of Applied Mathematics and Computational Science, NM-AIST, Arusha, Tanzania · AIMS Research and Innovation Centre, Kigali, Rwanda

  8. Expressibility of neural quantum states: a Walsh-complexity perspective

    Mar 27, 2026Taige WangNeural Quantum StatesNeural Network Expressivity

    Department of Physics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Physics, Harvard University, Cambridge, MA 02138, USA

  9. Kantorovich--Kernel Neural Operators: Approximation Theory, Asymptotics, and Neural Network Interpretation

    Mar 27, 2026Tian-Xiao HeNeural Network Approximation TheoryNeural Operators

    Illinois Wesleyan University, Bloomington, IL 61702

  10. CFNN: Continued Fraction Neural Network

    Mar 21, 2026Chao Wang, Xuancheng Zhou, Ruilin Hou +2Function ApproximationScientific ML

    School of Future Technology, Shanghai University, Shanghai, 200444, China. · Institute of Artificial Intelligence, Shanghai University, Shanghai, 200444, China. · School of Computer Science and Technology, Shanghai University, Shanghai, 200444, China. +2

  11. Deep Hilbert--Galerkin Methods for Infinite-Dimensional PDEs and Optimal Control

    Mar 19, 2026Samuel N. Cohen, Filippo de Feo, Jackson Hebner +1HJB EquationsPDE Solving

    Mathematical Institute, University of Oxford · Institut für Mathematik, Technische Universität Berlin

  12. Mathematical Foundations of Deep Learning

    Mar 19, 2026Xiaojing YeNeural Network Approximation TheoryGenerative Modeling

  13. Deep learning and the rate of approximation by flows

    Mar 16, 2026Jingpu Cheng, Qianxiao Li, Ting Lin +1Neural Network Approximation TheoryFunction Approximation

    Department of Mathematics, National University of Singapore, 117543, Singapore · Department of Mathematics and Institute for Functional Intelligent Materials, National University of Singapore, 117543, Singapore · School of Mathematical Sciences, Peking University, 100871, China

  14. Equivalence of approximation by networks of single- and multi-spike neurons

    Mar 13, 2026Dominik Dold, Philipp Christian PetersenSpiking Neural NetworksNeural Network Approximation Theory

  15. Topological DeepONets and a generalization of the Chen-Chen operator approximation theorem

    Mar 12, 2026Vugar IsmailovNeural Network Approximation TheoryDeep Operator Networks

  16. Efficient Approximation to Analytic and LpL^p functions by Height-Augmented ReLU Networks

    Mar 11, 2026ZeYu Li, FengLei Fan, TieYong ZengNeural Network Approximation TheoryReLU Neural Networks

    Chinese University of Hong Kong, Department of Mathematics, Hong Kong, 999077, NT, Hong Kong SAR, China · City University of Hong Kong, Department of Data Science, Hong Kong, 999077, Kowloon, Hong Kong SAR, China · Guangzhou Nanfang College, School of Mathematics and Statistics, Guangzhou, 510970, Guangdong, China +1

  17. Stochastic Port-Hamiltonian Neural Networks: Universal Approximation with Passivity Guarantees

    Mar 10, 2026Luca Di Persio, Matthias Ehrhardt, Youness OutalebPort-Hamiltonian SystemsHamiltonian Neural Networks

    Department of Computer Science - College of Mathematics University of Verona, Strada le Grazie 15 - 37134 Verona, Italy · University of Wuppertal, Chair of Applied and Computational Mathematics, Gaußstrasse 20, 42119 Wuppertal, Germany · Doctoral School in Mathematics, University of Trento Via Sommarive, 14 - 38123 Povo, Italy

  18. DualFlexKAN: Dual-stage Kolmogorov-Arnold Networks with Independent Function Control

    Mar 9, 2026Andrés Ortiz, Nicolás J. Gallego-Molina, Carmen Jiménez-Mesa +2Function ApproximationKolmogorov-Arnold Networks

    Department of Communications Engineering University of Malaga Malaga, Spain · DaSCI Andalusian Institute of Data Science and Computational Intelligence University of Granada Granada, Spain · Department of signal Theory, Communications and Telematics University of Granada Granada, Spain

  19. Neural Approximation and Its Applications

    Mar 4, 2026Wei-Hao Wu, Ting-Zhu Huang, Xi-Le Zhao +2Neural Network Approximation Theory

    School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, 611731, Sichuan, China. · School of Mathematical and Statistics, Xi’an Jiaotong University, Xi’an, 710049, Shaanxi, China. · Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Taiyuan, 030006, Shanxi, China.

  20. Extending Neural Operators: Robust Handling of Functions Beyond the Training Set

    Mar 4, 2026Blaine Quackenbush, Paul J. AtzbergerNeural OperatorsOOD Generalization

    Department of Mathematics, University of California Santa Barbara (UCSB). · Department of Mathematics, Department of Mechanical Engineering, University of California Santa Barbara (UCSB)

  21. Coalgebras for categorical deep learning: Representability and universal approximation

    Mar 3, 2026Dragan MašulovićEquivariant Representation LearningInvariant Representation Learning

    University of Novi Sad, Faculty of Sciences Department of Mathematics and Informatics Trg Dositeja Obradovića 3, 21000 Novi Sad, Serbia

  22. On the Expressive Power of Transformers for Maxout Networks and Continuous Piecewise Linear Functions

    Mar 3, 2026Linyan Gu, Lihua Yang, Feng ZhouTransformer ExpressivityNeural Network Approximation Theory

    School of Mathematics, Sun Yat-sen University · School of Information Science, Guangdong University of Finance and Economics

  23. Lattice-based Deep Neural Networks: Regularity and Tailored Regularization

    Mar 3, 2026Alexander Keller, Frances Y. Kuo, Dirk Nuyens +1Neural Network Approximation TheoryGeneralization Bounds

  24. Standard Transformers Achieve the Minimax Rate in Nonparametric Regression with Cs,λC^{s,λ} Targets

    Feb 24, 2026Yanming Lai, Defeng SunTransformer ExpressivityNeural Network Approximation Theory

    Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, China

  25. Quantitative Approximation Rates for Group Equivariant Learning

    Feb 23, 2026Jonathan W. Siegel, Snir Hordan, Hannah Lawrence +2Permutation-Equivariant Neural NetworksEquivariant Neural Networks

    Department of Mathematics, Texas A&M University, College Station, TX, 77840, USA · Faculty of Mathematics, Technion- Israel Institute of Technology, Haifa, 3200003, Israel · Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139 USA

  26. Smoothness Adaptivity in Constant-Depth Neural Networks: Optimal Rates via Smooth Activations

    Feb 23, 2026Yuhao Liu, Zilin Wang, Lei Wu +1Neural Network Approximation TheoryNeural Network Activation Functions

    Department of Mathematical Sciences, Tsinghua University · School of Mathematical Sciences, Peking University · Center for Machine Learning Research, Peking University +1

  27. Hardware-Friendly Input Expansion for Accelerating Function Approximation

    Feb 20, 2026Hu Lou, Yin-Jun Gao, Dong-Xiao Zhang +3Neural Network Approximation TheoryNeural Network Optimization

    Northwest Institute of Nuclear Technology, Xi’an, 710000, Shaanxi, China.

  28. From Approximation Rates to Loss-Landscape Barrier Decay in Shallow ReLU Networks

    Feb 19, 2026Saveliy BaturinShallow Neural NetworksReLU Neural Networks

    Independent Researcher, Russia.

  29. FEKAN: Feature-Enriched Kolmogorov-Arnold Networks

    Feb 18, 2026Sidharth S. Menon, Ameya D. JagtapFunction ApproximationKolmogorov-Arnold Networks

    Aerospace Engineering Department, Worcester Polytechnic Institute, Worcester, MA 01609, USA.

  30. Approximation Theory for Lipschitz Continuous Transformers

    Feb 17, 2026Takashi Furuya, Davide Murari, Carola-Bibiane SchönliebTransformerNeural Network Approximation Theory

    Faculty of Life and Medical Sciences, Department of Biomedical Engineering, Doshisha University, RIKEN AIP · Department of Applied Mathematics and Theoretical Physics, University of Cambridge