Neural Network Approximation Theory

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 15% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 163

All topics

From Approximation to Emergence: A Theory of Deep Learning
Jul 1, 2026Zhilin ZhaoNeural Network Generalization Deep Learning Optimization
Sun Yat-sen University
From Spectral Methods to Sample Complexity Bounds for Fourier Neural Operators
Jul 1, 2026Nisha Chandramoorthy, Daniel Sanz-Alonso, Nathan WaniorekDynamical Systems Fourier Neural Operator
University of Chicago
Domain-Decomposed Randomized Neural Networks for Partial Differential Equations in Unbounded Domains
Jun 30, 2026Haixin Wang, Haoning Dang, Fei Wang +1Neural PDE Solvers PDE Solving
School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, P.R. China. · School of Mathematics and Statistics & State Key Laboratory of Multiphase Flow in Power Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China.
Fast approximation and learning of binary classification tasks in o-minimal structures using ReLU neural networks
Jun 29, 2026Clemens Kinn, Philipp PetersenReLU Neural Networks Binary Classification
Faculty of Mathematics, University of Vienna, 1090 Vienna, Austria
Algorithmic Foundations of Deep Learning: Complexity-Theoretic Rates and a Characterization of Universal Approximation
Jun 25, 2026Anastasis Kratsios, Simone Brugiapaglia, Bum Jun Kim +2Universal Approximation Neural Network Approximation Theory
McMaster University & Vector Institute, Toronto, ON, Canada · Concordia University, Montréal, QC, Canada · University of Tokyo, Tokyo, Japan +2
EML Trees Are Universal Approximators
Jun 22, 2026Joe Germany, Elie Abdo, Joseph BakarjiUniversal Approximation Neural Network Approximation Theory
Department of Mathematics American University of Beirut Beirut, Lebanon · Department of Mechanical Engineering American University of Beirut Beirut, Lebanon
Expressivity Saturation: Reduced Affine Region Usage Under Increasing Task Complexity
Jun 19, 2026Xuan Qi, Yi Wei, Fanqi Yu +1Multilayer Perceptrons Neural Network Approximation Theory
AI for Good (AIGO), Istituto Italiano di Tecnologia · DITEN, University of Genoa · State Key Laboratory of Novel Software Technology +1
Recurrent neural networks approximate continuous functions
Jun 18, 2026Valentin Abadie, Clemens Hutter, Helmut BölcskeiReLU Neural Networks Recurrent Neural Networks
Score Approximation for Diffusion Models on Arbitrary Low-Dimensional Structures
Jun 18, 2026Xinhe Mu, Zaijiu Shang, Zhaoqi Zhou +4Neural Network Approximation Theory Score-Based Generative Modeling
Academy of Mathematics and Systems Sciences Chinese Academy of Sciences Beijing, China · Shanghai Institute for Mathematics and Interdisciplinary Sciences Shanghai, China · Huawei Technologies Co., Ltd. Beijing, China
Learning universal approximations for partial differential equations with Physics-Informed Broad Learning System
Jun 18, 2026Zhiwen Yu, Derong Yang, Liujian Zhang +5PDE Solving Universal Approximation
School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China · Peng Cheng Laboratory, Shenzhen 518000, China · School of Future Technology, South China University of Technology, Guangzhou 510006, China +2
Monotonic Kolmogorov-Arnold Networks: A Theoretical and Empirical Study of Monotonicity as an Inductive Bias
Jun 16, 2026Mikhail Krasnov, Blaž Bertalanič, Carolina FortunaRepresentation Learning Kolmogorov-Arnold Networks
Jozef Stefan Institute
Generalization Guarantees for Multi-Input Neural Operator Learning in Sobolev Spaces
Jun 16, 2026Yahong Yang, Zecheng Zhang, Wei Zhu +2Neural Network Approximation Theory Operator Learning
School of Mathematics, Georgia Institute of Technology, 686 Cherry Street, Atlanta, GA 30332-0160, USA. · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USA. · Department of Mathematics, Hong Kong Baptist University, FSC1202, Fong Shu Chuen Building, Hong Kong Baptist University, Kowloon Tong, Hong Kong.
Sobolev Approximation by Fixed-Size Neural Networks with Arbitrary Accuracy
Jun 15, 2026Baicheng Li, Haizhao Yang, Shijun ZhangNeural Network Activation Functions Universal Approximation
Department of Mathematics, University of Maryland, College Park, MD, USA · Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China
RepNN: Tackling spectral bias in deep neural networks for regression and PDE problems via parameter reparameterization
Jun 15, 2026Yong Wang, Tao Zhou, Xuhui MengPDE Solving Spectral Bias
Institute of Interdisciplinary Research for Mathematics and Applied Science, School of Mathematics and Statistics, Huazhong University of Science and Technology, Wuhan 430074, China · Institute of Computational Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China
The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints
Jun 14, 2026Thomas Dittrich, Oliver Potocki, Philipp GrohsRepresentation Learning Minimum Description Length
Johann Radon Institute of Computational and Applied Mathematics, Austrian Academy of Sciences · Faculty of Mathematics, University of Vienna
Representation Costs in Data Science: Foundations and the Quasi-Banach Spaces of Deep Neural Networks
Jun 12, 2026Greg Ongie, Rahul ParhiNeural Network Approximation Theory
Marquette University · University of California, San Diego
Limitations of Learning Tanh Neural Networks with Finite Precision
Jun 9, 2026Philipp Grohs, Matěj TrödlerSample Complexity Neural Network Approximation Theory
Faculty of Mathematics, University of Vienna · RICAM, Austrian Academy of Sciences
Weighted universal approximation of differentiable maps on infinite-dimensional manifolds
Jun 8, 2026Philipp Schmocker, Josef TeichmannUniversal Approximation Neural Network Approximation Theory
Compositional Approximation Can Strictly Outperform Superpositional Approximation
Jun 7, 2026Dennis Elbrächter, Philipp PetersenNeural Network Approximation Theory
Faculty of Mathematics, University of Vienna, Austria
Beyond Linear and Overcomplete Regimes: A Mean-Field Analysis of Bottleneck Autoencoders
Jun 5, 2026Santanu Das, Ramyak Bilas, Pascal Esser +1Autoencoders Mean-Field Theory
STCS department Tata Institute Of Fundamental Research Mumbai, India-400005 · Department of Mathematics Indiana University Bloomington Indiana, USA-47405 · Department of Mathematics Ludwig-Maximilians-Universität München Germany, München-80799 +1
Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activations
Jun 4, 2026Yizhe Ding, Runze Li, Jia Liu +1ReLU Neural Networks Neural Network Approximation Theory
Department of Statistics, The Pennsylvania State University
Rethinking Neural Width for Alternating Current Optimal Power Flow Proxies
Jun 2, 2026Dhruvi Khandelwal, Anurag Basistha, Ayushi Jolotia +1Neural Surrogate Modeling Optimal Power Flow
Department of Electrical Engineering, National Institute of Technology Kurukshetra, India. · Indraprastha Institute of Information Technology Delhi, India. · Department of Electrical Engineering, Indian Institute of Technology Roorkee, India.
Hierarchical RBF-KAN and RBF-SKAN Architectures for Multidimensional Function Approximation and Random Field Learning
Jun 1, 2026Mingtao Xia, Qijing ShenKolmogorov-Arnold Networks Neural Network Approximation Theory
University of Houston, Philip Guthrie Hoffman Hall, 3551 Cullen Blvd, Houston, 77204, Texas, United States of America · University of Birmingham, Watson Building, Birmingham, B15 2TT,, United Kingdom · University of Oxford, Henry Wellcome Building for Molecular Physiology, Old Road, Oxford, OX3 7BN, Oxfordshire, United Kingdom
Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization
May 31, 2026Jia-Qi Yang, Lei ShiNeural Network Approximation Theory Operator Learning
School of Mathematical Sciences and Shanghai Key Laboratory for Contemporary Applied Mathematics, Fudan University, Shanghai 200433, China.
Taming the Loss Landscape of PINNs with Noisy Feynman-Kac Supervision: Operator Preconditioning and Non-Asymptotic Error Bounds
May 30, 2026Nathanael Tepakbong, Hanyu Hu, Chengyu Liu +1Neural Network Approximation Theory Physics-Informed ML
Department of Data Science, City University of Hong Kong, 83 Tat Chee Ave, Hong Kong · Department of Mathematics, City University of Hong Kong, 83 Tat Chee Ave, Hong Kong
Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks
May 29, 2026Yunfei Yang, Jun FanReLU Neural Networks Neural Network Approximation Theory
School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Zhuhai, China. · Department of Mathematics, Hong Kong Baptist University, Kowloon, Hong Kong, China.
Expressive Power of Floating-Point Neural Networks with Arbitrary Reduction Orders and Inexact Activation Implementations
May 27, 2026Yeachan Park, Geonho Hwang, Wonyeol Lee +1Neural Network Activation Functions Universal Approximation
Learning Sparse Compositional Functions with Norm-Constrained Neural Networks
May 25, 2026Shuo Huang, Lorenzo Fiorito, Lorenzo Rosasco +1Neural Network Generalization Hierarchical Representation Learning
MaLGa, DIMA, Università degli Studi di Genova, Genoa, Italy · CBMM, Massachusetts Institute of Technology, Cambridge, MA, USA
Random Neural Network Expressivity for Non-Linear Partial Differential Equations
May 24, 2026Muhammed Ali Mehmood, Lukas GononNeural PDE Solvers PDE Solving
Department of Mathematics Imperial College London UK · School of Computer Science University of St. Gallen Switzerland
IV-Net: A neural network for elliptic PDEs with random and highly varying coefficients
May 24, 2026Shan Zhong, George BirosNeural PDE Solvers PDE Solving
Oden Institute for Computational Science and Engineering, The University of Texas at Austin, 201 E 24th St, Austin, 78712, TX, USA · Walker Department of Mechanical Engineering, The University of Texas at Austin, 3012026 E Dean Keeton St, Austin, 78712, TX, USA

Jul 1, 2026, cs.LGJ/K move · Enter open · S save

From Approximation to Emergence: A Theory of Deep Learning

Zhilin Zhao

Sun Yat-sen University

Neural Network Generalization Deep Learning Optimization In-Context Learning Neural Network Approximation Theory Neural Scaling Laws

Open paper AskPDF arXiv

Deep learning has outgrown any single mathematical explanation. From Approximation to Emergence develops a unified, proof-oriented account of modern deep learning theory, tracing a path from the classical foundations of approximation, optimization, and generalization to the contemporary mechanisms of overparameterization, robustness, generative modeling, transformers, in-context learning, scaling laws, interpretability, alignment, and emergence. Rather than presenting isolated results, the book organizes a broad literature into a coherent research narrative: each theory is examined through the object it controls, the assumptions that make it valid, and the phenomena it leaves unexplained. Written for researchers, graduate students, and mathematically trained practitioners, this monograph offers a rigorous map of deep learning theory as it stands today: powerful, incomplete, and increasingly centered on the question of how learned mechanisms arise from scale, data, architecture, and training.

Momentum

Related topics

Latest papers 163