Neural Network Approximation Theory

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 15% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 163

All topics

FOSLS-deRhaNN: native de Rham neural classes for H(div) and H(curl) with applications to first-order system least-squares neural network methods for partial differential equations
Oct 6, 2026Shun ZhangNeural PDE Solvers Neural Network Approximation Theory
Department of Mathematics, City University of Hong Kong, Kowloon Tong, Hong Kong, China
Singular parameters and missing limits in neural PDE solvers
Oct 5, 2026Daniel FernándezNeural PDE Solvers Neural Network Optimization
Chair for Dynamics, Control, Machine Learning, and Numerics (Alexander von Humboldt Professorship), Department of Mathematics, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91058 Erlangen, Germany.
Beyond Overparameterization: Provable Learning of Input-Convex Multi-Layer Polynomial Networks with Active Queries
Oct 4, 2026Jinqi Tang, Qian Chen, Shihong Ding +1Polynomial Neural Networks Input Convex Neural Networks
Peking University
Universal interpolation for deep residual self-attention networks
Oct 1, 2026Sibylle Marcotte, Joan BrunaSoftmax Attention Transformer
Department of Computer Science New York University
Minimax rates for learning spectral Barron functions by deep ReLU neural networks
Sep 30, 2026Songqiu Ma, Yunfei YangReLU Neural Networks Minimax Estimation
School of Mathematics (Zhuhai), Sun Yat-sen University, Zhuhai, China. · School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Zhuhai, China.
Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity
Sep 28, 2026Zilan Cheng, Li-Lian Wang, Zhongjian WangNeural Network Approximation Theory Feedforward Neural Networks
Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, 637371, Singapore.
From Distributions to Stochastic Processes: Neural Approximation of Measure-Valued Maps
Sep 27, 2026Yichen Wang, Ziyi Wang, Wenlian Lu +5Neural Network Approximation Theory
Shanghai Center for Mathematical Sciences, Shanghai, China · Department of Mathematics, Fudan University, Shanghai, China · Center for Applied Mathematics, Fudan University, Shanghai, China +2
Neural Scaling Laws of Transformer Operator Network
Sep 27, 2026Haoran Yan, Zhongjie Shi, Yuanzhe Xi +2Neural Network Generalization Neural Network Approximation Theory
School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332, USA · Department of Mathematics, Emory University, Atlanta, GA 30322, USA · School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
Activation-Flexible ANN-to-SNN Conversion with Finite-State Markov Neurons
Sep 24, 2026Ruiyu Jia, Zhuo-Cheng XiaoNeural Network Activation Functions Spiking Neural Networks
New York University Shanghai, Shanghai, China
Neural Approximation by Function Composition: Rigidity and Doubly Exponential Convergence
Sep 22, 2026Wentao Huang, Haizhang ZhangNeural Network Approximation Theory Neural Network Expressivity
Department of Mathematics, The University of Hong Kong, Hong Kong, P.R. China. · School of Mathematics, Sun Yat-sen University, Guangzhou, P.R. China.
Statistical Gains from Looped Estimation under Parameter Budgets
Sep 22, 2026Xinyu Tian, Xiaotong ShenParameter Estimation Minimax Estimation
School of Statistics, University of Minnesota
Optimal Tradeoffs Between Network Size and Parameter Magnitude in Neural Approximation and Minimax Regression
Sep 22, 2026Baicheng Li, Zuowei Shen, Haizhao Yang +1Neural Network Optimization Nonparametric Regression
Department of Mathematics, University of Maryland, College Park, MD 20742-4015, USA. · Department of Mathematics, National University of Singapore, Singapore 119076. · Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China.
Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks
Sep 17, 2026Xianjun Li, Yunfei YangShallow Neural Networks ReLU Neural Networks
School of Statistics and Data Science, Capital University of Economics and Business, Beijing, China. · School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Zhuhai, China.
Physics Informed Random Feature Neural Networks for Solving PDEs
Sep 14, 2026Chi-An Chen, Chunyang Liao, Ming ZhongPDE Solving Random Feature Methods
Department of Applied Mathematics, Mathematics, Illinois Institute of Technology, Chicago, IL, 60616 · Department of Mathematical Sciences, University of Arkansas, Fayetteville, AR, 72701 · Department of Mathematics, University of Houston, Houston, TX, 77204
Approximating Smooth Functionals with ReLU Networks
Sep 14, 2026Shuhao JiaoReLU Neural Networks Neural Network Approximation Theory
Department of Biostatistics, City University of Hong Kong, Hong Kong SAR, China.
Draining Fictitious Knots: Restoring Distance-Awareness Guarantees for High-Dimensional Spline Networks
Sep 14, 2026Masoud Ataei, Mohammad Javad Khojasteh, Vikas DhimanUncertainty Quantification Kolmogorov-Arnold Networks
Electrical and Computer Engg. University of Maine Orono, ME, USA · Electrical and Microelectronic Engg. Rochester Institute of Technology Rochester, NY, USA
Exact ReLU realization of binary affine refinement iterates via reflection folding and cone switching
Sep 14, 2026Boldsaikhan Bolorkhuu, Tsogtgerel GantumurReLU Neural Networks Neural Network Approximation Theory
National University of Mongolia · McGill University · Institute of Mathematics and Digital Technology, Mongolian Academy of Sciences
Shallow neural network approximation in mixed Sobolev spaces
Sep 4, 2026Yuwen Li, Guozhi ZhangNeural Network Approximation Theory Shallow Neural Networks
School of Mathematical Sciences, Zhejiang University, 866 Yuhangtang Road, Hangzhou 310058, Zhejiang, China
Residual neural networks overcome the curse of dimensionality for semilinear heat equations
Sep 3, 2026Ilkhom Mukhammadiev, Diyora SalimovaNeural PDE Solvers PDE Solving
Department of Applied Mathematics, University of Freiburg, Germany
Spectral Convergence of Random Feature Method in Multiple Dimensions
Sep 3, 2026Pingbing Ming, Hao YuPDE Solving Random Feature Methods
A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations
Sep 2, 2026Ruiyang Hong, Hrad Ghoukasian, Anastasis KratsiosNeural Network Approximation Theory Statistical Learning Theory
RecKAN: Kolmogorov-Arnold Networks with a Learnable Recursive Polynomial Basis
Sep 1, 2026Amirhosein AzarpourPolynomial Neural Networks Kolmogorov-Arnold Networks
Department of Computer Science Shahid Beheshti University Tehran, Iran
Why Multi-Layer Message Passing Works: Completeness Theory for Graph Neural Network Interatomic Potentials
Sep 1, 2026Pingbing Ming, Han WangMachine Learning Interatomic Potentials GNN Expressivity
SKLMS, Institute of Computational Mathematics and Scientific/Engineering Computing, AMSS, Chinese Academy of Sciences, No. 55, East Road Zhong-Guan-Cun, Beijing 100190, China · National Key Laboratory of Computational Physics, Institute of Applied Physics and Computational Mathematics, Fenghao East Road 2, Beijing 100094, P.R. China, and HEDPS, CAPT, College of Engineering, Peking University, Beijing 100871, P.R. China
A convolutional framework for detecting event-driven dynamics in energy price series
Aug 31, 2026Caixia Xu, Piotr FryzlewiczTime Series Classification Neural Network Approximation Theory
1Shanghai University of Finance and Economics · 2London School of Economics
Sharp Approximation Rates for Neural Networks with Affine Latent Parameterizations
Aug 31, 2026Shijun ZhangHypernetworks Neural Network Approximation Theory
Department of Applied Mathematics Hong Kong Polytechnic University
Kolmogorov--Arnold against bounded translations
Aug 31, 2026Sviatoslav V. DzhenzherAdversarial Robustness Kolmogorov-Arnold Networks
Moscow Institute of Physics and Technology 141701, Institutskii lane, 9, Dolgoprudny, Russia.
Every Layer Counts: An Exponential $L_2$ Depth Hierarchy for ReLU Networks
Aug 24, 2026Itay SafranShallow Neural Networks ReLU Neural Networks
Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev
HYDRA: Hyperbolic Dynamic Representation Architecture for Kolmogorov-Arnold Networks
Aug 12, 2026Zhao Su, Yuxin Xia, Haoran Li +4Kolmogorov-Arnold Networks Neural Network Approximation Theory
School of Information Science and Engineering, Lanzhou University · Department of Data Science and AI, Monash University · School of Computing and Information Technology, University of Wollongong +1
Accelerated Learning of High Dimensional Functions with a Tensor-Featured Training Network
Aug 11, 2026Karl Pierce, Yuehaw Khoo, Haizhao YangNeural Network Optimization Neural Network Approximation Theory
† The University of Maryland, College Park, College Park MD 20742 USA · ∗The University of Chicago, Chicago, IL 60637 USA
Training-Free Universal Approximation by Prompting Random Transformers
Aug 10, 2026Alexander Hsu, Rongjie LaiTransformer Expressivity Prompt Learning
Department of Mathematics, Purdue University

Oct 6, 2026, math.NAJ/K move · Enter open · S save

FOSLS-deRhaNN: native de Rham neural classes for H(div) and H(curl) with applications to first-order system least-squares neural network methods for partial differential equations

Shun Zhang

Department of Mathematics, City University of Hong Kong, Kowloon Tong, Hong Kong, China

Neural PDE Solvers Neural Network Approximation Theory

Open paper AskPDF arXiv

We construct neural approximation classes native to the graph spaces H(div) and H(curl), in two and three dimensions and, for H(div), in any dimension. Every realization lies in the space for all parameter values, and with kinked potentials, such as ReLU networks, the admissible jumps appear at finite width. The classes are images of scalar and componentwise networks under fixed operators of the de Rham complex, and do not involve a mesh or finite element emulation. For H(div) in R^n two native classes are given on an equal footing, with a skew-symmetric potential

A

\mathrm{Div}\,A+R_nq+\mathbf{h}

, with the divergence

q

as an explicit unknown, and

\mathrm{Div}\,A+\mathbf{z}

with an

H^1

field

\mathbf{z}

; for H(curl) the analogous classes are

\mathrm{grad}\,φ+Sr+\mathbf{h}

in two dimensions and

\mathrm{grad}\,φ+\mathbf{z}

in two and three dimensions. In all of them every interface jump of the field is carried by the potential term,

\mathrm{Div}\,A

\mathrm{grad}\,φ

, while the remaining part has no interface jump (it is an

H^1

field in the regular-decomposition classes); the classes with

\mathbf{z}

are the componentwise approach enriched by this term. Known or learned interface geometry enters the potential through factors with trainable amplitudes, and the remaining part if the divergence jumps. The classes lead to the FOSLS-deRhaNN method, first-order system least squares with de Rham neural networks, whose loss is the least-squares functional posed in the natural spaces of the weak formulation; for elliptic equations this includes

H^{-1}

right-hand sides and

H^{1/2}

Dirichlet data. Elliptic equations with discontinuous coefficients and curl-curl problems are treated as instances, with the functional equivalent to the error; linear transport with discontinuous solutions and conservation laws with shocks use the same flux classes.

Momentum

Related topics

Latest papers 163