Neural Network Approximation Theory

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 15% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 163

All topics

Optimization of randomized neural networks for transfer operator approximation
May 22, 2026Mohammad Tabish, Stefan KlusDynamical Systems Neural Network Optimization
Maxwell Institute for Mathematical Sciences, University of Edinburgh and Heriot–Watt University, Edinburgh, UK · School of Mathematical & Computer Sciences, Heriot–Watt University, Edinburgh, UK
Any-Dimensional Invariant Universality
May 22, 2026Shengtai Yao, Eitan Levin, Mateo DíazUniversal Approximation Neural Network Approximation Theory
Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA. · Department of Computing and Mathematical Sciences, Caltech, Pasadena, CA 91125, USA.
Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning
May 21, 2026Adrien Weihs, Hayden SchaefferNeural Network Approximation Theory Operator Learning
Department of Mathematics, University of California Los Angeles, Los Angeles, CA 90095, USA.
Holographic functions and neural networks
May 21, 2026Balazs SzegedyNeural Network Approximation Theory Neural Network Expressivity
Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximations
May 21, 2026Shuang Chen, Juncai He, Xue-Cheng TaiNeural Network Approximation Theory Neural Operators
Qiuzhen College, Tsinghua University, Haidian District, Beijing 100084, China · Yau Mathematical Sciences Center, Tsinghua University, Haidian District, Beijing 100084, China · Norwegian Research Center, Fantoftvegen 38, 5072 Bergen, Norway
Decision-Aware Quadratic ReLU Replacement for HE-Friendly Inference
May 21, 2026Rui Li, Wenyuan Wu, Weijie MiaoEfficient Neural Network Inference Homomorphic Encryption
Chongqing Key Laboratory of Secure Computing for Biology, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, 266 Fangzheng Avenue, Beibei District, Chongqing 400714, China · Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, China
Approximation Theory for Neural Networks: Old and New
May 20, 2026Soumendu Sundar Mukherjee, Himasish TalukdarUniversal Approximation Neural Network Approximation Theory
Statistics and Mathematics Unit, Indian Statistical Institute, Kolkata
Fast Reconstruction of Exact Maxwell Dynamics from Sparse Data
May 19, 2026Dan DeGenaro, Xin Li, Obed Amo +4Neural PDE Solvers Neural Network Approximation Theory
Department of Computer Science, Georgetown University · Department of Mathematics, Georgetown University · Department of Mathematics and Statistics, Old Dominion University +2
Dual-Channel Tensor Neural Networks: Finite-Sample Theory and Conformal Structure Selection
May 18, 2026Elynn Chen, Jiayu Li, Zheshi Zheng +1Tensor Networks Uncertainty Quantification
New York University · University of Michigan · Duke University
Shallow ReLU $^s$ Networks in $L^p$ -Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization
May 18, 2026Weizhao Li, Fanghui Liu, Lei ShiShallow Neural Networks Nonparametric Regression
School of Mathematical Sciences Fudan University Shanghai, China · School of Mathematical Sciences Institute of Natural Sciences and MOE-LSC2026 Shanghai Jiao Tong University Shanghai, China · School of Mathematical Sciences Shanghai Key Laboratory for Contemporary Applied Mathematics25 Fudan University Shanghai, China
Function graph transformers universally approximate operators between function spaces
May 18, 2026Takashi Furuya, David Mis, Ivan Dokmanić +2Transformer Attention Neural Network Approximation Theory
Doshisha University, RIKEN AIP · Rice University · University of Basel +2
InfoFlow: A Framework for Multi-Layer Transformer Analysis
May 18, 2026Penghao Yu, Haotian Jiang, Zeyu Bao +1Transformer Expressivity Transformer
Department of Mathematics National University of Singapore · Institute for Functional Intelligent Materials National University of Singapore · National University of Singapore +1
Universal Approximation of Nonlinear Operators and Their Derivatives
May 14, 2026Filippo de FeoNeural Network Approximation Theory Operator Learning
Institut für Mathematik, Technische Universität Berlin.
Wahkon: A Statistically Principled Deep RKHS Superposition Network
May 13, 2026Yongkai Chen, Wenxuan Zhong, Ping MaReproducing Kernel Hilbert Spaces Kernel Methods
Department of Statistics, Harvard University · Department of Statistics, University of Georgia
Approximation of Maximally Monotone Operators : A Graph Convergence Perspective
May 12, 2026Takashi Furuya, Yury Korolev, Takaharu YaguchiNeural Network Approximation Theory Operator Learning
Doshisha University, RIKEN AIP · Department of Mathematical Sciences, University of Bath, UK · Kyushu University, RIKEN AIP
Lower bounds for one-layer transformers that compute parity
May 12, 2026Daniel HsuTransformer Expressivity Self-Attention
Approximation Theory of Laplacian-Based Neural Operators for Reaction-Diffusion System
May 12, 2026Takashi Furuya, Ryo Ozawa, Jenn-Nan WangPDE Operator Learning Neural Network Approximation Theory
Doshisha University · RIKEN AIP · Tohoku University +1
Posterior Contraction Rates for Sparse Kolmogorov-Arnold Networks in Anisotropic Besov Spaces
May 12, 2026Jeunghun Oh, Kyeongwon Lee, Jaeyong Lee +1Bayesian Neural Networks Kolmogorov-Arnold Networks
Department of Statistics, Seoul National University · Department of Statistics, Inha University · Department of Mathematics, University of Maryland
Exact Fixed-Point Constraints in Neural-ODEs with Provable Universality
May 11, 2026Feliciano Giuseppe Pacifico, Duccio Fanelli, Lorenzo Buffoni +3Constrained Optimization Neural Network Approximation Theory
Department of Informatics and Computer Science, University of Pisa, Italy · Department of Physics and Astronomy and INFN, University of Florence, Sesto Fiorentino, Italy
Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defects
May 10, 2026Kevin Dao, Jose Israel RodriguezPolynomial Neural Networks Neural Network Approximation Theory
Department of Mathematics, University of Wisconsin-Madison, Wisconsin, USA.
Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity
May 9, 2026Zhongjie Shi, Wenjing LiaoNeural Network Generalization Transformer
School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332, United States
Structure-Preserving Reconstruction of Convex Lipschitz Functionals on Hilbert Spaces from Finite Samples
May 8, 2026Anastasis KratsiosInput Convex Neural Networks Neural Network Approximation Theory
A Deep Risk Estimator for Known Operator Learning
May 8, 2026Andreas Maier, Md Hasan, Paulina Conrad +1Neural Network Generalization Neural Network Approximation Theory
Pattern Recognition Lab · Friedrich-Alexander-Universität Erlangen-Nürnberg · Erlangen, Germany
Embedding Dimension Lower Bounds for Universality of Deep Sets and Janossy Pooling
May 8, 2026Ali Syed, Aditya Nambiar, Jonathan W. SiegelRepresentation Learning Deep Sets
Department of Mathematics Texas A&M University College Station, TX 77843
Approximation Error Upper and Lower Bounds for Hölder Class with Transformers
May 8, 2026Xin He, Yuling Jiao, Xiliang Lu +1Transformer Neural Network Approximation Theory
School of Mathematics and Statistics, Wuhan University, Wuhan, China · School of Artificial Intelligence, Wuhan University, Wuhan, China · Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan, China
Region Seeding via Pre-Activation Regularization: A Geometric View of Piecewise Affine Neural Networks
May 7, 2026Yi Wei, Xuan Qi, Furao ShenNeural Network Activation Functions Neural Network Approximation Theory
DITEN University of Genoa, Genoa, Italy · State Key Laboratory of Novel Software Technology School of Artificial Intelligence Nanjing University, Jiangsu, China
ConquerNet: Convolution-Smoothed Quantile ReLU Neural Networks with Minimax Guarantees
May 7, 2026Tianpai Luo, Fangwei Wu, Weichi WuQuantile Regression Minimax Estimation
Department of Statistics and Data Science Tsinghua University Beijing, 100084, China
AffineLens: Capturing the Continuous Piecewise Affine Functions of Neural Networks
May 7, 2026Yi Wei, Xuan Qi, Furao Shen +3Neural Network Interpretability Neural Network Approximation Theory
State Key Laboratory of Novel Software Technology School of Intelligence Science and Technology Nanjing University, Jiangsu, China · AI for Good Istituto Italiano di Tecnologia, Genoa, Italy · DITEN University of Genoa, Genoa, Italy +2
Structural Correspondence and Universal Approximation in Diagonal plus Low-Rank Neural Networks
May 7, 2026Ying Chen, Aoxi Li, Jihun Kim +1Low-Rank Matrix Decomposition Low-Rank Approximation
Department of Industrial Engineering & Operations Research University of California, Berkeley, CA 94720
Adaptivity Under Realizability Constraints: Comparing In-Context and Agentic Learning
May 6, 2026Anastasis Kratsios, A. Martina Neuman, Philipp PetersenReLU Neural Networks In-Context Learning
McMaster University and the Vector Institute, Main St., Hamilton, L8S 4L8, Ontario, Canada · University of Vienna, Faculty of Mathematics, Kolingasse 14-16, 1090 Wien, Austria

May 22, 2026, cs.LGJ/K move · Enter open · S save

Optimization of randomized neural networks for transfer operator approximation

Mohammad Tabish, Stefan Klus

Maxwell Institute for Mathematical Sciences, University of Edinburgh and Heriot–Watt University, Edinburgh, UK · School of Mathematical & Computer Sciences, Heriot–Watt University, Edinburgh, UK

Dynamical Systems Neural Network Optimization Neural Network Activation Functions Neural Network Approximation Theory Operator Learning

Open paper AskPDF arXiv

RaNNDy is a randomized neural network architecture for the data-driven approximation of transfer operators associated with complex dynamical systems. The weights and biases of the hidden layers of the network are randomly initialized and kept fixed, only the output layer is trained. This has several advantages over fully optimized neural networks, notably a closed-form solution for the output layer and significantly lower training costs. Despite these advantages, RaNNDy is restricted to the initial selection of weights and biases that parametrize the basis functions required for the operator approximation. Since the basis functions are determined by the activation function, choosing an appropriate activation function for the hidden layers is crucial. In this work, we propose an algorithm that optimizes the activation function itself, while keeping the weights and biases in the randomized neural network fixed, providing a more suitable dictionary. We illustrate the efficacy of the approach using various benchmark problems, including stochastic differential equations and random walks on graphons.

Momentum

Related topics

Latest papers 163