Neural Network Approximation

Neural network approximation focuses on using neural networks to efficiently and accurately represent complex functions, addressing limitations of traditional methods in various fields. Current research emphasizes improving approximation accuracy and efficiency for diverse applications, including solving partial differential equations, accelerating reinforcement learning algorithms, and approximating computationally expensive operations within other machine learning models, often employing techniques like low-rank approximations and specialized neural architectures tailored to specific function classes. These advancements have significant implications for accelerating scientific computations, enhancing the performance of machine learning systems, and enabling the application of complex models to previously intractable problems.

Papers

October 21, 2024

ReLU neural network approximation to piecewise constant functions
Zhiqiang Cai, Junpyo Choi, Min Liu
ReLU Network ReLU Layer Neural Network Approximation Layer ReLU Network Approximation Property Discontinuity Discrimination Constant Measure

August 29, 2024

Addressing Common Misinterpretations of KART and UAT in Neural Network Literature
Vugar Ismailov
Neural Network Kolmogorov Arnold Network Universal Approximation Neural Network Approximation Novel Deep Learning Framework

April 18, 2024

Neural Active Learning Beyond Bandits
Yikun Ban, Ishika Agarwal, Ziwei Wu, Yada Zhu, Kommy Weldemariam, Hanghang Tong, Jingrui He
Active Learning Neural Network Approximation Pool Based Active Learning Neural Active Learning

December 19, 2023

Neural Network Approximation for Pessimistic Offline Reinforcement Learning
Di Wu, Yuling Jiao, Li Shen, Haizhao Yang, Xiliang Lu
Deep Reinforcement Learning Offline Reinforcement Learning Neural Network Approximation

November 7, 2023

Enhanced physics-informed neural networks with domain scaling and residual correction methods for multi-frequency elliptic problems
Deok-Kyu Jang, Hyea Hyun Kim, Kyungsoo Kim
Physic Informed Neural Network Residual Model Elliptic Partial Differential Equation Neural Network Approximation Uniform Scaling Multi Frequency

September 27, 2023

Learning the Efficient Frontier
Philippe Chatigny, Ivan Sergienko, Ryan Ferguson, Jordan Weir, Maxime Bergeron
LeArning Abstract Neural Network Approximation Optimal Portfolio Efficient Frontier Input Optimization

August 1, 2023

Neural approximation of Wasserstein distance via a universal architecture for symmetric and factorwise group invariant functions
Samantha Chen, Yusu Wang
Wasserstein Distance Distance Function Neural Network Approximation Invariant Function Group Wise Symmetric Function

July 30, 2023

On Neural Network approximation of ideal adversarial attack and convergence of adversarial training
Rajdeep Haldar, Qifan Song
Adversarial Attack Adversarial Training Early Stage Convergence Adversarial Loss Learned Function Neural Network Approximation Optimal Attack

July 28, 2023

Weighted variation spaces and approximation by shallow ReLU networks
Ronald DeVore, Robert D. Nowak, Rahul Parhi, Jonathan W. Siegel
Average Approximation Shallow ReLU ReLU Neural Network Neural Network Approximation Sparse Approximation Variation Space

June 27, 2023

LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network
Yuchen Su, Zhineng Chen, Zhiwen Shao, Yuning Du, Zhilong Ji, Jinfeng Bai, Yong Zhou, Yu-Gang Jiang
Scene Text Detection Text Shape Neural Network Approximation Arbitrary Shape Text Detection

March 28, 2023

Function Approximation with Randomly Initialized Neural Networks for Approximate Model Reference Adaptive Control
Tyler Lekang, Andrew Lamperski
Neural Network Activation Function Function Approximation Random Participation Neural Network Approximation Model Reference Adaptive Integral Regression

January 16, 2023

On Using Deep Learning Proxies as Forward Models in Deep Learning Problems
Fatima Albreiki, Nidhal Belayouni, Deepak K. Gupta
Deep Learning Optimization Problem Particle Swarm Optimization Numerical Experiment Forward Model Neural Network Approximation Learning Proxy

October 21, 2022

Neural Network Approximations of PDEs Beyond Linearity: A Representational Perspective
Tanya Marwah, Zachary C. Lipton, Jianfeng Lu, Andrej Risteski
High Dimensional Partial Differential Equation Elliptic Partial Differential Equation Neural Network Approximation Human Representation

September 3, 2022

From Monte Carlo to neural networks approximations of boundary value problems
Lucian Beznea, Iulian Cimpean, Oana Lupascu-Stamate, Ionel Popescu, Arghir Zarnescu
Monte Carlo Neural Network Approximation Approximation Error Value Problem Poisson Equation

August 28, 2022

Neural Network Approximation of Continuous Functions in High Dimensions with Applications to Inverse Problems
Santhosh Karnik, Rongrong Wang, Mark Iwen
Neural Network Financial Application Inverse Problem High Dimension Low Complexity High Dimensional Space Continuous Space Neural Network Approximation

June 15, 2022

Variable Bitrate Neural Fields
Towaki Takikawa, Alex Evans, Jonathan Tremblay, Thomas Müller, Morgan McGuire, Alec Jacobson, Sanja Fidler
Neural Field Discrete Representation Neural Network Approximation Feature Grid

June 9, 2022

Overcoming the Spectral Bias of Neural Value Approximation
Ge Yang, Anurag Ajay, Pulkit Agrawal
Neural Tangent Kernel Policy Reinforcement Learning Kernel Regression Spectral Bias Neural Network Approximation Policy Deep Reinforcement Learning

May 28, 2022

Approximate Conditional Coverage & Calibration via Neural Model Approximations
Allen Schmaltz, Danielle Rasooly
Conformal Prediction Calibration Performance Neural Network Approximation Selective Classification Conformal Predictor Conditional Coverage Conformal P Value

May 25, 2022

An Experimental Comparison Between Temporal Difference and Residual Gradient with Neural Network Approximation
Shuyu Yin, Tao Luo, Peilin Liu, Zhi-Qin John Xu
Reinforcement Learning Gradient Descent Deep Q Learning Temporal Difference Neural Network Approximation Experimental Comparison Residual Information

May 23, 2022

Generic bounds on the approximation error for physics-informed (and) operator learning
Tim De Ryck, Siddhartha Mishra
Physic Informed Neural Network Operator Learning Physic Informed Neural Network Approximation Social Generic Approximation Error Physic Informed Operator Learning

Neural Network Approximation

Papers

ReLU neural network approximation to piecewise constant functions

Addressing Common Misinterpretations of KART and UAT in Neural Network Literature

Neural Active Learning Beyond Bandits

Neural Network Approximation for Pessimistic Offline Reinforcement Learning

Enhanced physics-informed neural networks with domain scaling and residual correction methods for multi-frequency elliptic problems

Learning the Efficient Frontier

Neural approximation of Wasserstein distance via a universal architecture for symmetric and factorwise group invariant functions

On Neural Network approximation of ideal adversarial attack and convergence of adversarial training

Weighted variation spaces and approximation by shallow ReLU networks

LRANet: Towards Accurate and Efficient Scene Text Detection with Low-Rank Approximation Network

Function Approximation with Randomly Initialized Neural Networks for Approximate Model Reference Adaptive Control

On Using Deep Learning Proxies as Forward Models in Deep Learning Problems

Neural Network Approximations of PDEs Beyond Linearity: A Representational Perspective

From Monte Carlo to neural networks approximations of boundary value problems

Neural Network Approximation of Continuous Functions in High Dimensions with Applications to Inverse Problems

Variable Bitrate Neural Fields

Overcoming the Spectral Bias of Neural Value Approximation

Approximate Conditional Coverage & Calibration via Neural Model Approximations

An Experimental Comparison Between Temporal Difference and Residual Gradient with Neural Network Approximation

Generic bounds on the approximation error for physics-informed (and) operator learning