Approximation Property

Approximation theory investigates how well different mathematical models, particularly neural networks, can represent complex functions. Current research focuses on optimizing network architectures (e.g., varying depth, width, and activation functions like ReLU and its variants) to achieve efficient and accurate approximations, particularly in high-dimensional spaces and for functions with specific smoothness properties. This work is crucial for advancing the theoretical understanding of neural networks and improving their performance in diverse applications, including adaptive control, image processing, and machine learning more broadly. Key challenges involve mitigating issues like over-smoothing in graph neural networks and developing approximation guarantees for networks trained via gradient descent.

Papers

October 21, 2024

ReLU neural network approximation to piecewise constant functions
Zhiqiang Cai, Junpyo Choi, Min Liu
ReLU Network ReLU Layer Neural Network Approximation Layer ReLU Network Approximation Property Discontinuity Discrimination Constant Measure

September 4, 2024

Optimal Neural Network Approximation for High-Dimensional Continuous Functions
Ayan Maiti, Michelle Michelle, Haizhao Yang
High Dimensional Approximation Property Intrinsic Neural Field

July 1, 2024

Bridging Smoothness and Approximation: Theoretical Insights into Over-Smoothing in Graph Neural Networks
Guangrui Yang, Jianfei Li, Ming Li, Han Feng, Ding-Xuan Zhou
Graph Neural Network Average Approximation Theoretical Analysis Undisciplined Over Smoothing Average Smoothness Approximation Theory Approximation Property Approximation Capability

May 8, 2024

Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators
Tony Lindeberg
Direct Convolution Gaussian Kernel Scale Space Approximation Property

March 25, 2024

Approximation with Random Shallow ReLU Networks with Applications to Model Reference Adaptive Control
Andrew Lamperski, Tyler Lekang
Neural Network Financial Application Average Approximation Adaptive Control Shallow ReLU Approximation Guarantee Approximation Property Model Reference Adaptive

December 27, 2023

Expressivity and Approximation Properties of Deep Neural Networks with ReLU$^k$ Activation
Juncai He, Tong Mao, Jinchao Xu
Deep Neural Network ReLU Activation Behavior Expressivity Style Deep ReLU Network Approximation Property

November 8, 2023

Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Norms
Yahong Yang, Yulong Lu
Deep Neural Network Average Approximation Simple Function Approximation Rate Approximation Property Deep Neural Network Approximation Sobolev Norm Function Approximator

October 16, 2023

Approximation properties of slice-matching operators
Shiying Li, Caroline Moosmueller
Approximation Property Slice Matching

September 9, 2023

Approximation Results for Gradient Descent trained Neural Networks
G. Welper
Neural Network Gradient Descent Gradient Flow Neural Tangent Kernel Smoothness Assumption Approximation Property

May 26, 2023

A Tale of Two Approximations: Tightening Over-Approximation for DNN Robustness Verification via Under-Approximation
Zhiyi Xue, Si Liu, Zhaodi Zhang, Yiting Wu, Min Zhang
Average Approximation Robustness Guarantee Cautionary TALE Approximation Property Tight Approximation

April 4, 2023

Measure theoretic results for approximation by neural networks with limited weights
Vugar Ismailov, Ekrem Savas
Neural Network Average Approximation Balancing Weight Continuous Space Sufficient Condition Approximation Property

November 21, 2022

DualApp: Tight Over-Approximation for Neural Network Robustness Verification via Under-Approximation
Yiting Wu, Zhaodi Zhang, Zhiyi Xue, Si Liu, Min Zhang
Robustness Guarantee Approximation Property Tight Approximation

September 17, 2022

Approximation results for Gradient Descent trained Shallow Neural Networks in $1d$
R. Gentile, G. Welper
Gradient Descent Non Convex Optimization Shallow Neural Network Shallow Network Approximation Property Width Network

September 7, 2022

On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learning
Washim Uddin Mondal, Vaneet Aggarwal, Satish V. Ukkusuri
Multi Agent Reinforcement Learning Optimal Policy Action Space Near Optimality Individual Agent Local Level Approximation Property

July 28, 2022

On the Multidimensional Random Subset Sum Problem
Luca Becchetti, Arthur Carvalho Walraven da Cunha, Andrea Clementi, Francesco d'Amore, Hicham Lesfari, Emanuele Natale, Luca Trevisan
O$ Sample Approximation Property Dimensional Random Vector

April 10, 2022

Expressiveness and Approximation Properties of Graph Neural Networks
Floris Geerts, Juan L. Reutter
Graph Neural Network GNN Architecture Expressive Speech Graph Learning Task Approximation Property Specific GNN

December 16, 2021

Approximation of functions with one-bit neural networks
C. Sinan Güntürk, Weilin Li
Average Approximation Simple Function Universal Approximation Approximation Property Image Derivative ReLU Regression

Approximation Property

Papers

ReLU neural network approximation to piecewise constant functions

Optimal Neural Network Approximation for High-Dimensional Continuous Functions

Bridging Smoothness and Approximation: Theoretical Insights into Over-Smoothing in Graph Neural Networks

Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators

Approximation with Random Shallow ReLU Networks with Applications to Model Reference Adaptive Control

Expressivity and Approximation Properties of Deep Neural Networks with ReLU$^k$ Activation

Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Norms

Approximation properties of slice-matching operators

Approximation Results for Gradient Descent trained Neural Networks

A Tale of Two Approximations: Tightening Over-Approximation for DNN Robustness Verification via Under-Approximation

Measure theoretic results for approximation by neural networks with limited weights

DualApp: Tight Over-Approximation for Neural Network Robustness Verification via Under-Approximation

Approximation results for Gradient Descent trained Shallow Neural Networks in $1d$

On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learning

On the Multidimensional Random Subset Sum Problem

Expressiveness and Approximation Properties of Graph Neural Networks

Approximation of functions with one-bit neural networks