Approximation Capability

Approximation capability research investigates the ability of neural networks to accurately represent complex functions, focusing on limitations imposed by bounded parameters and architectural choices. Current research explores the approximation power of various network architectures, including convolutional and multilayer perceptrons, examining the impact of activation functions, network depth and width, and the role of noise in stochastic models. Understanding these capabilities is crucial for optimizing neural network design, improving efficiency, and establishing theoretical foundations for their widespread applications in diverse fields like generative modeling and signal processing.

Papers

September 25, 2024

Numerical Approximation Capacity of Neural Networks with Bounded Parameters: Do Limits Exist, and How Can They Be Measured?
Li Liu, Tengchao Yu, Heng Yong
Neural Network Continuum Limit Universal Approximation Norm Bounded Approximation Power Approximation Capability

July 17, 2024

A Survey on Universal Approximation Theorems
Midhun T Augustine
Neural Network Timely Survey Universal Approximation Function Approximation Theorem Driven Question Approximation Capability

July 1, 2024

Bridging Smoothness and Approximation: Theoretical Insights into Over-Smoothing in Graph Neural Networks
Guangrui Yang, Jianfei Li, Ming Li, Han Feng, Ding-Xuan Zhou
Graph Neural Network Average Approximation Theoretical Analysis Undisciplined Over Smoothing Average Smoothness Approximation Theory Approximation Property Approximation Capability

December 29, 2023

Bespoke Approximation of Multiplication-Accumulation and Activation Targeting Printed Multilayer Perceptrons
Florentia Afentaki, Gurol Saglam, Argyris Kokkinis, Kostas Siozios, Georgios Zervakis, Mehdi B Tahoori
Multi Layer Perceptron Biological Circuit Energy Efficient Neural Network Multiply Accumulate Approximation Capability

December 20, 2023

Sampling Complexity of Deep Approximation Spaces
Ahmed Abdeljawad, Philipp Grohs
Neural Network Sample Complexity ReLU Activation Approximation Capability Information Complexity Approximation Space

December 13, 2023

Noise in the reverse process improves the approximation capabilities of diffusion models
Karthik Elamvazhuthi, Samet Oymak, Fabio Pasqualetti
Diffusion Model Industrial Disturbing Noise Neural Ordinary Differential Equation Neural Stochastic Differential Equation Approximation Capability Reverse Process

August 7, 2023

Developability Approximation for Neural Implicits through Rank Minimization
Pratheba Selvaraju
Low Rank Gaussian Curvature Neural Implicit Surface Implicit Surface 3D Surface Approximation Capability

January 11, 2023

Exploring the Approximation Capabilities of Multiplicative Neural Networks for Smooth Functions
Ido Ben-Shaul, Tomer Galanti, Shai Dekel
Deep Neural Network Multi Layer Smooth Function Approximation Capability Multiplicative Neural Network Band Limited Stochastic Gate

October 19, 2022

SignReLU neural network and its approximation ability
Jianfei Li, Han Feng, Ding-Xuan Zhou
Neural Network Deep Neural Network Activation Function ReLU Network Approximation Capability Better Approximation

September 27, 2022

Continuous approximation by convolutional neural networks with a sigmoidal function
Weike Chang
Neural Network Convolutional Neural Network Function Approximation Sigmoid Function Approximation Capability

July 16, 2022

Approximation Capabilities of Neural Networks using Morphological Perceptrons and Generalizations
William Chang, Hassan Hamad, Keith M. Chugg
Neural Network Strong Generalization Standard Deep Nonlinear Activation Artificial Neural Network Approximation Capability Morphological Neural Network

January 11, 2022

Uniform Approximation with Quadratic Neural Networks
Ahmed Abdeljawad
Neural Network Average Approximation Deep Neural Network Approximation Approximation Capability H\"older Function

Approximation Capability

Papers

Numerical Approximation Capacity of Neural Networks with Bounded Parameters: Do Limits Exist, and How Can They Be Measured?

A Survey on Universal Approximation Theorems

Bridging Smoothness and Approximation: Theoretical Insights into Over-Smoothing in Graph Neural Networks

Bespoke Approximation of Multiplication-Accumulation and Activation Targeting Printed Multilayer Perceptrons

Sampling Complexity of Deep Approximation Spaces

Noise in the reverse process improves the approximation capabilities of diffusion models

Developability Approximation for Neural Implicits through Rank Minimization

Exploring the Approximation Capabilities of Multiplicative Neural Networks for Smooth Functions

SignReLU neural network and its approximation ability

Continuous approximation by convolutional neural networks with a sigmoidal function

Approximation Capabilities of Neural Networks using Morphological Perceptrons and Generalizations

Uniform Approximation with Quadratic Neural Networks