Uniform Approximation

Uniform approximation studies how well neural networks and other mathematical tools can represent functions across their entire domain, focusing on minimizing the maximum error (the "uniform" norm). Current research emphasizes determining the minimum network width or number of parameters needed for accurate approximation, particularly for shallow networks using ReLU and related activation functions, and exploring alternative methods like Random Vector Functional Link networks and Randomized Hadamard Transforms for efficient computation. These advancements improve our understanding of neural network expressivity and efficiency, impacting fields like machine learning through improved theoretical guarantees for algorithms and data structures.

Papers

September 19, 2023

Minimum width for universal approximation using ReLU networks on compact domain
Namjun Kim, Chanho Min, Sejun Park
ReLU Network Universal Approximation Universal Approximators Minimum Width Continuous Domain Uniform Approximation

July 28, 2023

Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networks
Jonathan W. Siegel
Shallow Neural Network Shallow ReLU Approximation Rate Hausdorff Distance Uniform Approximation

July 24, 2023

Rates of Approximation by ReLU Shallow Neural Networks
Tong Mao, Ding-Xuan Zhou
Average Approximation Function Approximation Shallow Neural Network Shallow Network Shallow ReLU Universal Rate Uniform Approximation

June 30, 2023

Efficient uniform approximation using Random Vector Functional Link networks
Palina Salanevich, Olov Schavemaker
Hidden Layer ReLU Activation Random Weight Random Vector Functional Link Uniform Approximation

March 3, 2022

Uniform Approximations for Randomized Hadamard Transforms with Applications
Yeshwanth Cherapanamjeri, Jelani Nelson
Financial Application Weight Matrix Hadamard Transform Uniform Approximation

Uniform Approximation

Papers

Minimum width for universal approximation using ReLU networks on compact domain

Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networks

Rates of Approximation by ReLU Shallow Neural Networks

Efficient uniform approximation using Random Vector Functional Link networks

Uniform Approximations for Randomized Hadamard Transforms with Applications