Universal Approximation

Universal approximation theory explores the ability of neural networks to approximate any continuous function to arbitrary accuracy. Current research focuses on refining approximation bounds for various network architectures (including feedforward, recurrent, and transformer networks), investigating the impact of parameter constraints (e.g., bounded weights, quantization), and extending the theory to encompass broader input spaces (e.g., topological vector spaces, non-metric spaces) and operator learning. These advancements provide a stronger theoretical foundation for deep learning, informing model design, optimization strategies, and ultimately improving the reliability and efficiency of applications across diverse fields.

Papers

November 1, 2022

Recurrent Neural Networks and Universal Approximation of Bayesian Filters
Adrian N. Bishop, Edwin V. Bonilla
Recurrent Neural Network Universal Approximation Bayesian Filtering Temporal Latent Optimal Filter

October 2, 2022

Deep Invertible Approximation of Topologically Rich Maps between Manifolds
Michael Puthawala, Matti Lassas, Ivan Dokmanic, Pekka Pankka, Maarten de Hoop
Topological Data Analysis Persistent Homology Class Manifold Universal Approximation Universal Approximators Topological Mapping Topological Signature

September 23, 2022

Achieve the Minimum Width of Neural Networks for Universal Approximation
Yongqiang Cai
Neural Network Universal Approximation Wide Neural Network \Ell_p$ Norm Universal Approximators Minimum Width Approximation Power

September 6, 2022

Extending the Universal Approximation Theorem for a Broad Class of Hypercomplex-Valued Neural Networks
Wington L. Vital, Guilherme Vieira, Marcos Eduardo Valle
Universal Approximation Parameterized Hypercomplex Hypercomplex Algebra

August 18, 2022

Deep Neural Network Approximation of Invariant Functions through Dynamical Systems
Qianxiao Li, Ting Lin, Zuowei Shen
Dynamical System Universal Approximation Deep Neural Network Approximation Roto Translation Invariant Function Symmetric Function

August 10, 2022

ATLAS: Universal Function Approximator for Memory Retention
Heinrich van Deventer, Anna Bosman
Continual LEArning Catastrophic Forgetting Universal Approximation Atlas Reliability Map

May 26, 2022

April 24, 2022

Do ReLU Networks Have An Edge When Approximating Compactly-Supported Functions?
Anastasis Kratsios, Behnoosh Zamanlooy
Extreme Edge Average Approximation Simple Function ReLU Network Universal Approximation

February 11, 2022

Deep Signatures -- Learning Invariants of Planar Curves
Roy Velich, Ron Kimmel
Universal Approximation Dynamical Invariant Aggregated Curve Signature Analysis Planar Curve Learning Non Linear Invariant

January 31, 2022

Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer
Beatrice Acciaio, Anastasis Kratsios, Gudmund Pammer
Universal Approximation Statistical Manifold Geometric Structure Deep Causal Causal Deep Learning

December 30, 2021

December 16, 2021

Approximation of functions with one-bit neural networks
C. Sinan Güntürk, Weilin Li
Average Approximation Simple Function Universal Approximation Approximation Property Image Derivative ReLU Regression

December 2, 2021

Is Approximation Universally Defensive Against Adversarial Attacks in Deep Neural Networks?
Ayesha Siddique, Khaza Anuarul Hoque
Deep Neural Network Adversarial Attack Adversarial Robustness Universal Approximation Approximate Computing Approximate Multiplier Approximate DNN Accelerator