On shallow feedforward neural networks with inputs from a topological space

Vugar Ismailov

cs.LGApr 3, 2025

On shallow feedforward neural networks with inputs from a topological space

Authors: Vugar Ismailov

Organizations: Institute of Mathematics and Mechanics, Baku, Azerbaijan · Center for Mathematics and its Applications, Khazar University, Baku, Azerbaijan

Abstract

We study feedforward neural networks with inputs from a topological space (TFNNs). We prove a universal approximation theorem for shallow TFNNs, which demonstrates their capacity to approximate any continuous function defined on this topological space. As an application, we obtain an approximative version of Kolmogorov's superposition theorem for compact metric spaces.

Feedforward Neural Networks

Feedforward neural networks map inputs through layered computations to make predictions, and are studied for learning, expressivity, efficiency, and reliability.

See all 54 papers

Approximation Theory for Neural Networks: Old and New
May 20, 2026Soumendu Sundar Mukherjee, Himasish TalukdarUniversal Approximation Neural Network Approximation Theory
Statistics and Mathematics Unit, Indian Statistical Institute, Kolkata
Topological DeepONets and a generalization of the Chen-Chen operator approximation theorem
Mar 12, 2026Vugar IsmailovNeural Network Approximation Theory Deep Operator Networks
Distributionally robust approximation property of neural networks
Oct 10, 2025Mihriban Ceylan, David J. PrömelNeural Network Approximation Theory Universal Approximation

May 20, 2026J/K move · Enter open · S save

Approximation Theory for Neural Networks: Old and New

Soumendu Sundar Mukherjee, Himasish Talukdar

Statistics and Mathematics Unit, Indian Statistical Institute, Kolkata

Universal Approximation Neural Network Approximation Theory Feedforward Neural Networks Neural Network Expressivity Function Approximation

Open paper AskPDF arXiv

Universal approximation theorems provide a mathematical explanation for the expressive power of neural networks. They assert that, under mild conditions on the activation function, feedforward neural networks are dense in broad function classes, such as continuous functions on compact subsets of

\mathbb{R}^d

L^p

spaces, or Sobolev spaces. Over the past four decades, these qualitative universality results have evolved into a rich quantitative theory addressing approximation rates, parameter efficiency, and the role of architectural features such as depth and width. This survey presents several glimpses into this theory. We review classical density results for single-hidden-layer networks, as well as quantitative bounds that relate approximation error to network size and smoothness assumptions on target functions. Particular emphasis is placed on depth--width trade-offs and on results demonstrating that deeper architectures can achieve superior parameter efficiency for structured function classes. In addition to standard feedforward neural networks, we also review recent developments on Kolmogorov--Arnold Networks (KANs), which offer an alternative architectural paradigm and whose approximation-theoretic properties have begun to attract significant theoretical attention.

Abstract

Feedforward Neural Networks

Explore similar work

Approximation Theory for Neural Networks: Old and New

Topological DeepONets and a generalization of the Chen-Chen operator approximation theorem

Distributionally robust approximation property of neural networks