math.NTFeb 10, 2026
SaveThe minimal width of universal -adic ReLU neural networks
Abstract
We determine the minimal width of -adic neural networks with the universal approximation property for continuous -valued functions on compact open subsets with respect to the norms and the norm, where the activation function is a natural -adic analogue of the ReLU function.
More in this topic
Neural Network Approximation Theory
Neural network approximation theory studies how network size, depth, activations, and structure determine the accuracy of functions and operators.
See all 261 papersWe propose a new frame work of -adic neural network. Unlike the original -adic neural network by S.\ Albeverio, A.\ Khrennikov, and B.\ Tirrozi using a family of characteristic functions indexed by hyperparameters of precision as activation functions, we use a single injective -adic character on the topological Abelian group of -adic integers as an activation function. We prove the -adic universal approximation theorem for this formulation of -adic neural network, and reduce it to the feasibility problem of polynomial equations over the finite ring of integers modulo a power of .
The minimal width of neural networks and immersions of real projective space
We investigate the minimal width of deep neural networks that can approximate all continuous functions from to in the topology of uniform convergence on compact subsets. We show that whenever the real projective space admits a smooth immersion into , the minimal width is at least , assuming that the activation function is continuous and monotone. As a consequence, the minimal width of ReLU networks is whenever is equal to 2, 3, 4, 7 or 8 and , extending results of Hanin-Sellke and Hwang.
Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One
In this paper, we study the universal approximation property of residual neural networks, and obtain some new results. For input and output dimensions and , and LeakyReLU, ReLU, ReLU-like activation functions, the upper and lower bounds of the minimum block width are established. To achieve approximation on any compact domain, we show that the exact minimum block width is when each residual branch has inner width 1. Furthermore, we show that residual neural networks with block width can achieve uniform approximation on any compact domain under the constraint that each residual branch has inner width 1. Besides, for any activation function family, we prove that there exist functions that cannot be approximated by residual neural networks with block width less than , both in the sense and the uniform sense, regardless of inner width.