Organizations: Graduate School of Engineering Science, Yokohama National University, Yokohama 240-8501, Japan · Digital Healthcare Research Department, Hitachi, Ltd., Tokyo 185-8601, Japan · Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China · Faculty of Engineering, Yokohama National University, Yokohama 240-8501, Japan
As the input dimension n grows, rule-based machine learning, such as Learning Classifier Systems (LCSs), faces a fundamental scalability bottleneck for function approximation: both rule count and parameter count grow exponentially with n. Traditional LCSs partition the n-dimensional input space directly, requiring O(mn) rules for adequate coverage, where m is the per-variable resolution. This article breaks from this paradigm by reorganizing rules dimension-wise, guided by the Kolmogorov-Arnold representation theorem: any continuous n-dimensional function can be expressed as a finite superposition of one-dimensional functions. The proposed Kolmogorov-Arnold Classifier System (KACS) decomposes the target function into one-dimensional subproblems and assigns a dedicated ruleset to each, reducing the worst-case rule count from O(mn) to O(mn2) and replacing n-dimensional local models with one-dimensional models requiring only two parameters per rule, independent of n. We also provide the first constructive proof that an LCS, namely KACS, is a universal approximator for continuous functions on compact domains. Evaluated against a direct n-dimensional input space partitioning approach under otherwise identical conditions, KACS achieves competitive accuracy in many settings while using only 2% to 40% of the parameters. Our implementation is available at https://github.com/YNU-NakataLab/KACS.
We introduce Clifford Kolmogorov-Arnold Network (ClKAN), a flexible and efficient architecture for function approximation in arbitrary Clifford Algebra spaces. We propose the use of Randomized Quasi-Monte Carlo grid generation as a solution to the exponential scaling associated with higher-dimensional algebras. Our ClKAN also introduces new batch normalization strategies to deal with variable domain input. ClKAN finds application in scientific discovery and engineering, and is validated in synthetic and physics-inspired tasks.
Matthias Wolff, Francesco Alesiani, Christof Duhme +1
Data Science Div. NEC Labs Europe Heidelberg, Germany
Several machine learning models are defined for inputs of any size, such as graphs with different numbers of nodes and point clouds containing varying numbers of points. The universality properties of such any-dimensional models remain poorly understood, as universality is traditionally studied for models accepting inputs of a fixed size, defined on a compact subset of their domain. In sharp contrast, any-dimensional models can be viewed as sequences of functions defined on growing-sized inputs, and it is not clear in which sense they can be universal. We develop a systematic approach to establish any-dimensional universality, by identifying any-dimensional functions with a unique function taking inputs in a suitable infinite-dimensional limit space containing inputs of all finite sizes as well as their limits. Using the symmetries of these inputs and relations between inputs of different sizes, we show that this limit space admits a natural topology with rich families of compact sets on which any-dimensional universality can be established. We illustrate our approach by showing that several existing architectures fail to be universal, and we propose simple modifications that restore universality.
Shengtai Yao, Eitan Levin, Mateo Díaz
Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD 21218, USA. · Department of Computing and Mathematical Sciences, Caltech, Pasadena, CA 91125, USA.
We analyze the universal approximation property of Kolmogorov-Arnold Networks (KANs) in terms of their edge functions. If these functions are all affine, then universality clearly fails. How many non-affine functions are needed, in addition to affine ones, to ensure universality? We show that a single one suffices. More precisely, we prove that deep KANs in which all edge functions are either affine or equal to a fixed continuous function σ are dense in C(K) for every compact set K⊂Rn if and only if σ is non-affine. In contrast, for KANs with exactly two hidden layers, universality holds if and only if σ is nonpolynomial. We further show that the full class of affine functions is not required; it can be replaced by a finite set without affecting universality. In particular, in the nonpolynomial case, a fixed family of five affine functions suffices when the depth is arbitrary. More generally, for every continuous non-affine function σ, there exists a finite affine family Aσ such that deep KANs with edge functions in Aσ∪{σ} remain universal. We also prove that KANs with the spline-based edge parameterization introduced by Liu et al.~\cite{Liu2024} are universal approximators in the classical sense, even when the spline degree and knot sequence are fixed in advance.