Multi-Column RBF Neural Network Using Adaptive and Non-Adaptive Particle Swarm Optimization
Authors: Ammar Hoori, Yuichi Motai
Organizations: Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA · Department of Electrical and Computer Engineering, Virginia Commonwealth University, Richmond, VA 23284, USA
Abstract
The radial basis function neural network (RBFN) trained with a gradient descending algorithm provides an effective fully connected structure in both shallow and deep networks. The error correction (ErrCor), a state-of-the-art gradient-based training method, selects optimal hidden units to improve accuracy. Alternatively, as a population-based algorithm, the particle swarm optimization algorithm (PSO) uses the swarm experience to optimize RBFN parameters, offering global search and robustness to local minima. Adaptive PSO (APSO) has emerged as an improved variant of PSO. APSO algorithm improves convergence speed by dynamically adjusting swarm parameters during optimization. Both ErrCor and PSO demonstrate improved results and competitive convergence. However, with large datasets, these methods face scalability challenges such as excessive kernel computations and large hidden layer structures. A recent multi-column RBFN approach (MCRN) improves ErrCor performance by deploying small RBFNs in a parallel system. Inspired by MCRN's success, we propose two novel approaches to improve PSO performance: the multi-column RBFN with PSO (MC-PSO) and the multi-column RBFN with APSO (MC-APSO). These methods introduce parallel RBFN structures trained using evolutionary swarm methods. Each RBFN is independently trained on a specific spatial subset of the dataset using either PSO or APSO algorithms. These resulting specialist-trained RBFNs are tailored to their respective subsets. During testing, only selected RBFNs, where the test instance neighbors are located, contribute to the multi-column output. This specialization improves accuracy, while parallelism enhances speed. We evaluate the proposed methods on various benchmark datasets. The MC-PSO and MC-APSO outperform ErrCor, PSO, APSO, and MCRN in terms of accuracy and recall. They also demonstrate faster training and testing times in most experiments.
Kolmogorov-Arnold Networks (KANs) approximate multivariate functions using learnable univariate edge functions, typically parameterized by B-spline bases. Although effective, spline-based implementations can be computationally expensive. A modified version of KANs, called FastKAN, improves efficiency by replacing splines with Gaussian radial basis functions (RBFs), but it relies on a fixed kernel and shape parameter. In this work, we extend the RBF-based KAN framework by introducing a broader family of radial basis kernels and by initializing the kernel shape parameter using leave-one-out cross-validation (LOOCV). To the best of our knowledge, this is the first study that integrates LOOCV-based kernel scale estimation with deep KAN training. We also introduce Matérn and Wendland kernels into the KAN framework for the first time, enabling more flexible basis representations beyond the Gaussian kernel used in FastKAN. The LOOCV estimate provides a data-driven initialization of the kernel scale, which is subsequently refined during network training. The proposed adaptive RBF-KAN is evaluated on several two-dimensional benchmark functions. The results highlight the importance of kernel selection and adaptive shape parameters, with different kernels showing advantages for smooth functions, discontinuities, and oscillatory patterns. Overall, combining LOOCV-based initialization with adaptive kernel learning provides a practical strategy for improving RBF-based KAN models.
Roberto Cavoretto, Alessandra De Rossi, Adeeba Haider +1
Particle swarm optimization (PSO) is a widely used metaheuristic, prized for its simplicity and small parameter set. Although decades of research have produced numerous PSO variants that improve performance by modifying key components (e.g., parameter schedules, swarm topologies, or updating rules), two fundamental challenges persist. First, most existing approaches are problem-specific and hand-crafted, leading to poor cross-task generalization and forcing practitioners to navigate an impractically large design space, which also hinders systematic reuse of prior effective mechanisms. Second, mainstream implementations remain CPU-bound, constraining scalability and substantially increasing computational cost in real-world applications. To address these challenges, we propose {AutoPSO}, a highly automated metaframework for constructing customized PSO algorithms. AutoPSO formulates PSO-based optimization as a bi-level process: an outer search explores the joint space of effective PSO components, while an inner loop instantiates candidate variants to solve the target task and provide feedback. The outer search operates over a curated, open-design component pool, supporting flexible replacement of the component set and the outer optimizer. Crucially, by leveraging EvoX for population tensorization and batched evaluations, AutoPSO can efficiently assess thousands of particles within practical time budgets. Comprehensive experiments on numerical benchmarks and neuroevolution robotic control tasks demonstrate that AutoPSO consistently discovers novel PSO variants that significantly outperform strong baselines. Ablation and scalability studies further highlight the contribution of individual algorithmic components and confirm that AutoPSO achieves increasing performance gains with larger swarm sizes. Code is available at {https://github.com/EMI-Group/autopso}.
Particle swarm optimization (PSO) has been widely applied to solve complex optimization problems from real-world applications due to its efficient exploration of large solution spaces and the ability to converge towards optimal solutions without requiring gradient information. Common swarm topologies in standard PSO and its variants, e.g., Ring and Star, can be regarded as graphs, where each edge connects only two particles. Such topology structures allow direct interactions only between connected particle pairs, and thus often fail to directly capture the higher-order social relationships that are necessary for navigating complex search landscapes. Therefore, this article proposes a novel PSO variant termed Hypergraph-assisted Particle Swarm Optimization (HPSO). In HPSO, the topology of the particles in a swarm is modeled by a hypergraph, in which hyperedges are used to connect multiple particles. This allows multiple particles within a hyperedge to interact directly. Furthermore, an adaptive hypergraph updating strategy is designed to periodically reconstruct the topology based on cumulative average particle displacement, thereby maintaining swarm diversity throughout the evolutionary process. In the experiments, the effectiveness of HPSO is verified on the IEEE CEC'17 benchmark suite, and the results demonstrate that HPSO achieves promising performance across various types of functions. Furthermore, the ablation experiment demonstrates that HPSO has excellent search capabilities.