cs.LGFeb 18, 2026
SaveFEKAN: Feature-Enriched Kolmogorov-Arnold Networks
Organizations: Aerospace Engineering Department, Worcester Polytechnic Institute, Worcester, MA 01609, USA.
Abstract
Kolmogorov-Arnold Networks (KANs) have recently emerged as a compelling alternative to multilayer perceptrons, offering enhanced interpretability via functional decomposition. However, existing KAN architectures, including spline-, wavelet-, radial-basis variants, etc., suffer from high computational cost and slow convergence, limiting scalability and practical applicability. Here, we introduce Feature-Enriched Kolmogorov-Arnold Networks (FEKAN), a simple yet effective extension that preserves all the advantages of KAN while improving computational efficiency and predictive accuracy through feature enrichment, without increasing the number of trainable parameters. By incorporating these additional features, FEKAN accelerates convergence, increases representation capacity, and substantially mitigates the computational overhead characteristic of state-of-the-art KAN architectures. We investigate FEKAN across a comprehensive set of benchmarks, including function-approximation tasks, physics-informed formulations for diverse partial differential equations (PDEs), and neural operator settings that map between input and output function spaces. For function approximation, we systematically compare FEKAN against a broad family of KAN variants, FastKAN, WavKAN, ReLUKAN, HRKAN, ChebyshevKAN, RBFKAN, and the original SplineKAN. Across all tasks, FEKAN demonstrates substantially faster convergence and consistently higher approximation accuracy than the underlying baseline architectures. We also establish the theoretical foundations for FEKAN, showing its superior representation capacity compared to KAN, which contributes to improved accuracy and efficiency.
Figures & tables
Figure 1: Schematic representation of KAN vs. FEKAN architectures. In FEKAN, is the input feature map, .
Figure 2: Schematic representation of feature and enriched feature space.
Figure 3: Evolution of the NTK eigenvalue spectra during training for KAN and FEKAN in the function approximation task across different basis functions.
Figure 4: NTK eigenvalue spectra at for KAN and FEKAN in the function approximation task using spline and Fourier basis functions. It shows faster decay of eigenvalues for KAN while a comparatively slower decay for FEKAN.
Figure 5: FEKAN applied to diverse problems, including function approximation, partial differential equations, and operator learning tasks.
Figure 6: Spline Basis: (Left) Scaling law of KAN and FEKAN with increasing grid size using the spline basis and (Center) Computational cost as a function of grid size for the spline basis. (Right) Convergence of KAN and FEKAN on a high-frequency test function using the spline basis.
| Grid size ( ) | KAN | FEKAN | ||
| Rel. Err. | Time (sec/iter) | Rel. Err. | Time (sec/iter) | |
| 10 | 0.01430 0.00004 | 0.04874 | 0.00741 0.00098 | 0.05080 |
| 15 | 0.01458 0.00061 | 0.05142 | 0.00171 0.00018 | 0.05178 |
| 20 | 0.01378 0.00032 | 0.05448 | 0.00100 0.00007 | 0.05348 |
Table 1: Spline Basis Training Performance: Comparison of accuracy and computational time for KAN and FEKAN using the spline basis with fixed polynomial order across varying grid sizes . FEKAN incorporates a Fourier feature map consisting of orthogonal and pairs with varying frequencies .
Figure 7: Fourier Basis: (Left) Scaling behavior of KAN and FEKAN with increasing number of terms using the Fourier basis and (Center) Computational cost as a function of for the Fourier basis. (Right) Convergence of KAN and FEKAN on a high-frequency test function using the Fourier basis.
| Number of terms ( ) | KAN | FEKAN | ||
| Rel. Err. | Time (sec/iter) | Rel. Err. | Time (sec/iter) | |
| 10 | 0.01605 0.00010 | 0.02828 | 0.01148 0.00055 | 0.03028 |
| 50 | 0.01324 0.00044 | 0.03456 | 0.00082 0.00065 | 0.0366 |
| 100 | 0.00160 0.00013 | 0.03928 | 0.00454 0.00251 | 0.05472 |
Table 2: Fourier Basis Training Performance: Accuracy and computational time of KAN and FEKAN using the Fourier basis with varying number of terms . FEKAN is augmented with a Fourier feature map of orthogonal and pairs at multiple frequencies .
Figure 8: Radial Basis: (Left) Scaling behavior of KAN and FEKAN with increasing number of terms using the RBF basis and (Center) Computational cost as a function of for the RBF. (Right) Convergence of KAN and FEKAN on a high-frequency test function using the Fourier basis.
| Number of terms ( ) | KAN | FEKAN | ||
|---|---|---|---|---|
| Rel. Err. | Time (sec/iter) | Rel. Err. | Time (sec/iter) | |
| 50 | 0.03283 0.00507 | 0.03052 | 0.01554 0.00035 | 0.03352 |
| 100 | 0.03802 0.00840 | 0.03682 | 0.01589 0.00076 | 0.03340 |
| 200 | 0.04761 0.00982 | 0.03480 | 0.01507 0.00013 | 0.03634 |
Table 3: RBF Training Performance: Comparison of accuracy and computational time for KAN and FEKAN using the RBF basis with varying number of terms . FEKAN is enhanced with a Fourier feature map consisting of orthogonal and pairs at different frequencies .
| Architecture | Basis | Parameters | Rel. Err. | Time (sec/iter) | High fr. | Disc. | Low fr. |
| KAN | Spline | 0.01468 0.00072 | 0.0514 | ✗ | ✓ | ✓ | |
| FEKAN | Spline | 0.00175 0.00026 | 0.0517 | ✓ | ✓ | ✓ | |
| KAN | Fourier | 0.01344 0.00041 | 0.0345 | ✗ | ✓ | ✓ | |
| FEKAN | Fourier | 0.00103 0.00069 | 0.0366 | ✓ | ✓ | ✓ | |
| ChebyKAN [ 49 ] | Chebyshev | NaN | 0.0264 | ✗ | ✗ | ✗ | |
| FE-ChebyKAN | Chebyshev | 0.01475 0.00035 | 0.0264 | ✗ | ✓ | ✓ |
Table 4: Comparison of accuracy and computational time for KAN and FEKAN across different basis functions with architecture . FEKAN includes a Fourier feature map of orthogonal and pairs. High fr. , Disc. , and Low fr. indicate whether the model captures high-frequency, discontinuous, or low-frequency features, respectively. ( ✓ ) = fully learned, ( ✓ ) = partially learned, ( ✗ ) = not learned. See Appendix C for details.
Figure 9: Relative error for KAN across different basis functions as in Table 4 , evaluated over 10 random seeds. For the Chebyshev basis, KAN diverged in all 10 runs, while FEKAN diverged in 4/10 runs. Despite the instability of Chebyshev training, FEKAN stabilizes learning, achieving better accuracy and faster convergence. For Chebyshev, training stops at divergence, and uncertainty quantification is performed using the earliest divergence to maintain consistent dimensions.
Figure 10: (Left) Solutions of the Lorenz system with parameters , , for multiple random initial conditions. (Center) Approximation for an unseen initial condition using KAN, and (Right) FEKAN, both with the spline basis of polynomial order and grid size .
Figure 11: Convergence of the Lorenz system using KAN and FEKAN. KAN is trained without feature enrichment, while FEKAN uses feature enrichment with polynomial order and grid size .
Figure 12: Absolute error for the Helmholtz equation using the spline basis: (Left) PI-KAN, (Center) PI-FEKAN with 5 terms, and (Right) PI-FEKAN with 7 terms.
| Architecture | Feature Enrichment | Rel. Err. | Time (sec/iter) |
|---|---|---|---|
| PI-KAN | None | 0.27160 0.06727 | 0.09199 |
| PI-FEKAN | 5 terms | 0.24710 0.10925 | 0.09563 |
| PI-FEKAN | 7 terms | 0.12358 0.05403 | 0.11219 |
Table 5: Helmholtz Equation Training Performance: Accuracy and computational time for PI-KAN and PI-FEKAN using the spline basis with polynomial order and grid size . The architecture has configuration , where is the number of input features. PI-FEKAN is enriched with a Fourier feature map of orthogonal and pairs at varying frequencies .
Figure 13: Convergence for the Helmholtz equation using PI-KAN, PI-FEKAN with 5 feature enrichment terms, and PI-FEKAN with 7 feature enrichment terms.
Figure 14: Absolute error for the Helmholtz equation: (Left) PI-KAN with Chebyshev [ 49 ] , (Center) PI-FEKAN with modified Chebyshev [ 47 ] , and (Right) PI-FEKAN with Chebyshev.
| Architecture | Feature Enrichment | Rel. Err. | Time (sec/iter) |
|---|---|---|---|
| PI-KAN (Chebyshev [ 49 ] ) | None | 0.70608 0.19506 | 0.03714 |
| PI-KAN (Mod. Chebyshev [ 47 ] ) | None | 0.55833 0.26008 | 0.09798 |
| PI-FEKAN (Chebyshev) | 7 terms | 0.22368 0.43223 | 0.03475 |
Table 6: Helmholtz Equation Training Performance: Accuracy and computational time for PI-KAN and PI-FEKAN using the Chebyshev basis with polynomial order . The architecture has configuration , where is the number of input features. PI-FEKAN is enriched with a Fourier feature map of orthogonal and pairs at varying frequencies .
Figure 15: Convergence for the Helmholtz equation using PI-KAN with Chebyshev [ 49 ] , PI-FEKAN with modified Chebyshev [ 47 ] , and PI-FEKAN with Chebyshev.
| Architecture | Basis | Parameters | Rel. Err. | Time (sec/iter) |
| PI-KAN | Spline | 0.26230 0.10690 | 0.0977 | |
| PI-FEKAN | Spline | 0.08246 0.00989 | 0.0787 | |
| PI-KAN | Fourier | 0.09442 0.05181 | 0.0899 | |
| PI-FEKAN | Fourier | 0.08001 0.03074 | 0.1176 | |
| PI-KAN | Chebyshev | 0.70856 0.19506 | 0.0360 | |
| PI-FEKAN | Chebyshev | 0.48404 0.43223 | 0.0275 |
Table 7: Comparison of accuracy and computational time for PI-KAN and PI-FEKAN across different basis functions with architecture . PI-FEKAN is enriched with a Fourier feature map of 7 orthogonal and pairs per input dimension, giving for 2D Helmholtz, compared to for PI-KAN. See Appendix D for training details.
Figure 16: Relative error for KAN across different basis functions as in Table 7 , evaluated over 5 random seeds. ReLU and HReLU bases were trained for 50,000 epochs with early stopping upon convergence, while Wavelet basis was stopped at 50,000 epochs due to poor generalization of both KAN and FEKAN using DoG.
Figure 17: (Left) Time per iteration for PI-KAN (blue columns) and PI-FEKAN using the ReLU basis. (Right) Relative error for PI-KAN (blue columns) and PI-FEKAN with ReLU basis, polynomial order , and varying grid size .
Figure 18: Relative error for KAN and FEKAN for (Left) spline and (Right) Chebyshev basis functions, evaluated over 5 random seeds. Note that all FEKAN variants have 7 feature enrichment terms per dimension of the PDE for fair comparison.
| Architecture | Basis | Std. Dev. | Relative Error | Time (sec/iter) |
|---|---|---|---|---|
| PI-KAN | Spline | - | 0.26230 0.10690 | 0.0977 |
| PI-FEKAN | - | 0.08246 0.00989 | 0.0787 | |
| PI-FEKAN | 2 | 0.01826 0.01108 | 0.1067 | |
| PI-FEKAN | 10 | 0.77561 0.36992 | 0.1059 | |
| PI-KAN | Chebyshev | - | 0.70856 0.19506 | 0.0360 |
| PI-FEKAN | - | 0.21538 0.32079 | 0.0275 |
Table 8: Performance on Helmholtz Equation with Random Fourier Features: Accuracy and computational time for KAN and FEKAN using the spline basis with polynomial order and grid size . The Chebyshev basis uses polynomial order . The architecture has configuration , where is the number of input features. FEKAN is enriched with a Fourier feature map of orthogonal and pairs at varying frequencies . Here, is sampled from an isotropic Gaussian distribution, (0, ) for the stochastic case. More details on the comparison of the absolute error can be found in Appendix D .
Figure 19: (Left) Solution of the Allen-Cahn equation. (Center) Absolute error using PI-KAN, and (Right) PI-FEKAN.
| Architecture | Coll. | Relative Error | Time (sec/iter) |
|---|---|---|---|
| PI-KAN | 6000 | 0.10593 0.02699 | 0.02358 |
| PI-FEKAN | 0.07587 0.01765 | 0.02124 | |
| PI-KAN | 10000 | 0.11256 0.02732 | 0.04514 |
| PI-FEKAN | 0.06196 0.00611 | 0.04978 | |
| PI-KAN | 15000 | 0.10434 0.02991 | 0.03040 |
| PI-FEKAN | 0.06165 0.01132 | 0.05586 |
Table 9: Performance on Allen-Cahn Equation: Accuracy and computational time for PI-KAN and PI-FEKAN using the spline basis with polynomial order and grid size . The architecture has configuration , where is the number of input features. PI-FEKAN is enriched with a Fourier feature map of orthogonal and pairs at varying frequencies .
Figure 20: Convergence for the Allen-Cahn equation with and without feature enrichment using the spline basis. The number of collocation points is varied to evaluate the consistency of performance improvement of PI-FEKAN over PI-KAN.
Figure 21: The boundary value problem on domain is solved in four independent phases, with boundary data fed in batches for each Phase- .
Figure 22: Accuracy of (Top) PI-KAN and (Bottom) PI-FEKAN in satisfying the boundary conditions ( ) for the Helmholtz equation at different training phases using the spline basis with . The exact boundary conditions are shown in black dashed lines.
Figure 23: Accuracy of (Top) PI-KAN and (Bottom) PI-FEKAN in satisfying the boundary conditions ( ) for the Helmholtz equation at different training phases using the spline basis with . The exact boundary conditions are shown in black dashed lines.
Figure 24: Solution of the Helmholtz equation for PI-KAN and PI-FEKAN using the spline basis: (Top) and (Bottom) at Phase-4 of boundary data introduction. Phase-wise training plots are provided in Appendix E .
Figure 25: Absolute error for the states in the Lorenz equation using PI-KAN and PI-FEKAN: (Top) Spline basis, (Bottom) Chebyshev basis.
Figure 26: Relative error for the states in the Lorenz equation using PI-KAN and PI-FEKAN: (Top) Spline basis, (Bottom) Chebyshev basis.
Figure 27: (Left) 3D Helmholtz equation with , , . (Right: Top) Absolute error using the spline basis, and (Right: Bottom) Chebyshev basis with and without feature enrichment.
| Architecture | Basis | Relative Error | Time (sec/iter) |
|---|---|---|---|
| SPI-KAN | Spline | 0.19133 0.14544 | 0.01556 |
| SPI-FEKAN | 0.06078 0.01636 | 0.0159 | |
| SPI-KAN | Chebyshev | 0.58661 0.21128 | 0.01074 |
| SPI-FEKAN | 0.07231 0.02187 | 0.01338 |
Table 10: Performance on 3D Helmholtz Equation: Accuracy and computational time for KAN and FEKAN using the spline basis with polynomial order and grid size . The Chebyshev basis uses polynomial order . The architecture has configuration , where is the number of input features. FEKAN is enriched with a Fourier feature map of orthogonal and pairs at varying frequencies .
Figure 28: Relative error for the 3D Helmholtz equation using KAN and FEKAN with spline and Chebyshev basis functions, averaged over 10 random seeds.
| Architecture | Basis | Relative Error | Time (sec/iter) |
|---|---|---|---|
| SPI-KAN | Spline | 0.15162 0.10661 | 0.01312 |
| SPI-FEKAN | 0.00231 0.00133 | 0.01332 | |
| SPI-KAN | Chebyshev | NaN | 0.0106 |
| SPI-FEKAN | 0.00401 0.00050 | 0.01026 |
Table 11: Performance on Klein-Gordon Equation: Accuracy and computational time for KAN and FEKAN using the spline basis with polynomial order and grid size . The Chebyshev basis uses polynomial order . The architecture has configuration , where is the number of input features. FEKAN is enriched with a Fourier feature map of orthogonal and pairs at varying frequencies .
Figure 29: (Left) Klein-Gordon equation. (Right: Top) Absolute error using the spline basis, and (Right: Bottom) Chebyshev basis with and without feature enrichment.
Figure 30: Relative error for the Klein-Gordon equation using KAN and FEKAN with spline and Chebyshev basis functions, averaged over 10 random seeds.
Figure 31: Relative error for the high-frequency bubble dynamics learned using two-step DeepOKAN and two-step DeepOFEKAN with spline basis for trunk network and RBF for branch network.
Figure 32: Absolute error for two-step DeepOKAN and two-step DeepOFEKAN at pressure (p) = Pa and frequency = 2 MHz.
Figure 33: Should I use FEKAN or KAN?
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 34: Absolute error for a high-frequency test function using KAN and FEKAN with the spline basis ( , ).
Figure 35: Absolute error for a high-frequency test function using KAN and FEKAN with the Fourier basis ( ).
Figure 36: Absolute error for a high-frequency test function using FEKAN with Chebyshev polynomials ( ). Without feature enrichment, KAN diverged to NaN before completing training.
Figure 37: Absolute error for a high-frequency test function using KAN and FEKAN with the RBF basis ( ).
Figure 38: Absolute error for a high-frequency test function using KAN and FEKAN with the ReLU basis ( , , ).
Figure 39: Absolute error for a high-frequency test function using KAN and FEKAN with the HReLU basis ( , , ).
Figure 40: Absolute error for a high-frequency test function using KAN and FEKAN with the wavelet basis in the form of a Derivative of Gaussian (DoG).
Figure 41: Absolute error for the Helmholtz equation: (Left) PI-KAN and (Right) PI-FEKAN using (a) spline, (b) RBF, (c) Fourier, (d) Chebyshev, (e) ReLU, (f) HReLU, and (g) wavelet bases.
Figure 42: Absolute error for the Helmholtz equation: (From Left to Right) PI-KAN, PI-FEKAN, PI-FEKAN ( ), PI-FEKAN ( ) using spline basis function. Here, is the variance of the isotropic Gaussian distribution ( ) from which the frequencies for the feature enrichment terms were sampled.
Figure 43: Absolute error for the Helmholtz equation: (From Left to Right) PI-KAN, PI-FEKAN, PI-FEKAN ( ), PI-FEKAN ( ) using Chebyshev basis function. Here, is the variance of the isotropic Gaussian distribution ( ) from which the frequencies for the feature enrichment terms were sampled.
Figure 44: Solution of the Helmholtz equation for (Top) PI-KAN and (Bottom) PI-FEKAN using the spline basis ( ) at different phases of boundary data introduction.
Figure 45: Solution of the Helmholtz equation for (Top) PI-KAN and (Bottom) PI-FEKAN using the spline basis ( ) at different phases of boundary data introduction.